What Is Google LaMDA & How Does LaMDA work ? | CodewithHarsh

What Is Google LaMDA & How Does LaMDA work ?

Google works with LaMDA, BERT and MUM to help machines better understand user intent.

Creating the Google language model is nothing new; In fact, Google enlists the likes of LaMDA, BERT and MUM as a way for machines to better understand user intent.

Google has spent many years researching language-based models with the hope of training models that can conduct a practical and logical conversation on essentially any topic.

Google LaMDA

What Is Google LaMDA with CodewithHarsh?

LaMDA, which stands for Language Model for Dialog Application, was created to enable software to better engage in a fluid and natural conversation.

LaMDA is based on a transformer architecture similar to other language models such as BERT and GPT-3.

However, because of their training, LMDAs can understand subtle questions and conversations covering many different topics.

As with other models, due to the open nature of the conversation, you may end up talking about something completely different even though you initially focused on a single topic.

This behavior can easily confuse most conversational models and chatbots.

During last year's Google I/O announcement, we saw that LaMDA was built to address these issues.

The demonstration proved how the model could interact naturally on a randomly given subject.

Despite the stream of loose questions, the conversation remained on track, which was worth watching.

Chatbot

How Does LaMDA work with CodewithHarsh?

LaMDA was built on Google's open-source neural network, Transformer, which is used to understand natural language.

A text model is trained to recognize correlations between words, and to predict the words likely to occur next in a sentence.

It does this by studying datasets containing dialogue, rather than just individual words.
While a conversational AI system is similar to chatbot software, there are some important differences between the two.

For example, a chatbot is trained on a limited, specific dataset and can only conduct limited conversations based on the data and the exact questions on which it is trained.

On the other hand, because LaMDA is trained on many different datasets, it can have open-ended conversations.

During the training process, it picks up on the nuances of open-ended dialogue and customization.

It can answer questions on many different topics depending on the flow of the conversation.

Therefore, it enables conversations that are more akin to human interaction than chatbots can often provide.

How are LamDAs trained with CodewithHarsh?

Google explained that LaMDA has a two-stage training process, which includes pre-training and fine-tuning.

In total, the model has been trained on 1.56 trillion words with 137 billion parameters.

Pre-training 

For the pre-training phase, the Google team created a dataset of 1.56T words from several public web documents.

This dataset is then tokenized (converted into a string of characters to form a sentence) into 2.81T tokens, on which the model is initially trained.

During pre-training, the model uses general and scalable parallelization to predict the next part of the conversation based on the previous token.

Fine Tuning

The LaMDA is trained to perform the generation and classification tasks during the fine-tuning phase.

Essentially, the LaMDA generator, which predicts the next part of the dialogue, generates a number of contextual responses based on back-and-forth interactions.

The LaMDA classifier will then predict the safety and quality scores for each possible response.

Any responses with a low security score are filtered out before the top-scoring response is selected to continue the conversation.

Scores are based on safety, sensitivity, specificity and interest percentage.

LaMDA Key Objectives and Metrics with CodewithHarsh?

Three main objectives of the model are defined to guide the training of the model.

These are quality, safety and groundedness.

Quality:-

It is based on three human rater dimensions:-

• Sensitivity.
• Speciality
• Interest.

Quality scores are used to ensure that feedback makes sense in the context it is used, that it is specific to the question asked, and is considered insightful enough to create better dialogue.

Security

To ensure safety, the model adheres to the standards of responsible AI. A set of security objectives are used to capture and review the behavior of the model.

This ensures that the output does not provide any unintended feedback and avoids any bias.

Groundedness

Groundedness is defined as "the percentage of responses containing claims about the outside world".

Evaluation

Through an ongoing process of measuring progress, responses from pre-trained models, fine-tuned models and human raters are reviewed, to evaluate responses against the above mentioned quality, safety and baseline metrics.

The researchers have so far been able to determine the following:-

Quality metrics improve with the number of parameters.
Security improves with fine-tuning.
As the size of the model increases, the groundedness improves.

How would LaMDA  used? with CodewithHarsh 

While still a work in progress with no final release date, it is predicted that LaMDA will be used in the future to improve customer experience and enable chatbots to provide more human-like interactions.

In addition, using LaMDA to navigate search in Google's search engine is a real possibility.

Implications of LaMDA for SEO with CodewithHarsh ?

By focusing on language and conversational models, Google provides insight into their vision for the future of search and sheds light on changes in the way they develop their products.

This ultimately means that search behavior and the way users search for products or information may well change.

Google is constantly working on improving users' understanding of search intent to ensure that they get the most useful and relevant results in SERPs.

The LaMDA model will, undoubtedly, be an important tool to understand the questions being asked by the explorers.

All this further highlights the need to ensure that content is optimized for humans rather than search engines.

LaMDA for SEO

Making sure that the content is conversational and written with your target audience in mind means that as Google progresses, the content can continue to perform well.

It is also important to regularly refresh evergreen content to ensure that it evolves over time and remains relevant.

In a paper titled Rethinking Search: Making Experts Out of Detente, Google's research engineers share how they envision AI advancements such as LaMDA will further enhance "search as interaction with experts."

He shared an example built around the search question, "What are the health benefits and risks of red wine?"

Currently, Google will display an answer box list of bullet points as an answer to this question.

However, they suggest that in the future, a response could well be a paragraph explaining the benefits and risks of red wine with a link to the source information.

Therefore, ensuring that content is backed up by expert sources Google LaMDA will be more important than ever to generate search results in the future.

What's next for Google LaMDA with CodewithHarsh ?

Google is clear that open-ended dialogue models like LaMDA have benefits and risks and is committed to improving security and infrastructure to ensure a more reliable and fair experience.

Training LaMDA models on various data, including images or videos, is another thing we can look at in the future.

This opens up the ability to navigate the web even more using conversational prompts.

Google CEO Sundar Pichai said of LaMDA, "We believe LaMDA's conversational capabilities have the potential to make information and computing fundamentally more accessible and easier to use."

Although the rollout date is yet to be confirmed, there is no doubt that models like LaMDA will be the future of Google.