Google sees billions of searches each day and 15% of them are rare queries. This brings the need to return results for these non-anticipated queries. You may not know how to spell something or the right words. This may be because you don’t have sufficient knowledge of the topic. Now, search involves figuring out what you are searching for. With machine learning taking significant leaps in the last few years, there’s a quantum jump in how Google understands queries.
Google has introduced Bidirectional Encoder Representations from Transformers or BERT. This is an open-sourced neural network-based technique for natural language processing.
Google focuses on processing words which are in relation to other words in the sentence, instead of the one-by-one order. Google BERT Algorithm takes into account the full context of a word as it looks at words with coming before and after it. This helps understand the intent behind the google search query. The focus on hardware has led to Cloud TPUs which serve search results. The BERT model is applied to featured snippets and rankings in search to find the information you seek.
Conversational queries have words like ‘for’ and ‘to’ which mean a lot vis-a-vis the meaning. Google Bert Algorithm helps understand the context of the search in the query. This helps search in a natural way.
There are plenty of words and content out there. However, words can be problematic, ambiguous and synonymous. Google BERT Algorithm solves ambiguous phrases and sentences which have a lot of words with multiple meanings.
The meaning of a word changes depending on the surrounding sentence. The word ‘like’ can be used as a noun, verb or adjective. The word ‘like’ has no meaning by itself. It depends on the meaning of the words that surround it. Understanding context is easy for humans, but tough for machines. This is where Google BERT Algorithm comes in.
Bi-Directional:
Language models like Skip-gram and a continuous bag of words were uni-directional. They could move either to the left to right or right to left of the target word. This is a movement in a single direction, but not both at the same time. Google BERT Algorithm uses the bi-directional model which is a first of its kind. Google BERT sees the whole sentence on either side of the word. Contextual language modeling is done on almost all words at once.
Encoder Representations:
What gets encoded is automatically decoded. Just like an in-and-out mechanism.
Transformers:
You are pronouns where its easy to lose track of who is being spoken about in a conversation. Why machines, even humans struggle to understand this. Search engines struggle when you say he, she, they, we, it and so on. Transformers focus on pronouns and all word meanings which go with it. It helps understand who is being spoken to or what is being spoken about. Masked language basically stops the target word from seeing itself. When the mask is on Google BERT Algorithm simply guesses the missing word.
Let’s understand the Google BERT algorithm with an example. A popular search is ‘2020 German traveler to the USA’. The word ‘to’ is very important as it signifies the relationship to the other words in the query. It shows Germans traveling to the US and not the other way around. In the past Google Algorithms never understood the importance of this connection. (Context). It showed US citizens traveling to Germany. Google BERT algorithm understands the significance of the word ‘to’ and easily provides relevant results to the query.
You have the words bank account and bank. In a context-free environment, the word bank is the same as a bank account or even the bank of a river. Contextual models add a whole new dimension to the search. You have the sentenced Peter accessed the bank account. In a unidirectional contextual model, the word bank pops up as ‘I accessed the’, there’s no word on the account. However, with BERT you have Peter accessed the …. Bank account. This gives an accurate picture of the context of the search.
Let’s say you are a blogger who uses many SEO strategies to get the website ranking higher. However, your content may not be what the user seeks and the blog traffic drops. This is when you must change the SEO strategy.
You have a blog post on how to build a dining table. Based on the BERT update you have, how to build a dining table out of Rosewood. This improves the chances of getting a higher ranking.
Conclusion
BERT seeks to eliminate and reduce 'keyword-ese'. It’s the awkward language and phrasing used to help Google understand what you are trying to say. BERT wants your search to be human. The focus is on nuances and context which gives human reader-friendly content.