There’s a lot that goes into building a search engine. While it seems simple enough—you type a question, and we find results that match—we have taken on many deep, technical challenges to make Google what it is today.
And the reality is that Search is never a solved problem. Old challenges evolve, and new challenges are constantly popping up because the internet and the world are always changing.
Here are some of the biggest technological breakthroughs we have made through the years— and where we’re continuing to push the boundaries of innovation—as we build and improve Google Search.
Delivering quality results
It’s important that we rank information not only based on what’s relevant to your query but also what’s likely to be helpful and reliable.
This insight was what set Google apart from day one: Our PageRank algorithm did not just take into account not just whether words on a page matched. It looked at how sites linked to one another as a clue about which pages were important or authoritative.
We have adapted our techniques over the years as the web has evolved and as technology has improved.
For example, with the rise in misinformation, we have developed ways to understand if topics might be more susceptible to unreliable content, like conspiracy theories or medical misinformation, and orient our ranking more towards authoritativeness in those moments.
We do hundreds of thousands of quality evaluations every year to make sure we’re meeting our high bar for quality. We’ll regularly make broad updates to our systems, called core updates, as well as more specialised updates, like our helpful content update, to continue to deliver useful results.
There are dimensions of information quality that continue to evolve and require novel approaches. For example, one known challenge is information gaps, also known as "data voids," in which high-quality information just might not be available.
We have come up with ways to show notices when topics are rapidly evolving or when we don’t have high confidence in the quality of results, cautioning that people should approach these results with greater scrutiny.
We have also invested in other information literacy tools that help people check sources, get context, and evaluate what they’re finding.
Deciphering meaning
Information understanding — and in particular, understanding the meaning behind people’s queries — has been a critical focus area for Google over the years.
Early on, our systems were largely built on simple word matching. But that’s where things like spelling became really important. At the time, if you spelled something incorrectly, you’d only find pages that spelled the word wrong, too! So we built our first machine learning system in Search to tackle this challenge.
Over the years, our systems became much more sophisticated, understanding synonyms and the context of words better.
Breakthroughs like the Knowledge Graph helped us understand how people, places, and things in the world relate to each other — understanding the world more like people do — so we can get you exactly what you’re looking for, and even take the next step.
For example, searching for the hottest new movie and getting facts about the cast and running time, along with theatres near you and links to buy tickets.
Large language models like BERT, developed by the Google Research team, have helped us take huge leaps forward in deciphering natural language queries, so we can deliver more relevant results across languages used around the world.
Such models can take learnings from one language and apply them to others, so we can return better results in the many languages that Search is offered in. And we have built tools like Google Translate to help break down language barriers for people so they can understand more of the world’s information.
The latest generative AI technology and large language models can help us re-imagine Search, unlocking new types of questions and transforming the way we organise information.
We’re currently experimenting with applying generative AI to our Search experience through Search Labs, and we’re rapidly updating the experience as we explore what’s possible.
Understanding images, videos and more
There’s so much information in the world that is not text, and so many ways to ask for things that are not by typing words into a search box.
By applying the latest developments in Natural Language processing (NLP), in 2008 we launched the ability to search with your voice, making it more natural to search on mobile.
Then, we took voice input a step further with the ability to "hum to search," for all those times when you have a tune stuck in your head but don’t know what lyrics to look up.
In 2015, advances in computer vision made it possible to search what you see with Lens. We turned your mobile phone camera into a way to explore and ask questions about the world around you, so you can learn more about that flower or insect you saw while you were out for a walk in your neighbourhood.
Today, people do more than 12 billion visual searches every month with Lens.
Last year, we launched multi-search, which advanced these capabilities to allow you to add text to your visual searches. Now, you can do things like take a picture of a couch you like and add the word "chair," and Google will use the image and word to show you similar pieces to add to your living room set.
Spotting and stopping spam
Anyone who has ever looked into their email spam folder can appreciate all the work that goes into keeping that junk out of your inbox. On Search, we have built advanced systems to fight spam in the same way. Without our advanced protections, search results would be clogged with completely irrelevant information, phishing attempts, and links to malware.
We constantly develop new techniques and implement updates to our ranking systems to protect against spam. But spam also adapts and evolves, requiring constant attention from our teams.
In recent years, we have applied new AI-powered techniques to spam detection, which has helped us keep search results over 99 percent spam-free.
This remains a big area of investment for us: As long as people come to Google looking for information, spammers will be trying their hand at getting around our protections, so we have to stay vigilant and one step ahead.
Making Search safer
Over the years, we have maintained a strong commitment to our principles of maximising access to information while helping people stay safe and in control. We aim to help people find information that’s within the bounds of legal expression while not inadvertently exposing them to low-quality or harmful content that they haven’t asked to see.
We have approached this both through expanding our policy protections for people to remove sensitive personal information from our results and through improving our ranking systems with safety and inclusivity in mind.
For example, we have launched improvements to reduce unwanted explicit content from ranking highly in Search and updates to blur explicit imagery by default, and ranking improvements to limit the reach of sites that use exploitative practices.
Meanwhile, we have updated our policies so that people under the age of 18 can have images of themselves removed from Search, and launched new tools like Results about you to make it easy to control how your personal information shows up in search results.
*Dr Alistair Mokoena is the country director for Google South Africa.
** The views expressed do not necessarily reflect the views of IOL or Independent Media.
IOL