A useful guide

Want to succeed in SEO? First, learn how search engines work

SEO  |  9 min read

Search engines and intelligent agents such as Hey Google, Siri, Alexa and the devices they support have become crucial to business success. Even when searchers know what they want to find, search engines use data and machine learning to determine “Search Intent” to retrieve the desired results.

Understanding the process behind every search engine query is a powerful exercise that prepares companies for making products and services rank highly.

You can be ready when potential clients in Singapore – or anyone anywhere – requests information via search engine. Helping a search tool’s software to “Learn” about your pages (sometimes called Indexing) or “Rank” your pages by evaluating their content can make a big difference in how your pages respond to a search enquiry.

What are search engines and how do they work?

To users, the best search engines filter through all the Web’s contents to find relevant, timely information. Companies use software to gather details from websites and data about the pages they contain. These are the raw materials that power search engines. It starts when Google, Yahoo, Bing, and other engines send out software (or web crawlers) that explores the Web to gather data from – and about – all the world’s websites.

To keep searchers happy (and the revenue rolling in from advertising), search engines rank their results using multiple factors such as page speed, fresh content, links, specific words.

Types of search engines

There are different types of search engines. How each one performs its work will affect the results in their database and explains why the exact same search terms on the same day may produce different rankings or entirely unique results.

1. Automated, software-driven search uses crawler-based systems (often called spiders) to crawl around to “read” your pages.
2. Human curated search, use editors who scan and evaluate images, personal reviews, comments, and other website contents. Many companies, for example, use human reviewers to identify and remove objectionable material, disturbing images or to localise results in different languages.
3. Hybrid combinations use the best of both with human editing of automated results. These can be for humans who confirm current information or curate hyper-local communities.
4. Specialised search engines may have very specific databases (such as IMDb.com about movies and TV, for example). It may only target a specific website or even data that may not available on the public Internet.

Meta search engines are those used to search through results from other existing data sets.

Most mass-market search engines – such as Google, Bing, Baidu, etc. – use a hybrid approach. There is a limit to human contributions, so the majority of work is done by software bots to crawl the web and discover new pages.

How search engines work

Pages are discovered and crawled by bots and then indexed and served (and ranked by search algorithms) once they are given a score. Data about a website, including the content on these pages – data, words, or images – and any links or identifiable brand names or other data sources about a known company, contributes to its ranking and its rating determines the order in which it’s placed in a list of results.

Users are far more inclined to click on the top-ranked, first page results of their query, so companies vie for that spot. Google does not accept money to rank companies highly, insisting the work is done programmatically by software (though it will allow advertisers to pay for appearing on the first page).

This crawl-index-serve process changes over time as search terms and habits evolve. It ensures that someone looking for a boba tea shop in Singapore doesn’t see a coffee importer or maker of golf tees. Data gathered from other sources – government records,  social media, reviews, etc. – confirm details or reliability of search results.

Other types of search engines

People typically use a search engine to answer a question; gather information or verify details; seek a solution to a problem. For some users, the search is a comparison tool for finding reviews, comments before making a purchase or decision.

General-purpose search engines are the household names – Google, Bing, Yahoo in English, Baidu in Chinese. In various industries, for example, Amazon and Alibaba became top search engines for merchandise, and each has its strengths or weaknesses. Vimeo, YouTube, or DailyMotion are resources for searching through video, TV, and movie contents.

Globally, different languages create other search tools such as Baidu in Chinese, Naver in Korean, and Yandex in Russian.

Private search engines

Newer search engines – including Qwant in Europe, DuckDuckGo in the US, and PetalSearch by the Chinese company, Huawei – are capitalising on how much web data has been collected and how quickly popular sites are updating complex pages.

Search results may be useful to you, but they are money in the bank for search engine companies. Your clicks tell them what sites are relevant and help them do their work. Many search sites also retain details such as your location/geography, IP address, and device type.

Confidential search tools now promise ‘do not track’ for personal history and privacy for the clicks you select or ranking those choices. Some are promising encrypted private search results that don’t track responses.

Other search sites are promising to be advertising-free and not giving advertisers prime positioning at the top of the results page. Often, those advertisers are the top “results” even when they don’t match the search requests.

What are the most used search engines in Singapore?

Globally, Google received more than 80% of web search activity in 2021. Rivals include Bing, Yahoo, Qwant, and others.

According to Statcounter.com, the market share results for Singapore in April 2022 were:

• Google – 95.68%
• Bing – 2.13%
• Yahoo – 1.35%

Ranking highly in Google is difficult enough for household names and companies with global resources. Depending on your industry or where your customers are searching, Singapore companies might want to focus on performing well in these local search resources:

• Yelp – local businesses, focused on retail restaurants, hotels, services.
• Burrple – specialty search for food and beverage establishments
• Houzz – renovation contractors and interior design
• PropertyGuru – real estate properties for sale and rental
• MyCareersFuture – free jobs listing site established by the Singapore government

How did Google get so important?

Google became “to search for answers” as a verb just as your parents might have used “to Xerox” (another company name) as a verb for making a photocopy on paper in the 1970s and 1980s.

Estimates by Internet Live Stats report that Google handles almost 3.5 billion searches a day on average. And the number of people using the search giant daily comes to more or less 1.7 billion. Because those search requests are saved, Google knows what people are searching for and uses that to price advertising, or to use enquiries as a proxy for tracking weather, illnesses such as global spread of flu, or other trends.

Another reason is metadata – data about data and activity – makes Google grow more influential as more people use it. Data patterns and user habits, such as past search queries, browser choice, and content preferences, are just part of what Google knows from analysing billions of interactions worldwide.

Speed is another factor as attention spans get shorter in waiting for results. That sparked developments such as “Auto Complete” (where Google guesses at your request and tries to complete your thought to save typing) and “People Also Asked” to encourage a next, related enquiry.

The locations, times of day, types of devices, and other details are all saved in Google’s search histories. Voice searches, typing inputs, mis-spellings, and countless other bits of activity contribute to the influence Google wields by knowing what people want to know.

Search engines are constantly developing and adapting technology and algorithms – consider voice search and its impact of intelligent agents that combine sources such as restaurant reservations and traffic reports – to provide services such as “Leave Now” advice. The world’s demands are growing more complex than just accepting accurate, useful search results.

How can I help search bots to crawl and index my pages?

One starting point is making sure that webpages are linked properly and are made to be crawled. Some are tagged “No Index” to avoid attention and won’t show up on SERPs.

You can submit site maps to a search engine and share webpage URLs via social platforms or other third-party websites, such as blogs, to get more attention.

Using standard data tags – structured data, sometimes called “schema markup” are ways of helping search engine bots to better understand the page and contents.

Synonyms, industry terms, and other tags should be tested – such as “restaurant” and “dinner” and “lunch” and “sushi” might all work for a Japanese-style restaurant. All such tags labelled reflect the keywords or concepts being emphasised.

Typically, one concept or idea per page, and be sure that tags or meta descriptions reflect that concept. Connect, and link, webpages together under a specific topic.

When pages have quality content that provides new insights not seen on other websites, rankings can follow.  Providing a good user experience – easy reading, links to add value, illustrations, or images – is another way to win attention.

Some companies compare their pages with competitors to see how search results and rankings perform.