How Search Engines Work
Most search engines store a snapshot of the web in a database, and use
this database to generate results for the user queries. To compile this
database the search engines use robots, or spiders, that visit the sites
on the web and store a copy of the crawled pages in the search engine's
cache. Google currently indexes the largest number of pages out of all
of the search engines.
Because computational complexity of a search conducted on billions of
pages is so high, most search engines distribute the result generation
over a few machines. To rank the pages, the search engines apply algorithms
that determine how relevant a particular page is to the user’s query.
These algorithms tend to change from time to time as the search engines
work to improve the search results.
A common perception of the search engine robots is that they are dumb.
True, they don't understand your pages in the direct sense of the word.
The robots themselves are relatively simple creatures – their main
function is to collect data on sites by crawling web links without performing
any analysis. However, a lot of search engines have recently started to
make their algorithms smarter by programming functions that simulate understanding
so that the search results can be more relevant to the user’s query.
Some search engines even want their search function to learn user behavior
to produce more customized and relevant results.
For more information on how specific search engines work, read our section
that describes the major ones.
Related documents:
 |