During the recent years, Internet has grown manifold and thousands of pages are being added every second. Therefore, it becomes all the more imperative that there is some kind of proper indexing system.
Over the years there has been continuous progress in the way in which indexing is done. The first step in this regard was the development of “spiders”. These were the programs that crawled all through the Internet and stored the content of above 100 pages every second. The spiders used to start from some important site and then tracked down its entire links and then followed all the links from those sites, and the process continued.
The information collected is henceforth stored in a compact description on the company’s computers. This index includes the URLs, the words that appeared on the pages, and some other bits of information.
This means that when you type a query to search something, it’s actually the index that you are searching in and not the Internet.
Previously the search engines used to show the pages in the order that they were found. However, this system used to mostly bury the best matching query somewhere deep down in the rankings.
The next innovation was called the search algorithm. This is a method that tells how appropriately a page matches a query. But this method too failed to produce accurate results as web designers used to add deceptive terms in their titles to be ranked above in the search engines.
Further advancement led to the introduction of PageRank system in 1997. This idea was developed by Google founders Sergey Brin and Larry Page. PageRank finds out how many pages link to the page that is being seen and how many sites link to those pages.
These days many techniques are used to sift out unwanted clutter. For instance, Google keeps on altering its algorithm twice or thrice daily.
Another way to guess what all pages you might be interested in seeing is called ‘Personalization’. Here the question of privacy creeps in as a search engine can have access to your browsing history, e-mail records and the documents on your computer.
Similarly, a search engine can also come up with search results that might be similar to the query that you typed in.
Therefore we can see that many new technologies have led to the improvement in the way search engines function and this has surely made life easier for the Internet user.
