A new survey of the 11 most widely used search engines, reported in a July, 1999 issue of Nature, showed that the search engines are falling behind in keeping up with the expansion of the World Wide Web (which is now estimated to contain 800 million pages). No single search engine, the study found, covered more than 16% of the Web's contents. In contrast, a study in Science in 1998 found that the best search engines covered more than 50% of all pages. A comparison of these results is shown below:
|
|
|||
|
% Coverage |
|||
|
Search Engine |
Dec 97 |
July 99 |
|
|
|
|||
|
|
Northern Light |
32.9 |
16.0 |
|
|
|||
|
|
AltaVista |
46.5 |
15.5 |
|
|
|||
|
|
Snap |
- |
15.5 |
|
|
|||
|
|
HotBot |
57.5 |
11.3 |
|
|
|||
|
|
Microsoft |
- |
8.5 |
|
|
|||
|
|
GO Network |
16.5 |
8.0 |
|
|
|||
|
|
- |
7.8 |
|
|
|
|||
|
|
Yahoo |
- |
7.4 |
|
|
|||
|
|
Excite |
23.1 |
5.6 |
|
|
|||
|
|
Lycos |
4.4 |
2.5 |
|
|
|||
|
|
Euroseek |
- |
2.0 |
|
|
|||
Even when combined, the top 11 search engines cover just 42% of World Wide
Web pages. The study recommended that for exhaustive searches one use a search
engine that harnesses multiple single engines. One such search engine is MetaCrawler
,
which I frequently use. MetaCrawler queries Lycos,
Infoseek (GO Network), WebCrawler,
Thunderstone, About, Excite,
AltaVista, GoTo,
LookSmart, and Yahoo, and returns the results from all
ten search engines.
Last updated 11/17/99
