Evaluation of Three Search Engines

Searching about the Hoiho or Yellow-eyed penguin

Search terms used: hoiho, yellow eyed penguin

Search engines used: Alta Vista, SearchNZ, Excite.

Relevancy defined as : Which sites I would use to get information to use in a project /speech etc. about this topic.

Alta Vista

SearchNZ

Excite

What differences in relevance did you find?

Restricted search to English. Hoiho : 89 hits

hoiho -math : 60 hits

+yellow +eyed +penguin : 1119 hits

Top 20 of each simple search had 70%-90% directly relevant, with yellow eyed penguin getting more relevant hits.

Using advanced search technique:

"yellow eyed penguin" : 222 hits.

Though not ranked in percentages, 1st 10 not very relevant, more relevant hits when searching on simple searches!

hoiho : 33 hits with majority (80%) in first 20 directly relevant.

yellow eyed penguin : 97 hits, but within first 20 sites, 5 or 6 would have been relevant, every other site was commercial.

Better to search on hoiho than yellow-eyed penguin!

.Had option to show results of search by pages or web sites.

Took web site display option:

hoiho : showed 40 top web sites

hoiho -math : found 29 web sites.

yellow eyed penguin : showed top 40 sites also

Majority of web sites were of direct or secondary relevance for both terms, although I still found some obvious anomolies!

How did differences in the search facility affect your search?

Ability to define searches (with use of " or - etc) was really useful - lowered result number and increased relevancy in first 10 when using the simple search option.

Advanced searching would be more useful for currency (with date defining option) but I personally didn't find the advanced searching technique more useful - in fact it seemed to get less relevant results than previous simple searches!

Search tips concentrated on spelling rather than search punctuation to help one limit search results.

Because I was not able to define searches at all (ie. cut out all commercial sites etc) any relevant sites were swamped by commercial sites, thus making searching on the results very time consuming.

Helped that web pages were able to be ranked together under main web site page wth URL's. This option made it easier to judge relevancy of results, as this meant I was able to see immediately that the Maori language and Math sites were irrelevant., as was the foreign language play that used the term: hoiho.

The Web site display also showed the cohesiveness of the documents on a site.

Didn't show totals of searches, in document display : showed just top 10 in first page and then next 10 in next page etc - a little annoying because I was not sure how many pages were worth searching through.

However, in Web site display, Excite only showed Top 40 sites and I was unable to go to the next 40 (if indeed there were more)

How difficult is it to judge the relevance of results to query?

Gave summeries of results which help to judge which pages to look at.

Difficult to find relevant results on yellow eyed penguin because of all the commercial sites that were included. Yet it was easy to see that the commercial-based results were not relevant!

Not as useful though as Alta Vista and Excite because SearchNZ was getting results from people who tend to use the terms hoiho and yellow eyed penguin more often in names and irrelevant sites.

Easier to judge relevance of results when using Web site display option, rather than document display. Otherwise comparable to Alta Vista.

Return to Assignments

	Alta Vista	SearchNZ	Excite
What differences in relevance did you find?	Restricted search to English. `Hoiho` : 89 hits `hoiho -math` : 60 hits `+yellow +eyed +penguin` : 1119 hits Top 20 of each simple search had 70%-90% directly relevant, with yellow eyed penguin getting more relevant hits. Using advanced search technique: `"yellow eyed penguin"` : 222 hits. Though not ranked in percentages, 1st 10 not very relevant, more relevant hits when searching on simple searches!	`hoiho` : 33 hits with majority (80%) in first 20 directly relevant. `yellow eyed penguin` : 97 hits, but within first 20 sites, 5 or 6 would have been relevant, every other site was commercial. Better to search on hoiho than yellow-eyed penguin!	.Had option to show results of search by pages or web sites. Took web site display option: `hoiho` : showed 40 top web sites `hoiho -math` : found 29 web sites. `yellow eyed penguin` : showed top 40 sites also Majority of web sites were of direct or secondary relevance for both terms, although I still found some obvious anomolies!
How did differences in the search facility affect your search?	Ability to define searches (with use of " or - etc) was really useful - lowered result number and increased relevancy in first 10 when using the simple search option. Advanced searching would be more useful for currency (with date defining option) but I personally didn't find the advanced searching technique more useful - in fact it seemed to get less relevant results than previous simple searches!	Search tips concentrated on spelling rather than search punctuation to help one limit search results. Because I was not able to define searches at all (ie. cut out all commercial sites etc) any relevant sites were swamped by commercial sites, thus making searching on the results very time consuming.	Helped that web pages were able to be ranked together under main web site page wth URL's. This option made it easier to judge relevancy of results, as this meant I was able to see immediately that the Maori language and Math sites were irrelevant., as was the foreign language play that used the term: hoiho. The Web site display also showed the cohesiveness of the documents on a site. Didn't show totals of searches, in document display : showed just top 10 in first page and then next 10 in next page etc - a little annoying because I was not sure how many pages were worth searching through. However, in Web site display, Excite only showed Top 40 sites and I was unable to go to the next 40 (if indeed there were more)
How difficult is it to judge the relevance of results to query?	Gave summeries of results which help to judge which pages to look at.	Difficult to find relevant results on `yellow eyed penguin` because of all the commercial sites that were included. Yet it was easy to see that the commercial-based results were not relevant! Not as useful though as Alta Vista and Excite because SearchNZ was getting results from people who tend to use the terms hoiho and yellow eyed penguin more often in names and irrelevant sites.	Easier to judge relevance of results when using Web site display option, rather than document display. Otherwise comparable to Alta Vista.