Please use this identifier to cite or link to this item:
Title: Evaluation of web-based search engines using user-effort measures
Authors: Tang, Muh-Chyun
Sun, Ying
Keywords: Library and information science
Issue Date: 2003
Source: Tang, M. & Sun, Y. (2003). Evaluation of web-based search engines using user-effort measures. Library and Information Science Research E-Journal, 13(2), 1-9.
Journal: Library and Information Science Research E-Journal 
Abstract: This paper presents a study of the applicability of three user-effort-sensitive evaluation measures —“first 20 full precision,” “search length,” and “rank correlation”—on four Web-based search engines (Google, AltaVista, Excite and Metacrawler). The authors argue that these measures are better alternatives than precision and recall in Web search situations because of their emphasis on the quality of ranking. Eight sets of search topics were collected from four Ph.D. students in four different disciplines (biochemistry, industrial engineering, economics, and urban planning). Each participant was asked to provide two topics along with the corresponding query terms. Their relevance and credibility judgment of the Web pages were then used to compare the performance of the search engines using these three measures. The results show consistency among these three ranking evaluation measures, more so between “first 20 full precision” and search length than between rank correlation and the other two measures. Possible reasons for rank correlation’s disagreement with the other two measures are discussed. Possible future research to improve these measures is also addressed.
ISSN: 1058-6768
DOI: 10.32655/LIBRES.2003.2.1
Rights: © 2003 Tang Muh-Chyun and Sun Ying. All rights reserved.
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:Library and Information Science Research E-journal (LIBRES)

Files in This Item:
File Description SizeFormat 
Evaluation of web-based search engines using user-effort measures.pdf619.67 kBAdobe PDFThumbnail

Page view(s)

Updated on May 31, 2023

Download(s) 50

Updated on May 31, 2023

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.