(Date of publication: 2007-05-15)
https://hdl.handle.net/20.500.11932/28303 
| Collection: |
NIMS成果物 |
| Form: |
 Online resource
|
| Language: |
English |
| Physical description: |
|
| Subject: |
|
| Classification: |
|
| Tag: |
|
| Identifier: |
Handle URI: https://hdl.handle.net/20.500.11932/28303
|
| Full-text file |
|
| Abstract: |
This paper presents a simple approach to utilize
past test collections as a material for user experiments.
We have built a Web-based user interface for
NTCIR-5 WEB run results, and conducted a user experiment
with 29 subjects to investigate whether performance
evaluation metrics of information retrieval
systems used in test collections such as TREC and NTCIR
comparable to user performance. In this experiment,
we selected three types of systems from among
systems that participated in NTCIR-5 WEB, and then
selected three topics with roughly the same values from
among several search topics. The results of the experiment
showed no significant differences among these
systems and topics in the time for search. While, in
general, the user experiment itself have been successfully
conducted and shown similar trends with prior
study, the approach seems to have some limitations
mainly on interactivity and cached page display. |
| Note: |
|
