The prediction of the high-cost non-cumulative discounted gain and precision performance metrics in information retrieval evaluation
Document Type
Conference Item
Publication Date
1-1-2021
Abstract
Information retrieval systems play the critical role of meeting information needs of users. Therefore, high effectiveness is expected from these systems since users make decisions based on the retrieval results. The effectiveness of these systems varies and is only known through retrieval evaluation and the main approach of evaluating these systems is the test collection model which comprises of a corpus of documents, topics and relevance judgments. One limitation of this approach is the cost of generating the relevance judgments. A recent solution to this limitation is the prediction of performance metrics at the high evaluation depths of documents using the system scores of other performance metrics computed at the low evaluation depths. However, this solution has a drawback of the inaccurate predictions of the non-cumulative discounted gain (nDCG) and the precision performance metrics at the high evaluation depths of documents using other performance metrics computed at the low evaluation depths. Therefore, this study addresses this drawback by proposing an approach that predicts the nDCG and precision performance metrics at the high evaluation depths of documents using topic scores of other performance metrics computed at the low evaluation depths. This study has shown that the proposed approach performs better predictions of the nDCG and precision performance metrics than the existing method. © 2021 IEEE.
Keywords
Correlation, Evaluation, Information retrieval, Prediction
Divisions
infosystem
Funders
Universiti Malaya
Publication Title
Proceedings - CAMP 2021: 2021 5th International Conference on Information Retrieval and Knowledge Management: Digital Technology for IR 4.0 and Beyond
Event Title
5th International Conference on Information Retrieval and Knowledge Management, CAMP 2021
Event Location
Virtual
Event Dates
15-16 June 2021
Event Type
conference