A user study on features supporting subjective relevance for information retrieval interfaces

Abstract

Objective relevance regards retrieved documents as relevant without considering users’ tasks. Subjective relevance, in contrast, focuses on usefulness of documents for users’ contexts. This paper aims to enhance objective relevance and address its limitations by designing a mock-up interface based on ACM Digital Library so as to enhance its support for users’ evaluating the subjective relevance of documents. Important features elicited using a factor analytic approach from an earlier study were used to inform design of the search, results list and document record pages in the mock-up. A pilot study was conducted to gather users’ feedback about usefulness of designed features. Findings indicated that majority of important features designed were useful. However, subjects suggested that the design of some features could be further improved to facilitate their support for SR. The paper concludes by discussing recommendations for improving the mock-up.