User evaluation of a task for shortlisting papers from researcher’s reading list for citing in manuscripts
Sesagiri Raamkumar, Aravind
Date of Issue2017-09
Wee Kim Wee School of Communication and Information
Purpose: Although many interventional approaches have been proposed to address the apparent gap between novices and experts for literature review (LR) search tasks, there have been very few approaches proposed for manuscript preparation (MP) related tasks. This paper describes a task and an incumbent technique for shortlisting important and unique papers from the reading list of researchers, meant for citation in a manuscript. Design/methodology/approach: A user evaluation study was conducted on the prototype system which was built for supporting the shortlisting papers (SP) task along with two other LR search tasks. A total of 119 researchers who had experience in authoring research papers participated in this study. An online questionnaire was provided to the participants for evaluating the task. Both quantitative and qualitative analyses were performed on the collected evaluation data. Findings: Graduate research students prefer this task more than research and academic staff. The evaluation measures relevance, usefulness and certainty were identified as predictors for the output quality measure ‘good list’. The shortlisting feature and information cues were the preferred aspects while limited dataset and rote steps in the study were ascertained as critical aspects from the qualitative feedback of the participants. Originality/value: Findings point out that researchers are clearly interested in this novel task of shortlisting papers from the final reading list prepared during literature review. This has implications for digital library, academic databases and reference management software where this task can be included to benefit researchers at the manuscript preparatory stage of the research lifecycle.
Aslib Journal of Information Management
© 2017 Emerald. This is the author created version of a work that has been peer reviewed and accepted for publication by Aslib Journal of Information Management, Emerald. It incorporates referee’s comments but changes resulting from the publishing process, such as copyediting, structural formatting, may not be reflected in this document. The published version is available at: [http://dx.doi.org/10.1108/AJIM-01-2017-0020].