Please use this identifier to cite or link to this item:
Title: Asking clarifying questions: to benefit or to disturb users in web search?
Authors: Zou, Jie
Sun, Aixin
Long, Cheng
Mohammad Aliannejadi
Kanoulas, Evangelos
Keywords: Engineering::Computer science and engineering
Issue Date: 2023
Source: Zou, J., Sun, A., Long, C., Mohammad Aliannejadi & Kanoulas, E. (2023). Asking clarifying questions: to benefit or to disturb users in web search?. Information Processing and Management, 60(2), 103176-.
Project: IAF-ICP
Journal: Information Processing and Management
Abstract: Modern information-seeking systems are becoming more interactive, mainly through asking Clarifying Questions (CQs) to refine users’ information needs. System-generated CQs may be of different qualities. However, the impact of asking multiple CQs of different qualities in a search session remains underexplored. Given the multi-turn nature of conversational information-seeking sessions, it is critical to understand and measure the impact of CQs of different qualities, when they are posed in various orders. In this paper, we conduct a user study on CQ quality trajectories, i.e., asking CQs of different qualities in chronological order. We aim to investigate to what extent the trajectory of CQs of different qualities affects user search behavior and satisfaction, on both query-level and session-level. Our user study is conducted with 89 participants as search engine users. Participants are asked to complete a set of Web search tasks. We find that the trajectory of CQs does affect the way users interact with Search Engine Result Pages (SERPs), e.g., a preceding high-quality CQ prompts the depth users to interact with SERPs, while a preceding low-quality CQ prevents such interaction. Our study also demonstrates that asking follow-up high-quality CQs improves the low search performance and user satisfaction caused by earlier low-quality CQs. In addition, only showing high-quality CQs while hiding other CQs receives better gains with less effort. That is, always showing all CQs may be risky and low-quality CQs do disturb users. Based on observations from our user study, we further propose a transformer-based model to predict which CQs to ask, to avoid disturbing users. In short, our study provides insights into the effects of trajectory of asking CQs, and our results will be helpful in designing more effective and enjoyable search clarification systems.
ISSN: 0306-4573
DOI: 10.1016/j.ipm.2022.103176
Schools: School of Computer Science and Engineering 
Research Centres: Singtel Cognitive and Artificial Intelligence Lab for Enterprises@NTU
Rights: © 2022 Elsevier Ltd. All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Journal Articles

Citations 50

Updated on Jul 23, 2024

Page view(s)

Updated on Jul 22, 2024

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.