Citation & Export
Citation & Export
Simple citation
Mandal, Soumik. Clarifying user's information need in conversational information retrieval. Retrieved from https://doi.org/doi:10.7282/t3-dtbx-wa64
Export
Description
TitleClarifying user's information need in conversational information retrieval
NameMandal, Soumik (author); Shah, Chirag (chair); Belkin, Nicholas J. (internal member); Ognyanova, Katherine (internal member); Galley, Michel (outside member); Rutgers University; School of Graduate Studies
Degree Date2021-05
Date Created2021
DescriptionWith traditional information retrieval systems, users are expected to express their information need adequately and accurately to get appropriate responses from the system. This setup generally works well for simple tasks. However, with the increase of task complexities, users face difficulties in expressing information need in the form as expected by the system. Therefore, the case of clarifying the user's information need by the system arises. In current search systems, support in such cases is provided in the form of query suggestions or query recommendations.
In contrast, conversational information retrieval systems enable the user to interact with the system in the form of dialogs. The conversational approach to information retrieval enables the system to better support the user's information need by asking clarifying questions. However, current research in both natural language processing and information retrieval systems does limited explaining how to form such questions and at what stage of dialog clarifying questions should be asked. To address the research gap, this dissertation investigates the nature of a user's information-seeking conversations with a dialog agent, where the latter is simulating the role of an intelligent system supporting the user's information need. The goal is to identify the type of questions and their patterns an automated intermediary should ask to negotiate and clarify the user's information need. More specifically, this research explores how an intelligent search system should ask the user questions to clarify her information need in complex task scenarios.
This dissertation used the Taskmaster-1 dataset, which collected prior, simulated written and spoken conversations between the user and a conversational agent from multiple task domains. In this research, a subset of these conversations was qualitatively coded by expert annotators at the utterance level. The coding labels were derived from Taylor's (1967) questions and negotiations in information-seeking conversations. Additionally, the utterances from the selected conversations were also labeled with the speaker's conversational roles in the utterance as per the COR model (1992). A domain-independent typology of clarification questions was established from the analysis of the coded dialogs. Our analysis further revealed the difference in the agent's negotiation plans between the two modalities. In written dialogs, the agent asked most questions on the user's topic of information need compared to the emphasis on understanding the user's motivation observed in spoken conversations. Moreover, the agent mostly used a sequential order of clarification question types while negotiating the need in written dialogs. Thus, the negotiation strategy was more straightforward and without any back and forth transitions between different clarification question types in this modality. In comparison, in spoken dialogs, more complex negotiation strategies were observed in the agent's utterances involving loops between two clarification types. Such loops were observed between clarification questions on the user's preference and anticipating the type of information that the user was after.
Our work on prediction models of clarification questions suggests that prior user's utterance characteristics are important for determining when, within a conversation, a dialog agent should ask a clarification question to the user; however, such characteristics are not so helpful in determining what questions to be asked during a conversation.
In contrast, conversational information retrieval systems enable the user to interact with the system in the form of dialogs. The conversational approach to information retrieval enables the system to better support the user's information need by asking clarifying questions. However, current research in both natural language processing and information retrieval systems does limited explaining how to form such questions and at what stage of dialog clarifying questions should be asked. To address the research gap, this dissertation investigates the nature of a user's information-seeking conversations with a dialog agent, where the latter is simulating the role of an intelligent system supporting the user's information need. The goal is to identify the type of questions and their patterns an automated intermediary should ask to negotiate and clarify the user's information need. More specifically, this research explores how an intelligent search system should ask the user questions to clarify her information need in complex task scenarios.
This dissertation used the Taskmaster-1 dataset, which collected prior, simulated written and spoken conversations between the user and a conversational agent from multiple task domains. In this research, a subset of these conversations was qualitatively coded by expert annotators at the utterance level. The coding labels were derived from Taylor's (1967) questions and negotiations in information-seeking conversations. Additionally, the utterances from the selected conversations were also labeled with the speaker's conversational roles in the utterance as per the COR model (1992). A domain-independent typology of clarification questions was established from the analysis of the coded dialogs. Our analysis further revealed the difference in the agent's negotiation plans between the two modalities. In written dialogs, the agent asked most questions on the user's topic of information need compared to the emphasis on understanding the user's motivation observed in spoken conversations. Moreover, the agent mostly used a sequential order of clarification question types while negotiating the need in written dialogs. Thus, the negotiation strategy was more straightforward and without any back and forth transitions between different clarification question types in this modality. In comparison, in spoken dialogs, more complex negotiation strategies were observed in the agent's utterances involving loops between two clarification types. Such loops were observed between clarification questions on the user's preference and anticipating the type of information that the user was after.
Our work on prediction models of clarification questions suggests that prior user's utterance characteristics are important for determining when, within a conversation, a dialog agent should ask a clarification question to the user; however, such characteristics are not so helpful in determining what questions to be asked during a conversation.
NotePh.D.
NoteIncludes bibliographical references
Genretheses, ETD doctoral
Graduate ProgramCommunication, Information and Library Studies
SchoolSchool of Graduate Studies
CollectionSchool of Graduate Studies Electronic Theses and Dissertations
Persistent URLhttps://doi.org/doi:10.7282/t3-dtbx-wa64
LanguageEnglish
Organization NameRutgers, The State University of New Jersey
RightsThe author owns the copyright to this work.