Controllable neural text generation in persona-based dialogue systems

Abstract

Persona-based dialogue system, a specialized area of open-domain dialogue system, focuses on creating more engaging and personalized conversational agents. Distinguished from the open-domain dialogue system, it incorporates a consistent and distinct persona that directs its interaction style, which is either explicitly defined or learned from large datasets of human interactions. The recent advance of pre-trained language models boosts the dialogue system, significantly enhancing the user experience by delivering fluent, diversified, and human-like conversations. However, the challenge of maintaining persona consistency and contextual coherence still persists in persona-driven dialogue generation, due to the overlook of user intent and persona identification. Therefore, it is proposed to create a pipeline dialogue system consisting of natural language understanding module for precise user intent detection and persona extraction, and dialogue management and generation module for multiple source inputs integration. The research goals of this thesis include controllable sentence generation for persona-consistent and contextual coherent conversation generation, controllable structural extraction by exploiting persona attributes in conversation histories to facilitate natural language understanding in a dialogue system, and controllable data synthesis for low-resource extraction scenario.

Natural language understanding and dialogue management and generation, which elevated the performance of a task-oriented dialogue system, are introduced to the persona-based dialogue system. In the first work, a lightweight hierarchical intent-inferring pointer network is proposed for multi-source persona-driven dialogue generation. For natural language understanding, the proposed method involves detecting interlocutor intents, retrieving the most relevant persona attributes in chitchat, and utilizing pseudo-labeling and natural language inference techniques to generate intent labels. For dialogue management and generation, a multi-source pointer-generator is designed to leverage the useful information from multiple input sources and filter out irrelevant textual noises.

However, matching the relevant persona attribute is a retrieval-based method and requires predefined persona information, nontransferable to realistic conversations. This motivates the second framework, which formulates and investigates persona attribute extraction from dialogues utilizing pre-trained language models in generalized zero-shot setting. Moreover, a Meta-Variational-AutoEncoder sampler with contrastive structured constraint is presented to tackle the hard negative samples in this task. By leveraging more reliable text-label matching criteria, a task-specific dataset with more detailed relation types and consistent entity relation annotation is created for persona attribute extraction. The aforementioned method is applicable to cases when seen data and unseen relation types are available. A more stringent condition has not been explored where no information about the unseen data is accessible, which is genuine zero-shot persona attribute extraction. In the third work, thus, a Chain-of-Proposal prompting framework is developed for unseen relation generation, controllable support sentence synthesis conditioned on hypothesized relations, synthetic data denoising, and zero-shot persona triplet extraction. Extensive experiments on benchmark and built datasets demonstrate the superior performance of the proposed frameworks compared to strong baseline models. A further discussion and empirical study is conducted on the potential constructive effects of persona attribute understanding on personalized conversational sentiment analysis.