Semantic, syntactic and joint deep learning of event extraction

Abstract

Much of human decision-making is based on a cognition of events. Events, ranging from simple occurrences to complex structures, form the essence of human communication, encapsulating rich information about actions, entities, and relationships. In the era of big data, empowering machines to recognize events in human language emerges as an important research problem for unlocking valuable insights from unstructured textual data. Event Extraction (EE), one of the main Information Extraction (IE) tasks in the field of Natural Language Processing (NLP), is immensely useful for real-world applications across different fields such as media, business, cybersecurity, and biomedical research.

Event extraction methods aims at automatically extracting event-related mentions from text documents written in natural language. However, traditional rule-based and statistical methods for event extraction often struggle to handle the inherent complexity and variability of natural language. This limitation has spurred significant interest in leveraging deep learning and neural network techniques to tackle the challenges of event extraction. Neural methods offer the promise of automatically learning intricate patterns and representations from data, enabling more effective and efficient event extraction systems.

Motivated by the growing importance of event extraction and the potential of neural approaches, this thesis aims to explore novel methods for enhancing event extraction from different aspects of deep learning. By delving into the nuances of event representation learning, semantic and syntactic understanding, and multi-task learning optimization, the thesis aims to push the boundaries of current event extraction systems and pave the way for more accurate and comprehensive event understanding.

In summary, the main contributions are: first, we propose the Contrastive Learning Framework via Semantic Type Prototype Representation Modeling for Event Detection (SemPRE), which utilizes pre-defined event type labels to capture event type semantics. On two benchmark datasets, namely ACE 2005 and MAVEN, SemPRE achieves state-of-the-art results and demonstrates excellent performance in situations with limited training data, sentences containing multiple events, and trigger word disambiguation, thus offering a robust solution for more accurate and efficient event detection systems.

Second, we propose Syntactic Reinforcement for Neural Event Extraction (SRE) model for sentence-level and document-level event extraction. Sentence-level SRE outperforms other models on ACE 2005, CASIE, and PHEE, while document-level SRE outperforms prior state-of-the-art on MUC-4. It also achieves absolute improvements of 1.10\%-11.81\% in recall over the existing models on MUC-4. This is the first work that explores the feasibility of intrinsic syntactic mechanisms for event extraction, pushing the boundaries of extraction performance in both fine-grained sentence-level tasks and broader document-level event extraction.

Third, we propose the Adaptive Weighting Method for Joint Information Extraction (AWIE), a novel gradient-based optimization method to balance task losses for joint information extraction dynamically. On three multi-task IE datasets, AWIE outperforms the existing state-of-the-art Uncertainty and MGDA-UB techniques over all the tasks in F1 scores. For event extraction, AWIE improves the static-weighting DYGIE++ baseline by 1.7\%-3.1\% in F1 scores for the event trigger extraction and argument classification subtasks. It offers an effective strategy that dynamically balances task losses during joint IE model training.

Finally, we propose a domain-specific model for biomedical event trigger extraction, Bio-SemSyntEE, which incorporates the semantic-based mechanism from our proposed SemPRE model and the syntax-based mechanism from our proposed SRE model. Bio-SemSyntEE outperforms both discriminative and generative state-of-the-art models across three benchmark datasets. The results demonstrate the generalisability and robustness of our proposed mechanisms for domain-specific applications.