Knowledge enhanced deep learning under small data

Abstract

Deep learning has become a dominant method for a wide range of natural language processing (NLP) tasks. At present, the majority of deep learning techniques rely entirely on data-driven strategies, typically demanding substantial datasets to develop models that perform effectively. In some applications, however, the data available is very limited. How to train a well-performing model under small data is a challenging yet important issue in NLP.

This dissertation investigates enhancing deep learning under small data by integrating domain and linguistic knowledge. Two approaches are proposed: the SetFit model for few-shot text classification, achieving 88.42% accuracy and 88.37% F1 score on the SST-2 dataset with 100 samples, and the K-BERT model, which enhances BERT with knowledge graphs (e.g., CN-DBpedia with 5.17 million triplets, MedicalKG with 13,864 triplets). Tested on tasks like sentiment classification, NER, Q&A, and domain-specific datasets (e.g., Medicine_NER), K-BERT shows significant performance gains over BERT. Key achievements include reduced data dependency, improved interpretability through semantic constraints, and a shift to a "knowledge + data co-driven" NLP paradigm, offering practical solutions for low-resource fields and new insights for future research.