Test case generation from specifications using natural language processing and large language models

Abstract

This report investigates the feasibility and effectiveness of using Natural Language Processing (NLP) and Large Language Models (LLMs), particularly GPT-4o, to automate the generation of software test cases from system specifications. Motivated by the growing need to reduce human effort and time spent on documentation tasks in software development, this project proposes an agentic framework leveraging LangChain to decompose the complex task of test case generation into smaller sub-problems. Each agent in the system is responsible for extracting features, functionalities, and test cases from natural language specifications, with validation agents ensuring alignment and consistency at each step. The methodology is evaluated using a set of diverse sample specifications, and results show that while LLMs can effectively identify key features and generate relevant test cases, the outputs are often limited in depth and completeness. Challenges include prompt sensitivity, validation inconsistencies, and output formatting constraints. Although the current implementation yields moderately successful results, it provides a strong foundation for future enhancements in automated documentation generation, with potential applications in broader areas such as use case and class diagram generation.