X-ray report generation

Abstract

Medical imaging and radiology reports are indispensable parts of the modern-day medicine, allowing doctors to diagnose and treat patients effectively. Nevertheless, the growing shortage of radiologists results in healthcare bottlenecks and increased workload for radiologists. Automated radiology report generation using machine learning can potentially overcome pain points in healthcare systems and improve the quality of healthcare services. Previous works on Vision Language Models have shown moderate success in radiology report generation, highlighting the potential for more improvement. In this work, efforts were made to improve the performance of LLaVA-Med and LLaVA for the task of free-text radiology report generation on single Chest X-ray (CXR) images by finetuning the Vision Encoder, Multi-Layer Perceptron (MLP) and the Large-Language Model (LLM) components of the model. Experimental results indicate that most models showed better performance compared to the pretrained models. The best model attained a CheXpert F1 score of 36.9%, a significant improvement from the baseline score of 5.3%.