Multi-modal large language model for drug development

Abstract

The development of drug delivery systems is crucial for improving therapeutic effectiveness. Currently, AI models assessing drug efficacy often concentrate on specific tasks and necessitate extensive fine-tuning of specialized architectures, restricting their applicability and scalability. As such, we investigate utilising Large Language Model (LLM) to predict the efficacy of lipid nanoparticles (LNP), a new class of drugs exemplified by the COVID-19 mRNA vaccines. Our method employs many-shot in-context learning (ICL) and batched querying, enabling LLMs to seamlessly integrate and process a diverse range of drug-related information within the input prompt.

We assess performance through experimentation across phases. First, we perform ablation studies to examine the impact of ICL exemplars and batch sizes on performance, revealing optimal configurations which enhance model accuracy. Second, we explore parameter-efficient fine-tuning (PEFT) strategies. The experiments, carried out with LLM variants including GPT-4o and Llama 3.1, showed that ICL has potential in improving the model's predictive accuracy. Additionally, we explore PEFT implementations for managing complex drug efficacy data, indicating possible enhancements over standard methods.

The findings highlight the promise of LLMs by offering a flexible, efficient, and scalable method for predicting LNP efficacy. This strategy reduces the need for extensive AI technical knowledge among biomedical researchers, making drug development process more accessible and efficient. We believe that utilising LLMs will lead to a significant transformation in drug discovery, providing a comprehensive tool across different therapeutic approaches.