Integrating large language models into time series predictions in healthcare applications

Abstract

Predicting patient outcomes in intensive care units (ICUs) is challenging due to the irregularity and sparsity of medical data. Traditional time-series models usually fail to capture all patient information, while large language models (LLMs) lack temporal awareness and are prone to hallucinations, potentially leading to incorrect predictions. This study introduces a novel modular self attention fusion approach that integrates time-series data with LLMs by treating them as independent modules. Each module processes data separately before extracting the most informative features, which are then fused through its own self-attention fusion layer before being passed downstream for mortality prediction Experiments were conducted using the MIMIC-III dataset, comparing the proposed model against pure time-series models, LLM-based models, and the Multimodal Transformer benchmark. Initial results showed that while the composite model outperformed standalone time-series and LLM models, it initially fell well short of the Multimodal Transformer benchmark in PR-AUC. To address this, the loss function was changed from weighted cross-entropy to Focal Loss, which significantly improved PR-AUC and narrowed the performance gap. The results suggest that focusing on hard-to-classify samples mitigates false positives and enhances predictive accuracy, which makes the novel self-attention fusion approach a promising alternative for integrating structured and unstructured clinical data in ICU mortality prediction.