Multimodal late fusion for heart failure diagnosis using tabular and chest X-ray data: A comparative study of fusion strategies

Document Type

Article

Publication Date

7-1-2026

Abstract

Heart failure (HF) remains a major clinical challenge due to its heterogeneous presentation and the absence of a single definitive diagnostic test, particularly in early-stage disease. In routine practice, clinicians rely on a combination of structured clinical measurements, textual admission notes, and chest X-ray (CXR) imaging to form diagnostic decisions. However, most existing artificial intelligence (AI) approaches for HF diagnosis focus on a single data modality, limiting their clinical realism and robustness. In this study, we proposed a multimodal decision-level late fusion framework that integrated structured clinical variables, short emergency department (ED) admission notes, and CXR images for HF diagnosis. Structured and textual data were extracted from MIMIC-IV and MIMIC-IV-ED, with admission notes encoded using BioClinicalBERT embeddings and dimensionality reduction via principal component analysis (PCA). CXR images from MIMIC-CXR-JPG were enhanced using contrast-limited adaptive histogram equalization (CLAHE) and further optimized through automated removal of non-anatomical artifacts using a YOLOv8-based object detection pipeline. Independent machine learning and deep learning models were trained for each modality, and their predictions were combined using probability averaging, majority voting, and Bayesian fusion. Experimental results demonstrate that all multimodal fusion strategies consistently outperformed unimodal models. The LSTM + VGG16 combination with probability averaging achieved the best internal test performance, with an accuracy of 0.9731 and AUC-ROC of 0.9962. External validation on a real-world cohort from University Malaya Medical Centre (UMMC) further demonstrated encouraging generalizability despite limited sample size. These findings highlight the value of multimodal late fusion in leveraging complementary clinical and imaging information.

Keywords

Chest X-ray, Deep learning, Electronic health records (EHR), Heart failure diagnosis, Late fusion strategy

Publication Title

Biomedical Signal Processing and Control

ISSN

1746-8094

DOI

10.1016/j.bspc.2026.110106

Volume

120

Share

COinS