Multimodal late fusion for heart failure diagnosis using tabular and chest X-ray data: A comparative study of fusion strategies
Document Type
Article
Publication Date
7-1-2026
Abstract
Heart failure (HF) remains a major clinical challenge due to its heterogeneous presentation and the absence of a single definitive diagnostic test, particularly in early-stage disease. In routine practice, clinicians rely on a combination of structured clinical measurements, textual admission notes, and chest X-ray (CXR) imaging to form diagnostic decisions. However, most existing artificial intelligence (AI) approaches for HF diagnosis focus on a single data modality, limiting their clinical realism and robustness. In this study, we proposed a multimodal decision-level late fusion framework that integrated structured clinical variables, short emergency department (ED) admission notes, and CXR images for HF diagnosis. Structured and textual data were extracted from MIMIC-IV and MIMIC-IV-ED, with admission notes encoded using BioClinicalBERT embeddings and dimensionality reduction via principal component analysis (PCA). CXR images from MIMIC-CXR-JPG were enhanced using contrast-limited adaptive histogram equalization (CLAHE) and further optimized through automated removal of non-anatomical artifacts using a YOLOv8-based object detection pipeline. Independent machine learning and deep learning models were trained for each modality, and their predictions were combined using probability averaging, majority voting, and Bayesian fusion. Experimental results demonstrate that all multimodal fusion strategies consistently outperformed unimodal models. The LSTM + VGG16 combination with probability averaging achieved the best internal test performance, with an accuracy of 0.9731 and AUC-ROC of 0.9962. External validation on a real-world cohort from University Malaya Medical Centre (UMMC) further demonstrated encouraging generalizability despite limited sample size. These findings highlight the value of multimodal late fusion in leveraging complementary clinical and imaging information.
Keywords
Chest X-ray, Deep learning, Electronic health records (EHR), Heart failure diagnosis, Late fusion strategy
Publication Title
Biomedical Signal Processing and Control
ISSN
1746-8094
DOI
10.1016/j.bspc.2026.110106
Recommended Citation
Teoh, Jing Ru; Hasikin, Khairunnisa; Wong, Hsiu Ding; Ng, Wei Lin; Lee, Kee Wei; Kiew, Lik Voon; Wu, Xiang; and Lai, Khin Wee, "Multimodal late fusion for heart failure diagnosis using tabular and chest X-ray data: A comparative study of fusion strategies" (2026). Research Publications (2026 to 2030). 537.
https://knova.um.edu.my/research_publications_2026_2030/537
Volume
120