A Multimodal Deep Fusion Method for Stock Movement Prediction Using Heterogeneous Data Source
Document Type
Article
Publication Date
1-1-2026
Abstract
Stock movement prediction remains challenging due to the complexity of integrating heterogeneous data sources and capturing intricate temporal-semantic relationships. Existing approaches suffer from incomplete semantic preservation, shallow fusion mechanisms, and inadequate multi-scale temporal modeling. In this paper, we propose a multimodal deep fusion framework that addresses these limitations through three synergistic components: 1) An SRTE module based on pretrained BERT that preserves 100% of textual content by segmenting long sequences and applying average pooling, recovering 17.82% more tokens than conventional truncation; 2) An MsLSTM architecture that simultaneously captures short-term price dynamics and long-term market trends through parallel processing of original and temporally smoothed sequences, and; 3) A CMCF mechanism that performs dual-level integration via across-temporal interval fusion and along-temporal step aggregation. We compare our proposed method against baseline models on the StockNet dataset, achieving superior predictive performance. Additionally, we conduct comprehensive ablation studies, back-testing analysis, long-text modeling comparisons, and interpretability analysis. The results validate the effectiveness of deep multimodal fusion for financial forecasting and provide actionable insights for data-driven decision-making in volatile market environments.
Keywords
BERT, deep learning model, Financial time series forecasting, heterogeneous feature embedding, LSTM, multimodal fusion strategy, stock movement prediction
Publication Title
IEEE Access
DOI
10.1109/ACCESS.2026.3654950
Recommended Citation
Wen Wang, Tian; Sook Lu, Yong; Soon Ku, Chin; Mohammed, Okmi; Raj, Bryan; and Yee Por, Lip, "A Multimodal Deep Fusion Method for Stock Movement Prediction Using Heterogeneous Data Source" (2026). Research Publications (2026 to 2030). 1853.
https://knova.um.edu.my/research_publications_2026_2030/1853
Volume
14
First Page
9882
Last Page
9896