An empirical comparison of ensemble and deep learning models for multi-level Arabic fake news detection using the JoNewsFake dataset

Document Type

Article

Publication Date

1-1-2026

Abstract

We present a comparative study of ensemble and deep learning architectures for multi-label, multi-level Arabic fake news detection on the JoNewsFake dataset, a dataset of 50,000 Facebook posts from 12 verified Jordanian news agencies annotated with 22 main categories, 75 subcategories, and Fake/Real labels. The models include Random Forest, Extra Trees, LightGBM, Convolutional Neural Network (CNN) + Bi-LSTM, CNN + Bi-GRU, and a Transformer encoder. All systems use a hybrid representation that concatenates AraBERT semantic embeddings, POS-based syntactic features, and emotion indicators (894 dimensions). Extra Trees yields the strongest overall performance with a Macro F1-score of ≈0.95 (Main), ≈0.98 (Sub), and ≈0.95 (Fake/Real), while the Transformer achieves a Subcategory Macro F1-score of ≈0.93, suggesting that self-attention offers benefits for fine-grained label spaces. We provide a clear account of data collection, filtering, annotation, and evaluation to support reproducibility. The findings show that carefully engineered features paired with efficient ensembles remain highly competitive for Arabic news, whereas Transformer encoders excel when hierarchical granularity increases. This work offers a rigorous, data-driven baseline for Arabic fake news detection across multiple classification levels and can guide future extensions (e.g. cross-dialect coverage and cross-platform validation).

Keywords

AraBERT embeddings, Arabic fake news detection, Arabic NLP, Deep learning, Ensemble learning, JoNewsFake dataset, Multi-level and multi-label classification, Transformer model

Publication Title

Peerj Computer Science

DOI

10.7717/peerj-cs.3510

Volume

12

Share

COinS