Class-specific data augmentation of an imbalanced dataset for signer-independent fingerspelling recognition
Document Type
Article
Publication Date
1-1-2026
Abstract
Background: Signer-independent fingerspelling recognition (FSR) aims to convert hand configurations into text or speech while remaining robust to inter-signer variations. Although existing models perform well in signer-dependent settings, their accuracy drops significantly in signer-independent scenarios. A key challenge is dataset imbalance, including class imbalance and class-subject imbalance, where underrepresented classes or signer-specific samples contribute to misclassification, especially among visually similar handshapes. Methods: This study proposed a class-specific data augmentation strategy to improve signer-independent FSR within the Massey dataset of static American Sign Language (ASL) images. To address class-subject imbalance, an initial brightness augmentation was applied, followed by targeted augmentation of frequently misclassified class groups (0, 2, 6, O, V, W, M, N, and T) to mitigate persistent misclassification patterns. A lightweight three-layer end-to-end convolutional neural network (CNN) was trained to extract discriminative spatial features across 36 alphanumeric classes. All experiments adopted a leave-one-subject-out cross-validation (LOSO-CV) to evaluate signer-independent recognition. Results: The proposed strategy improved signer-independent FSR accuracy on the Massey dataset from a baseline of 83.60 to 88.22 through multiple class-specific augmentation techniques. This improvement was statistically significant (p = 0.0198), indicating reduced misclassification among visually similar signs and improved robustness in a static-image signer-independent setting under constrained data conditions.
Keywords
Convolutional neural network (CNN), Data augmentation, Feature extraction, Imbalanced dataset, Inter-signer variability, Lightweight deep learning model, Massey dataset, One-subject-out cross-validation (LOSO-CV), Sign language recognition, Signer-independent fingerspelling recognition
Publication Title
Peerj Computer Science
DOI
10.7717/peerj-cs.3784
Recommended Citation
Mohamed, Noraini; Mustafa, Mumtaz Begum; Jomhari, Nazean; Raja Yusof, Raja Jamilah; and Abdul Sani, Asmiza, "Class-specific data augmentation of an imbalanced dataset for signer-independent fingerspelling recognition" (2026). Research Publications (2026 to 2030). 1526.
https://knova.um.edu.my/research_publications_2026_2030/1526
Issue
4