Class-specific data augmentation of an imbalanced dataset for signer-independent fingerspelling recognition

Document Type

Article

Publication Date

1-1-2026

Abstract

Background: Signer-independent fingerspelling recognition (FSR) aims to convert hand configurations into text or speech while remaining robust to inter-signer variations. Although existing models perform well in signer-dependent settings, their accuracy drops significantly in signer-independent scenarios. A key challenge is dataset imbalance, including class imbalance and class-subject imbalance, where underrepresented classes or signer-specific samples contribute to misclassification, especially among visually similar handshapes. Methods: This study proposed a class-specific data augmentation strategy to improve signer-independent FSR within the Massey dataset of static American Sign Language (ASL) images. To address class-subject imbalance, an initial brightness augmentation was applied, followed by targeted augmentation of frequently misclassified class groups (0, 2, 6, O, V, W, M, N, and T) to mitigate persistent misclassification patterns. A lightweight three-layer end-to-end convolutional neural network (CNN) was trained to extract discriminative spatial features across 36 alphanumeric classes. All experiments adopted a leave-one-subject-out cross-validation (LOSO-CV) to evaluate signer-independent recognition. Results: The proposed strategy improved signer-independent FSR accuracy on the Massey dataset from a baseline of 83.60 to 88.22 through multiple class-specific augmentation techniques. This improvement was statistically significant (p = 0.0198), indicating reduced misclassification among visually similar signs and improved robustness in a static-image signer-independent setting under constrained data conditions.

Keywords

Convolutional neural network (CNN), Data augmentation, Feature extraction, Imbalanced dataset, Inter-signer variability, Lightweight deep learning model, Massey dataset, One-subject-out cross-validation (LOSO-CV), Sign language recognition, Signer-independent fingerspelling recognition

Publication Title

Peerj Computer Science

DOI

10.7717/peerj-cs.3784

Issue

4

Share

COinS