Micro-expression recognition in wild video environments: Latent feature-based ANN (LFANN) from 3D reconstructed faces
Document Type
Article
Publication Date
4-1-2025
Abstract
Facial micro-expression (ME) recognition has garnered substantial interest in both computer vision and psychology for its ability to detect nonverbal emotions through subtle, involuntary facial muscle movements. Despite this potential, the practical deployment and commercialization of ME recognition systems have been hindered by the lack of comprehensive datasets, leaving these systems underdeveloped and limiting their application in real-world scenarios. This study addresses these challenges by introducing a novel ME recognition system specifically designed for natural environments and evaluated using the in-the-wild benchmark dataset, MEVIEW. The proposed system, namely the Latent Feature-based ANN (LFANN), begins with 3D facial reconstruction to align faces, effectively handling challenges such as head rotation, occlusions, and inconsistent lighting. The system further identifies multiple peak frames based on landmark coordinates in predefined facial regions, ensuring a discriminative representation of the video. The displacement of facial muscle movements across consecutive frames is captured via landmark coordinate shifts and transformed into a latent space for improved feature representation. This end-to-end proposed framework is capable of achieving notable performance metrics, including 70% accuracy, 70.83% F1-score, 75.56% UAR, and 67.07% UF1. Additionally, 75% of the detected peak frames fall within the onset and offset periods, validating the effectiveness of the peak frame detection method. Comprehensive evaluations and ablation studies highlight the significance of each system component, positioning this approach as a key advancement in ME recognition study.
Keywords
Micro-expression, 3D reconstruction, In-the-wild, Latent space, Emotion recognition
Divisions
fsktm
Funders
National Science and Technology Council (NSTC) [Grant No: NSTC 111-2221-E-035-059-MY3; NSTC 113-2221-E-035-024-]
Publication Title
Neurocomputing
Volume
625
Publisher
Elsevier
Publisher Location
RADARWEG 29, 1043 NX AMSTERDAM, NETHERLANDS