Published Paper Details:

: Facial Emotion Recognition (FER) is a vital part of affective computing as well as human-computer interaction since it allows intelligent systems to recognize human emotional states based on their facial expression. Although convolutional neural networks (CNNs) with deep learning have been used to successfully perform spatial models in images of faces, these still have a shortcoming in their ability to capture the temporal changes in emotions that are in the real world. On the other hand, time models enhance dynamic expression but have low spatial discrimination. In response to these shortcomings, this paper will suggest a Hybrid Spatial-Temporal Deep Learning Framework that learns to model both the appearance and emotional state of faces of both still images and video frames. The framework proposed combines the CNN-based spatial feature extraction with the Long Short-Term Memory (LSTM) based temporal modeling in a unified end-to-end implementation. The comprehensive experiments done on FER-2013 and CK+ data sets prove that the hybrid CNN-LSTM model greatly outperforms the spatial-only baselines, having a higher accuracy, stability, and generalization. Besides, statistical significance testing establishes that the performance gains realized are not deceptive and are not as a result of random variation. The findings confirm such a problem as the efficacy of hybrid spatial-temporal learning of strong facial emotion recognition in unconstrained settings.