Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Zhang, Zixing, Han, Jing, Coutinho, E ORCID: 0000-0001-5234-1497 and Schuller, Bjoern W
(2019) Dynamic Difficulty Awareness Training for Continuous Emotion Prediction. IEEE Transactions on Multimedia, 21 (5). 1289 - 1301.

This is the latest version of this item.

[img] Text
difficulty-learning-slash.pdf - Accepted Version

Download (704kB)


Time-continuous emotion prediction has become an increasingly compelling task in machine learning. Considerable efforts have been made to advance the performance of these systems. Nonetheless, the main focus has been the development of more sophisticated models and the incorporation of different expressive modalities (e.g., speech, face, and physiology). In this paper, motivated by the benefit of difficulty awareness in a human learning procedure, we propose a novel machine learning framework, namely, dynamic difficulty awareness training (DDAT), which sheds fresh light on the research - directly exploiting the difficulties in learning to boost the machine learning process. The DDAT framework consists of two stages: information retrieval and information exploitation. In the first stage, we make use of the reconstruction error of input features or the annotation uncertainty to estimate the difficulty of learning specific information. The obtained difficulty level is then used in tandem with original features to update the model input in a second learning stage with the expectation that the model can learn to focus on high difficulty regions of the learning process. We perform extensive experiments on a benchmark database REmote COLlaborative and affective to evaluate the effectiveness of the proposed framework. The experimental results show that our approach outperforms related baselines as well as other well-established time-continuous emotion prediction systems, which suggests that dynamically integrating the difficulty information for neural networks can help enhance the learning process.

Item Type: Article
Uncontrolled Keywords: emotion prediction, difficulty awareness learning, dynamic learning
Depositing User: Symplectic Admin
Date Deposited: 05 Sep 2018 09:13
Last Modified: 18 Nov 2021 01:10
DOI: 10.1109/TMM.2018.2871949
Related URLs:
URI: https://livrepository.liverpool.ac.uk/id/eprint/3025879

Available Versions of this Item