Human emotion, reflected as gut feeling in the body and the way we interpret and appraise a situation in our mind can be defined as complex patterns of responses to events that matter to us. Human emotions has three types of joint and interactive elements: 1) Physiological elements via body heart rate, breathing, and hormone release; 2) Behavioral elements via facial expressions and actions in response to a stimulus; 3) Cognitive elements via the way we appraise and interpret a thing or situation. Different from long-lasting mood and general term affect, “emotion” is an immediate, brief, and specific response to an internal or external stimulus.
Multi-modal sensory data have been used to classify arousal, valence, and emotion using e.g., contrastive language signal pre-training.
We extract human emotions obtained from multiple modalities and physiological, behavioral, and cognitive elements considering both “Discrete Emotion Theory” and “Core Affect Circumplex Model”.
[1] P. Singh, R. Budhiraja, A. Gupta, A. Goswami, M. Kumar, & P. Singh, “EEVR: A dataset of paired physiological signals and textual descriptions for joint emotion representation learning,” NeurIPS, 2024.
[2] D. L. Schacter, D. T. Gilbert, & D. M. Wegner, Psychology. Macmillan, 2009.
[3] P. Ekman, “Emotions revealed,” Bmj, 2004.
[4] J. A. Russell, “A circumplex model of affect.” Journal of personality and social psychology, 1980.