情绪是认知活动中产生的一种对客观事物的反应，情绪识别是人机交互、情感计算领域的重要组成部分。与行为信号相比，生理信号的时间分辨率高且难以伪装，可以提供更可靠且丰富的情绪信息。本文主要研究基于生理信号的情绪识别方法，从脑电信号和外周信号两个角度，分别总结情绪识别领域常用的信号特征，提出对单一和混合情绪的离线或实时的识别方法，在多个生理信号-情绪数据集上对提出的方法进行评估，并讨论了不同特征、不同方法用于情绪识别的不同效果及可能的原因。在基于脑电信号的识别部分，我们首先使用效价-唤醒度模型定义情绪空间，提出稀疏的动态图卷积神经网络算法对多通道脑电信号的电极间关联结构施以稀疏约束，以改进现有的识别方法。该网络在SEED、DEAP、DREAMER 和CMEPD四个脑电-情绪数据集上都可以获得比现有算法更好的识别效果。其次，我们使用基于线性回归模型和长短时记忆网络构建的单独目标、多目标堆叠和回归链集多目标识别框架，对由多种情绪构成的混合情绪状态进行识别，分别预测每种情绪在混合状态中所占比例。实验结果表明了我们所提出的方法的可行性，也展示出与刺激材料本身的特征相比，脑电特征可以得到更好的识别效果。在基于外周生理信号的识别部分，我们用深度置信网络构建了特征选择与情绪分类融合的实时情绪识别框架。测试结果表明，与现有的实时情绪识别方法相比，我们的实时识别方法在直接识别和基于先验知识的分层识别两种方案下，都可以获得更高的识别准确率。
Emotion is a response to objective things in human cognitive activities. Emotion recognition is an important part of human-computer interaction and affective computing. Compared with behavioral signals, physiological signals have high temporal resolutions and are difficult to disguise. Therefore, physiological signals provide rich and reliable emotional information. In this paper, we mainly study emotion recognition based on physiological signals. From the perspectives of electroencephalography (EEG) and peripheral signals, we summarize commonly used features, propose and evaluate off-line and real-time recognition methods for single and mixed emotions, and discuss emotion recognition results obtained by different features and methods and possible reasons.For EEG signal, we first use the valence-arousal model to define the emotion space, and propose a sparse graph convolutional neural network to sparsely constrain connections among EEG channels. This network performs better than existing algorithms on SEED, DEAP, DREAMER and CMEPD datasets. Then, we use the multi-target emotion recognition methods, i.e., Single Target (ST), Multi-Target Stacking (MTS) and Ensemble of Regressor Chains (ERC) based on Linear Regression (LR) or Long Short-Term Memory (LSTM) to identify mixed emotions, generating the ratio of each emotion. The recognition results show that the recognition method is feasible, and that the features of EEG signal perform better than the features of stimulation videos. For peripheral physiological signals, we combined feature selection together with emotion classification based on Deep Belief Network (DBN) to predict emotions in real time. Results show that compared with the existing real-time emotion recognition method, our method obtains higher accuracies when performing both direct recognition and hierarchical recognition which is based on prior knowledge.
