面部表情在情感计算、人机交互、智能监控等领域具有重要价值。现有表情识别方法多集中于空间域,通过卷积神经网络提取特征,但难以同时兼顾全局结构信息和局部细节特征。同时,提升性能通常伴随着计算开销的增加,限制了其在实际应用中的广泛性。近年来,频域方法因其在特征分离、去噪能力以及全局信息提取方面的优势受到关注,但单一的频域特征提取方案通常难以全面捕获表情图像信息。针对这些问题,提出了一种基于频域低通滤波与Gabor特征融合的多通道人脸表情识别方法,通过多通道融合机制,将低通滤波提取的全局结构信息与Gabor滤波提取的细节信息相结合,在保持新增网络开销微小（新增参数仅为0.02 MB）的同时提升了识别性能。在公开的人脸表情识别数据集FER2013和RAF-DB上,该方法的准确率分别提升了0.78%和1.86%。实验结果表明,该方法有效弥补了传统方法的局限性,并在几乎不增加网络计算开销的前提下使识别准确率有一定的提升,为实际应用提供了一种解决方案。
Abstract
The current face expression recognition methods mainly focus on the spatial domain when facing scenes such as emotional computing, human-computer interaction, intelligent monitoring. However, the spatial domain method has the problem of being difficult to deal with the noise, while the frequency domain processing is limited to the details that cannot be well taken into account in the global. In order to solve this problem, a multi-channel face expression recognition method based on frequency domain low-pass filtering and Gabor feature fusion is proposed, which is able to combine the global structural information of the image in the low-pass frequency domain filtering and the global detail information in the Gabor filtering through channel fusion under the premise of small changes in network overhead, so as to make up for the defects of the conventional processing methods, and ultimately improve the face expression recognition model. Finally, the accuracy of the face expression recognition model is improved. The experimental results show that the method improves the accuracy by 0.78% and 1.86% on the publicly available face expression recognition datasets FER2013 and RAF-DB, respectively. It is also demonstrated by means of ablation experiments that this combination method can make up for the defects of each other to a certain extent, which shows that this fusion method has a better effect.
关键词
频率域滤波 /
Gabor变换 /
多通道融合 /
人脸表情识别 /
图像处理
Key words
frequency domain filtering /
Gabor transformation /
multi-channel fusion /
facial expression recognition /
image processing
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] MEHRABIAN A,WIENER M.Decoding of inconsistent communications[J].J Pers Soc Psychol,1967,6(1):109-114.
[2] ZHANG M X,ZHANG H G.A Survey on Interpretability of Facial Expression Recognition[J].Chinese Journal of Computers,2024,47(12):2819-2851.
[3] 张淼萱,张洪刚.人脸表情识别可解释性研究综述[J].计算机学报,2024,47(12):2819-2851.
[4] NG B,CUI X M,JIANG H B,et al.New advances in lightweight networks for facial expression recognition[J].2024, 41(3):663-670.
[5] 蒋斌,崔晓梅,江宏彬,等.轻量级网络在人脸表情识别上的新进展[J].计算机应用研究,2024,41(3):663-670.
[6] 李星燃,张立言,姚树婧.结合特征融合和注意力机制的微表情识别方法[J].计算机科学,2022,49(2):4-11.
[7] 蒋斌,钟瑞,张秋闻,等.采用深度学习方法的非正面表情识别综述[J].计算机工程与应用,2021,57(8):48-61.
[8] 邓翔宇,裴浩媛,盛迎.基于网络融合的改进MobileViT人脸表情识别[J].计算机工程与科学,2024,46(6):102-1080.
[9] 倪锦园,张建勋.多尺度坐标注意力金字塔卷积的面部表情识别[J].计算机工程与应用, 2023, 59(22): 242-250.
[10] 王金伟,曾可慧,张家伟,等.基于空频联合卷积神经网络的GAN生成人脸检测[J].计算机科学,2023,50(6):216-224.
[11] Xu K,Qin M,Sun F,et al.Learning in the Frequency Domain[J].CVPR,2020:1737-1746.
[12] Tang Y,Zhang X,Hu X,et al.Facial Expression Recognition Using Frequency Neural Network[J].IEEE Transactions on Image Processing,2021,30:444-457.
[13] Jiang S,Song P,Wang T,et al.Spatial-and Fourier-domain ptychography for high-throughput bio-imaging[J].Nature Protocols,2023,18:2051-2083.
[14] STUCHI J A,GIL CANTO N,ATTUX R R F,et al.A frequency-domain approach with learnable filters for image classification[J].Applied Soft Computing,2024,155:111443.
[15] 南亚会,华庆一,刘继华.嵌入注意力的Gabor CNN快速人脸表情识别方法[J].软件导刊,2023,22(9):182-189.
[16] 王康毅,邵苏杰.基于Gabor变换的多角度人脸表情识别方法[J].计算机仿真,2024,41(4):233-236+526.
[17] Jabar Sabat Ahily H,Al Dujaili M,Abdulzahra Al Dulaimi M.Automatic human identification using fingerprint images based on Gabor filter and SIFT features fusion.Open Engineering,2024,14(1):20240020.
[18] 石敏. 基于深度残差网络与人脸关键点的表情识别[J].软件导刊,2022,21(4):226-232.
[19] 邵志文,周勇,谭鑫,等.基于深度学习的表情动作单元识别综述[J].电子学报,2022,50(8):2003-2017.
[20] 江盛,朱建鸿.基于ME-ResNet人脸微表情识别方法[J].计算机科学,2024,51(S2):292-298.
[21] 肖振久,陶嘉伟.改进伪三维残差网络的微表情识别方法[J].软件导刊,2024,23(11):63-73.
[22] 徐其华,孙波.基于深度学习和证据理论的表情识别模型[J].计算机工程与科学,2021,43(4):704-711.
基金
江苏省教育厅项目,项目编号：2023SJYB0677; 南京理工大学紫金学院校科研项目,项目编号：2022ZXKX0401012
