为深挖消费者评价中消费偏好与需求信息,解决传统的情感分析方法中语言碎片化和情感偏差利用不足的问题,提出基于BERT-BiLSTM-Attention（BERT-BA）的层次化情感分析模型。首先,模型通过BERT预训练模型提取特定领域文本的语义内容;接着通过BiLSTM双向循环神经网络搭建词汇与上下文的语义关联,精准识别情感表达的细微变化;最后通过注意力机制,针对性解决评论情感极性判断偏差等核心问题,对文本中关键情感词汇自适应地赋予更高权重。模型实现对中文商品评论积极、中立、消极三分类。实验结果表明,本文提出的基于BERT-BA的层次化情感分析模型在公开数据集表现出色,在准确率、F1值等指标上优于CNN、BiLSTM、BERT-GRU-Attention模型。
Abstract
In order to solve the problem of language fragmentation and insufficient utilization of sentiment bias in the sentiment analysis methods based on deep learning. This paper proposes the hierarchical sentiment analysis model based on BERT-BiLSTM-Attention (BERT-BA). Initially, the pre-trained BERT model extracts semantic information from the domain-specific text. Then, the BiLSTM enables it to accurately identify subtle shifts in sentiment expression by establishing semantic associations between target token and context. Finally, the attention mechanisms adaptively assign higher weights to key sentiment-related words through addressing the difficulty of extracting Chinese character features and sentiment bias. The experimental results demonstrate that the proposed hierarchical sentiment analysis model based on BERT-BA achieves high performance on public datasets and outperforms CNN, BiLSTM, and BERT-GRU-Attention models in terms of accuracy and F1 score.
关键词
情感分析 /
BERT /
BiLSTM /
注意力机制
Key words
sentiment analysis /
BERT /
BiLSTM /
attention mechanism
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 邱锡鹏. 神经网络与深度学习[M].北京:机械工业出版社,2020.
[2] Lai S,Xu L,Liu K, et al.Recurrent convolutional neural networks for text classification[C]//Proceedings of the AAAI conference on artificial intelligence.2015,29(1):2267-2273.
[3] 张严,李天瑞.面向评论的方面级情感分析综述[J].计算机科学,2020,47(6):200-206.
[4] Turney P D.Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[J].arXiv preprint cs/0212032,2002.
[5] Cao G,Ma Y,Meng X,et al.Emotion recognition based on CNN[C]//2019 Chinese Control Conference (CCC).IEEE,2019:8627-8630.
[6] Naseem U,Razzak I,Musial K,et al.Transformer based deep intelligent contextual embedding for twitter sentiment analysis[J].Future Generation Computer Systems,2020,113:58-69.
[7] Cai R,Qin B,Chen Y,et al.Sentiment analysis about investors and consumers in energy market based on BERT-BiLSTM[J].IEEE access,2020,8:171408-171415.
[8] Devlin J,Chang M W,Lee K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics:human language technologies,volume 1 (long and short papers).2019:4171-4186.
[9] Staudemeyer R C,Morris E R.Understanding LSTM—a tutorial into long short-term memory recurrent neural networks[J].arXiv preprint arXiv:1909.09586,2019.
[10] Huang Z,Xu W,Yu K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv preprint arXiv:1508.01991,2015.
