In order to solve the problem of language fragmentation and insufficient utilization of sentiment bias in the sentiment analysis methods based on deep learning. This paper proposes the hierarchical sentiment analysis model based on BERT-BiLSTM-Attention (BERT-BA). Initially, the pre-trained BERT model extracts semantic information from the domain-specific text. Then, the BiLSTM enables it to accurately identify subtle shifts in sentiment expression by establishing semantic associations between target token and context. Finally, the attention mechanisms adaptively assign higher weights to key sentiment-related words through addressing the difficulty of extracting Chinese character features and sentiment bias. The experimental results demonstrate that the proposed hierarchical sentiment analysis model based on BERT-BA achieves high performance on public datasets and outperforms CNN, BiLSTM, and BERT-GRU-Attention models in terms of accuracy and F1 score.
Key words
sentiment analysis /
BERT /
BiLSTM /
attention mechanism
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
References
[1] 邱锡鹏. 神经网络与深度学习[M].北京:机械工业出版社,2020.
[2] Lai S,Xu L,Liu K, et al.Recurrent convolutional neural networks for text classification[C]//Proceedings of the AAAI conference on artificial intelligence.2015,29(1):2267-2273.
[3] 张严,李天瑞.面向评论的方面级情感分析综述[J].计算机科学,2020,47(6):200-206.
[4] Turney P D.Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[J].arXiv preprint cs/0212032,2002.
[5] Cao G,Ma Y,Meng X,et al.Emotion recognition based on CNN[C]//2019 Chinese Control Conference (CCC).IEEE,2019:8627-8630.
[6] Naseem U,Razzak I,Musial K,et al.Transformer based deep intelligent contextual embedding for twitter sentiment analysis[J].Future Generation Computer Systems,2020,113:58-69.
[7] Cai R,Qin B,Chen Y,et al.Sentiment analysis about investors and consumers in energy market based on BERT-BiLSTM[J].IEEE access,2020,8:171408-171415.
[8] Devlin J,Chang M W,Lee K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics:human language technologies,volume 1 (long and short papers).2019:4171-4186.
[9] Staudemeyer R C,Morris E R.Understanding LSTM—a tutorial into long short-term memory recurrent neural networks[J].arXiv preprint arXiv:1909.09586,2019.
[10] Huang Z,Xu W,Yu K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv preprint arXiv:1508.01991,2015.
Funding
厦门大学嘉庚学院校级科研孵化项目,项目编号：YM2024L02
