ZHANG Xiaofeng,TAO Hongcai.Aspect-Level Sentiment Analysis Model based on BERT with Multi-Information Fusion[J].Journal of Chengdu University of Information Technology,2024,39(04):397-403.[doi:10.16836/j.cnki.jcuit.2024.04.001]
基于BERT的多信息融合方面级情感分析模型
- Title:
- Aspect-Level Sentiment Analysis Model based on BERT with Multi-Information Fusion
- 文章编号:
- 2096-1618(2024)04-0397-07
- Keywords:
- aspect-level sentiment analysis; syntactic dependency tree; BERT; multi-information fusion
- 分类号:
- TP391.1
- 文献标志码:
- A
- 摘要:
- 方面级情感分析任务旨在判断文本语句不同方面的情感极性,是自然语言处理领域的热点任务之一。当前基于BERT的方面级情感分析方法大多仅将其作为预训练词嵌入工具,没有充分利用其本身的语义提取能力和任务处理能力,在利用句法及方面词位置等信息时,常忽略了不同外部信息的关联性。针对上述问题,提出一种基于BERT的多信息融合网络模型。首先,构建基于BERT词嵌入的主路径和辅路径,辅路径可以避免对词嵌入特征产生干扰; 其次,在辅路径下,根据文本语句的句法依存树和多信息融合算法让不同词语与方面词的句法距离、位置距离等信息产生交互; 最后,使用卷积神经网络和注意力机制将主路径与辅路径的特征融合。3个数据集的实验结果表明,提出的模型是有效的。
- Abstract:
- Aspect-level sentiment analysis aims to determine the sentiment polarity of different aspects of a text sentence, which is one of the hot tasks in the field of natural language processing. Most of the current BERT-based aspect-level sentiment analysis methods only use it as a pre-trained word embedding tool, without making full use of its own semantic extraction capabilities and task processing capabilities, and often ignore the correlation of different external information when using information such as syntax and aspect word position. To solve the above problems, this paper proposes a multi-information fusion network model based on BERT. Firstly, the main path and auxiliary path based on BERT word embedding are constructed, and the auxiliary path could avoid interference with the word embedding features. Secondly, under the auxiliary path, the syntactic distance, position distance and other information between different words and aspect words are interacted according to the syntactic dependency tree of the text sentence and the multi-information fusion algorithm. Finally, convolutional neural network and attention mechanism are used to fuse the features of the main path and the auxiliary path. Experimental results on three datasets show that the proposed model is effective.
参考文献/References:
[1] Tang D,Qin B,Liu T.Learning semantic representations of users and products for document level sentiment classification[C].Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th international joint conference on natural language processing,2015:1014-1023.
[2] Yang Z,Yang D,Dyer C,et al.Hierarchical attention networks for document classification[C].Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics:human language technologies,2016:1480-1489.
[3] 汤凌燕,熊聪聪,王嫄,等.基于深度学习的短文本情感倾向分析综述[J].计算机科学与探索,2021,15(5):794-811.
[4] Wang H,Liu B,Li C,et al.Learning with noisy labels for sentence-level sentimentclassification[C].Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,2019:6286-6292.
[5] Baccianella S,Esuli A,Sebastiani F.Sentiwordnet 3.0:an enhanced lexical resource for sentiment analysis and opinion mining[C].International Conference on Language Resources and Evaluation,2010:2200-2204.
[6] Cambria E,Poria S,Hazarika D,et al.SenticNet 5:Discovering conceptual primitives for sentiment analysis by means of context embeddings[C].Proceedings of the AAAI conference on artificial intelligence,2018:1795-1802.
[7] Mohammad S,Turney P.Emotions evoked by common words and phrases:Using mechanical turk to create an emotion lexicon[C].Proceedings of the NAACLHLT 2010 workshop on computational approaches to analysis and generation of emotion in text,2010:26-34.
[8] 郁圣卫,卢奇,陈文亮.基于领域情感词典特征表示的细粒度意见挖掘[J].中文信息学报,2019,33(2):112-121.
[9] Hochreiter S,Schmidhuber J.Long Short-Term Memory[J]. Neural Computation,1997,9(8):1735-1780.
[10] Socher R,Perelygin A,Wu J Y,et al.Recursive deep models for semantic compositionality over a sentiment treebank[C].Proceedings of the 2013 conference on empirical methods in natural language processing,2013:1631-1642.
[11] Tang D Y,Qin B,Feng X C,et al.Effective LSTMs for Target-Dependent Sentiment Classification[C].Proceedings of 26th International Conference on Computational Linguistics,2016:3298-3307.
[12] Kim Y.Convolutional Neural Networks for Sentence Classification[C].Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing,2014:1746-1751.
[13] Zhang C,Li Q C,Song D W.Aspect-based Sentiment Classification with Aspect specific Graph Convolutional Networks[C].Proceedings of Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,2019:4568-4578.
[14] Bahdanau D,Cho K,Bengio Y.Neural machine translation by jointly learning to align and translate[J].arXiv preprint arXiv:1409.0473,2014.
[15] Wang Y,Huang M,Zhu X,et al.Attention-based LSTM for aspect-level sentiment classification[C].Proceedings of the 2016 conference on empirical methods in natural language processing,2016:606-615.
[16] Ma D H,Li S J,Zhang X D,et al.Interactive attention networks for aspect-level sentiment classification[C].Proceedings of the 26th International Joint Conference on Artificial Intelligence,2017:4068-4074.
[17] Fan F,Feng Y,Zhao D.Multi-grained attention network for aspect-level sentiment classification[C].Proceedings of the 2018 conference on empirical methods in natural language processing,2018:3433-3442.
[18] Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrases and their compositionality[C].Proceedings of the 26th International Conference on Neural Information Processing Systems,2013:3111-3119.
[19] Pennington J,Socher R,Manning C.GloVe:Global Vectors for Word Representation[C].Proceedings of the 2014 conference on empirical methods in naturallanguage processing(EMNLP),2014:1532-1543.
[20] Devlin J,Chang M W,Lee K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C].Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics,2019:4171-4186.
[21] Song Y W,Wang J,Jiang T,et al.Attentional encoder network for targeted sentiment classification[J].arXiv preprint arXiv:1902.09314,2019.
[22] Zeng B,Yang H,Xu R,et al.LCF:A Local Context Focus Mechanism for Aspect-Based Sentiment Classification[J].Applied Sciences,2019,9(16):3389.
[23] Phan M H,Ogunbona P O.Modelling context and syntactical features for aspect-based sentiment analysis[C].Proceedings of the 58th annual meeting of the association for computational linguistics,2020:3211-3220.
[24] Wang K,Shen W,Yang Y,et al.Relational Graph Attention Network for Aspect-based Sentiment Analysis[C].Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,2020:3229-3238.
[25] Zhang Z,Zhou Z,Wang Y.SSEGCN:Syntactic and semantic enhanced graph convolutional network for aspect-based sentiment analysis[C].Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.2022:4916-4925.
[26] Pontiki M,Galanis D,Pavlopoulos J,et al.Semeval-2014 task 4:Aspect based sentiment analysis[C].The 8th International Workshop on Semantic Evaluation,2014:27-35.
[27] Dong Li,Wei F R,Tan C Q,et al.Adaptive recursive neural network for target-dependent twitter sentiment classification[C].The 52ndAnnual Meeting of the Association for Computational Linguistics,2014:49-54.
备注/Memo
收稿日期:2024-03-15
基金项目:国家自然青年基金资助项目(61806170)