ZHANG Biyi,TAO Hongcai.Sentiment Analysis of Chinese Film Review based on XLNet-BiLSTM Model[J].Journal of Chengdu University of Information Technology,2021,36(03):264-269.[doi:10.16836/j.cnki.jcuit.2021.03.004]
基于XLNet-BiLSTM模型的中文影评情感分析
- Title:
- Sentiment Analysis of Chinese Film Review based on XLNet-BiLSTM Model
- 文章编号:
- 2096-1618(2021)03-0264-06
- Keywords:
- sentiment analysis; word vector; XLNet; BiLSTM
- 分类号:
- TP391.12
- 文献标志码:
- A
- 摘要:
- 对影评进行情感分析可以为用户观看电影提供一定的参考意见。针对Word2Vec等静态词向量技术不能学习文本的深层信息、解决一词多义及RNN存在的长期依赖和上下文深层语义挖掘不充分的问题,提出一种基于XLNet-BiLSTM的中文影评情感分类模型。首先,使用XLNet预训练语言模型生成具有上下文依赖的词向量来对影评信息进行分布式表征; 然后,将词向量输入到BiLSTM网络中,对评论的深层语义进行分析和计算; 最后,使用softmax函数实现影评情感极性分类。通过爬取豆瓣电影上的评论对模型进行训练和测试,实验结果表明,模型的准确率为0.924,损失率为0.184,相比于相关的情感分析模型取得了更好的效果。
- Abstract:
- Sentiment analysis of film reviews can provide some reference for users to watch movies.The word vector technology such as Word2Vec can’t learn the deep representation of text and solve the polysemy of the word,and the RNN can’t fully exploit the deep semantics of context and has the characteristics of long-term dependence. In order to solve these problems, this paper proposed a Chinese film review sentiment classification model based on XLNet-BiLSTM neural network.Firstly,we use the XLNet model to generate a context-dependent word vector for distributed representation of the information,and then input the word vectors into the BiLSTM network to analyze and calculate the deep semantics of comments.Finally,we used the softmax function to classify the sentiment polarity.The model is trained and tested by reviews in Douban film.The experimental results show that the accuracy of the model is0.924 and the loss rate is 0.184,which is better than the related sentiment analysis model.
参考文献/References:
[1] 陈珂,梁斌,柯文德,等.基于多通道卷积神经网络的中文微博情感分析[J].计算机研究与发展,2018,55(5):945-957.
[2] Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrase and their compositionality[C].Advances in Neural Information Processing Systems,2013:3111-3119.
[3] Kiros R,Zhu Y,Salakhutdinov R R,et al.Skip-thought vectors[C].Advances in Neural Information Processing Systems,2015:3294-3302.
[4] Debg L,YU D.Deep learning:methods and applications[J].Foundations and Trends in Signal Processing,2014,7(3/4):197-387.
[5] Yang X,Macdonald C,Ounis I.Using word embeddings in twitter election classification[J].Information Retrieval Journal,2018,21(2/3):183-207.
[6] HarpreetKaur,VeenuMangat,Nidhi.A Survey of Sentiment Analysis techniques[C].2017 International Conference on I-SMAC(IoT in Social,Mobile,Analytics and Cloud)(I-SMAC),IEEE,2017:921-925.
[7] Zvarevashe K,Olugbara O O.A framework for sentiment analysis with opinion mining of hotel reviews[C].2018 Conference on Information Communications Technology and Society(ICTAS),IEEE,2018:1-4.
[8] Kumar A,Sangwan S R,Arora A,et al.Sarcasm Detection Using Soft Attention-Based Bidirectional Long Short-Term Memory Model with Convolution Network[J].IEEE Access,2019,7:23319-23328.
[9] Bandana R.Sentiment Analysis of Movie Reviews Using Heterogeneous Features[C].2018 2nd International Conference on Electronics,Materials Engineering & Nano-Technology(IEMENTech),IEEE,2018:1-4.
[10] 李文江,陈诗琴.基于深度学习的商品评论情感分类研究[J].知识管理论坛,2018,3(6):353-363.
[11] Kim Y.Convolutional Neural Networks for Sentence Classification[C].2014 Conference on Empirical Methods in Natural Language Processing(EMNLP),2014:1746-1751.
[12] 刘龙飞,杨亮,张绍武,等.基于卷积神经网络的微博情感倾向性分析[J].中文信息学报,2015,29(6):159-165.
[13] 梁军,柴玉梅,原慧斌,等.基于深度学习的微博情感分析[J].中文信息学报,2014,28(5):155-161.
[14] Yangsen J Y Z,Tong Y.Study of Sentiment Classification for Chinese Microblog Based on Recurrent Neural Network[J].Chinese Journal of Electronics,2016,25(4):601-607.
[15] Wang X,Liu Y,Sun C J,et al.Predicting polarities of tweets by composing word embeddings with long short-term memory[C].Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing,2015,1:1343-1353.
[16] Zhu X,Sobihani P,Guo H.Long short-term memory over recursive structures[C].Proceedings of the International Conference on Machine Learning,2015:1604-1612.
[17] 王业沛,宋梦姣,王譞,等.基于深度学习的判决结果倾向性分析[J].计算机应用研究,2019,36(2):335-338.
[18] Benglo Y,Ducharme R,Vincent P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2003(3):1137-1155.
[19] Pennington J,Socher R,Manning C D.Glove:Global vectors for word representation[C].Proceedings of the 2014 conference on empirical methods in natural language processing
(EMNLP),2014:1532-1543.
[20] 胡益淮.基于XLNET的抽取式多级语义融合模型[J].通信技术,2020,53(7):1630-1635.
[21] 李舟军,范宇,吴贤杰.面向自然语言处理的预训练技术研究综述[J].计算机科学,2020,47(3):162-173.
[22] Radford A,Wu J,Child R,et al.Language models are unsupervised multitask learners[J].Open AI Blog,2019,1(8):8-9.
[23] Lundervold A S,Lundervold A.An overview of deep learning in medical imaging focusing on MRI[J].ZeitschriftfürMedizinischePhysik,2019,29(2):102-127.
[24] Young T,Hazarika D,Poria S,et al.Recent trends in deep learning based natural language processing[J].IEEE Computational intelligence magazine,2018,13(3):55-75.
[25] Yang Z,Dai Z,Yang Y,et al.Xlnet:Generalized autoregressive pretraining for language understanding[C].Advances in neural information processing systems,2019:5754-5764.
[26] Gong X,Jin J,Zhang T.Sentiment Analysis Using Autoregressive Language Modeling and Broad Learning System[C].2019 IEEE International Conference on Bioinformatics and
Biomedicine(BIBM),IEEE,2019:1130-1134.
[27] 姚贵斌,张起贵.基于XLnet语言模型的中文命名实体识别[J/OL].计算机工程与应用.http://kns.cnki.net/kcms/detail/11.2127.TP.20201120.1251.012.html,2021-01-03.
[28] Kudo T,Richardson J.SentencePiece:A simple and language independent subword tokenizer and detokenizer for Neural Text Processing[C].2018 Conference on Empirical Methods in
Natural Language Processing(EMNLP),2018:66-71.
相似文献/References:
[1]杨 頔,文成玉.结合关联规则的情感分析模型研究[J].成都信息工程大学学报,2019,(05):501.[doi:10.16836/j.cnki.jcuit.2019.05.011]
YANG Di,WEN Chengyu.Research on Emotional Analysis Model based on Association Rules[J].Journal of Chengdu University of Information Technology,2019,(03):501.[doi:10.16836/j.cnki.jcuit.2019.05.011]
[2]陈宏松,安俊秀,陶全桧,等.基于BERT-VGG16的多模态情感分析模型[J].成都信息工程大学学报,2022,37(04):379.[doi:10.16836/j.cnki.jcuit.2022.04.003]
CHEN Hongsong,AN Junxiu,TAO Quanhui,et al.Multi-modal Sentiment Analysis Model based on BERT-VGG16[J].Journal of Chengdu University of Information Technology,2022,37(03):379.[doi:10.16836/j.cnki.jcuit.2022.04.003]
备注/Memo
收稿日期:2021-04-03
基金项目:国家自然科学基金资助项目(61806170)