LIU Xueming,DU Zhibo.An Attack Language Recognition and Classification Method based on Bayesian Optimization BERT-BiLSTM Model[J].Journal of Chengdu University of Information Technology,2025,40(03):294-299.[doi:10.16836/j.cnki.jcuit.2025.03.006]
基于贝叶斯优化BERT-BiLSTM模型的攻击性语言识别与分类方法
- Title:
- An Attack Language Recognition and Classification Method based on Bayesian Optimization BERT-BiLSTM Model
- 文章编号:
- 2096-1618(2025)03-0294-06
- Keywords:
- BERT model; BiLSTM model; Bayesian optimization
- 分类号:
- TP309.2
- 文献标志码:
- A
- 摘要:
- 当前基于BERT模型的攻击性语言的识别与分类方法中存在特征稀疏和上下文关联性少的问题,影响攻击性语言识别与分类的准确性,并且在参数优化方面存在人工优化费时费力、成本高、效果差等问题。为此,提出一种基于BERT-BiLSTM模型的攻击语言识别方法,并利用基于概率寻优的贝叶斯优化方法解决超参数优化问题。首先通过BERT模型训练攻击性语言数据集并提取数据集中的攻击性词特征,之后再使用BiLSTM模型捕获深层次的上下文关联性,最后将获得的特征向量输入到回归模型中进行分类。经过对CLODataset中文数据集的测试,并将BERT模型和BiLSTM模型进行对比实验,证明该方法有效地捕获序列特征和上下文信息,从而提升文本分类性能,使模型在测试集上的F1值提升了0.11。
- Abstract:
- The current recognition and classification methods for aggressive languages based on the BERT model suffer from sparse features and low contextual relevance, which affects the accuracy of aggressive language recognition and classification. In addition, there are problems such as time-consuming and laborious manual optimization, high cost, and poor performance in parameter optimization. A method for attack language recognition based on the BERT-BiLSTM model is proposed, and a Bayesian optimization method based on probability optimization is used to solve the hyperparameter optimization problem. Firstly, the aggressive language dataset is trained using the BERT model and the aggressive word features are extracted from the dataset. Then, the BiLSTM model is used to capture deep-level contextual correlations. Finally, the obtained feature vectors are input into the regression model for classification. After testing the CLODataset Chinese dataset and comparing the BERT model with the BiLSTM model, it was proven that this method effectively captures sequence features and contextual information, thereby improving text classification performance and increasing the F1 value of the model by 0.11 on the test set.
参考文献/References:
[1] Jahan M S,Oussalah M.A systematic review of Hate Speech automatic detection using Natural Language Processing[J].Neurocomputing,2023:126232.
[2] Zampieri M,Malmasi S,Nakov P,et al.Predicting the type and target of offensive posts in social media[J].arXiv preprint arXiv:1902.09666,2019.
[3] Zou H,Tang X,Xie B,et al.Sentiment classification using machine learning techniques with syntax features[C].2015 International Conference on Computational Science and Computational Intelligence(CSCI).IEEE,2015:175-179.
[4] Zampieri M,Malmasi S,Nakov P,et al.Semeval-2019 task 6:Identifying and categorizing offensive language in social media(offenseval)[J].arXiv preprint arXiv:1903.08983,2019.
[5] Zhou L,Cabello L,Cao Y,et al.Cross-Cultural Transfer Learning for Chinese Offensive Language Detection[J].arXiv preprint arXiv:2303.17927,2023.
[6] Zampieri M,Malmasi S,Nakov P,et al.Predicting the type and target of offensive posts in social media[J].arXiv preprint arXiv:1902.09666,2019.
[7] Ahmed M F,Mahmud Z,Biash Z T,et al.Cyberbullying detection using deep neural network from social media comments in bangla language[J].arXiv preprint arXiv:2106.04506,2021.
[8] Althobaiti M J.Bert-based approach to arabic hate speech and offensive language detection in twitter:Exploiting emojis and sentiment analysis[J].International Journal of Advanced Computer Science and Applications,2022,13(5).
[9] El-Alami F,El Alaoui S O,Nahnahi N E.A multilingual offensive language detection method based on transfer learning from transformer fine-tuning model[J].Journal of King Saud University-Computer and Information Sciences,2022,34(8):6048-6056.
[10] 方晓东,刘昌辉,王丽亚,等.基于BERT的复合网络模型的中文文本分类[J].武汉工程大学学报,2020,42(6):5.
[11] 夏林中,叶剑锋,罗德安,等.基于BERT-BiLSTM模型的短文本自动评分系统[J].Journal of Shenzhen University Science & Engineering,2022,39(3).
[12] Ran X,Shan Z,Fang Y,et al.An LSTM-Based Method with Attention Mechanism for Travel Time Prediction[J].Sensors,2019,19(4):861.
[13] Graves A,Jürgen Schmidhuber.Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J].NeuralNetworks,2005,18(5/6):602-610.
[14] Deng J,Zhou J,Sun H,et al.Cold:A benchmark for chinese offensive language detection[J].arXiv preprint arXiv:2201.06025,2022.
[15] 崔佳旭,杨博.贝叶斯优化方法和应用综述[J].软件学报,2018,29(10):3068-3090.
[16] Wang Z,Jegelka S,Kaelbling L P,et al.Focused model-learning and planning for non-Gaussian continuous state-action systems[C].2017 IEEE International conference on robotics and automation(ICRA).IEEE,2017:3754-3761.
[17] 杨欢,吴震,王燚,等.侧信道多层感知器攻击中基于贝叶斯优化的超参数寻优[J].计算机应用与软件,2021,38(5):323-330.
[18] 魏佳恒,郭惠勇.基于贝叶斯优化BiLSTM模型的输电塔损伤识别[J].振动与冲击,2023,42(1):238-248.
[19] Vidgen B,Derczynski L.Directions in abusive language training data,a systematic review:Garbage in,garbage out[J].Plos one,2020,15(12):e0243300.
备注/Memo
收稿日期:2023-12-27
基金项目:四川省科技计划资助项目(2021ZYD0011)
通信作者:杜之波.E-mail:duzb@cuit.edu.cn