A study on news headline classification based on BERT-CNN-LSTM model with fusion adversarial training

Due to the high complexity of the BERT model, it is prone to overfitting on small datasets, and short texts usually contain limited sentiment information, which makes them susceptible to noise interference during training. To address these issues, we propose a text categorization model that integrates adversarial training with BERT-CNN-LSTM. First, we incorporate adversarial training techniques into the BERT model, using adversarial samples as inputs. The encoded outputs from the pre-trained BERT model are then used as inputs for both the CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory) models. Finally, we fuse the outputs from the CNN and LSTM and produce classification results through a fully connected layer. We also conducted experimental comparisons with models such as BERT, BERT-CNN and BERT-DPCNN, and the results demonstrate that our proposed model exhibits superior generalization ability in short text classification.