Attention-based BiLSTM with positional embeddings for fake review detection

The proliferation of fake reviews has notably impacted consumer decision-making. However, due to the diversity and subtlety of fake reviews, traditional sequential models face challenges in capturing relative positional information and suffering from the loss of critical information of long texts. Therefore, we propose PosAtt-BiLSTM, a Bidirectional Long Short-Term Memory Neural Network (BiLSTM) model with a Hybrid Attention Mechanism (HAM) based on the Non-Negative Sinusoidal Positional Encoding (NN-SPE) for fake review detection. Firstly, NN-SPE modifies the traditional sinusoidal positional encoding by adjusting the sine and cosine functions to ensure that the encoding values are non-negative, and this positional information is then merged with the word embeddings. Then, the sequence model BiLSTM is employed to extract sequential features from the text, while HAM is applied to effectively weight both global and local features, ensuring the accurate capture and transmission of key information. Three public datasets Spam, Yelp Hotel, and Yelp Restaurant are selected. The classification results demonstrate that PosAtt-BiLSTM model outperforms traditional models such as BiRNN and BiLSTM across all datasets. Furthermore, the results of different positional encoding methods reveal that NN-SPE leads to superior performance than other positional encoding techniques.