Hybrid approach sentiment analysis using Transformer-LSTM in the Indonesian language

Sentiment analysis is widely used as a tool to find valuable insight from texts without explicitly expressed. Lots of techniques have already been used to get it but there still have shortcomings in data source or the model strategy itself. Indonesian language approximately has 199 million speakers across the world where 44 million speakers natively. Even with that great number, the resources for the Indonesian language's natural language processing are still limited, and hard to find the perfect way to define sentiment analysis in the Indonesian language. The state-of-the-art sentiment analysis method uses LSTM on a small corpus while the best in town is Transformer where it's easy to transfer learning from the Transformer pre-trained model into specific tasks. From this potential, combining the Transformer pre-trained model with LSTM can be an innovative strategy. This research compared the hybrid model of Transformer-LSTM build using three Indonesian languages’ Transformer-based pre-trained model on sentiment analysis task which can surpass the Transformer model with the highest increased 2.40% accuracy and 3.02% on F1 score.