Face Expression Recognition on Ojek Online for Safety Improvement Using Deep Learning

During COVID-19, many workers were laid off because the company was struggling with financial challenges. Hence, laid-off workers have switched professions to become Ojek Online (Ojol) drivers because they have the flexibility to work with adequate income. In addition, the COVID-19 period has changed people’s behavior to choose Ojol because it is easy and practical. The growing number of demands has the risk of accidents due to fatigue and lack of adequate rest time. This study aims to implement deep learning methods such as Convolutional Neural Network (CNN), Bidirectional Long-Short Term Memory (BiLSTM), and Bidirectional Gated Recurrent Unit (BiGRU) to minimize the accident risk with face expression recognition. It will also compare the three methods to find the best method. About 350 primary data were collected from the participants’ questionnaires, grouped into seven facial expression classes: angry, sad, happy, sleepy, surprised, afraid, and neutral. Afterwards, the augmentation was implemented to solve the imbalance dataset problem. At the EDA stage, the positive correlation between object size, area, and resolution received a score of 1, while the negative correlation of the attribute’s light intensity and saturation was - 0.428. The result of performance evaluation on the three methods is that the BiGRU method achieved the best value with accuracy, precision, recall, and F1-score values, respectively, 0.67, 0.69, 0.71, and 0.69. From this study, BiGRU is recommended as the best method. In addition, increasing the number of datasets and facial expression groups is necessary, as well as combining deep learning methods with evolutionary algorithms to find the best parameters to improve model performance automatically.