Identifiering av Signal For Help-gesten på obemannade autonoma bussar : En undersökning av lämpliga modaliteter och utvärdering av ett nytt ramverk för gestigenkänning i video

There is financial and social interest in the notion of unmanned, autonomous buses, which could help improve availability of public transit and reduce reliance on personal automotive vehicles. One challenge with this is the need for a point of contact and personal security on board a bus without a driver who, in addition to driving, is also a representative of the transit company capable of answering questions and maintaining a degree of oversight on the activities of the passengers. This thesis contains a comparison on the advantages of using visual and auditory signals, concluding that a gesture is the most suitable signaling form largely due to being possible to perform discretely. A framework then is proposed using YOLOv11 for hand detection, MediaPipe hand landmark detection for feature extraction, and a LSTM for gesture recognition. YOLO and the LSTM model is trained for the specific task of detecting the Signal For Help (SFH) gesture. The training is conducted using several publicly available datasets for hand detection, as well as data collected in a laboratory. It then implements and assesses a gesture recognition system based on this framework, concluding that while the prototype created is decidedly not sufficiently accurate to be practically useful, it seems like current technology does have the potential to achieve good results, with sufficient investment and good access to data. Finally, there is a discussion on the factors that were found to impact the system’s performance coupled with suggestions for future research.