Gesture-Controlled Drone: A Vision-Based Approach Using Coppeliasim, Raspberry Pi and Pixhawk

This work showcases the design of a computer vision and embedded system-based gesture-controlled drone system. The design involves two stages: simulation in CoppeliaSim and realtime implementation with Raspberry Pi and Pixhawk. In the simulation stage, a laptop webcam records live hand gestures, which are processed by OpenCV and MediaPipe. Identified gestures are translated to movements of the drone and sent to the simulated quadcopter through CoppeliaSim's remote API. This stage provides for clean motion transitions and optimizes gesture recognition accuracy prior to deployment in the real world. During this hardware implementation stage, the identical gesture recognition method is implemented on a physical drone. Wireless handout of detected gestures to a Raspberry Pi converts them into flight instructions for the Pixhawk controller through MAVLink. Flight parameters are configured, telemetry monitored, and drone performance assessed before and after missions with Mission Planner. It provides flight stability and security in real-time operations. Experimental results confirm the efficacy of gesturebased UAV control, providing an interactive alternative to conventional remote controllers. Future improvements will be on response time, gesture recognition, and maneuverability optimization for improved efficiency.