WildLive: Near Real-time Visual Wildlife Tracking onboard UAVs

Live tracking of wildlife via high-resolution video process-<br/>ing directly onboard drones is widely unexplored and most<br/><br/>existing solutions rely on streaming video to ground sta-<br/>tions to support navigation. Yet, both autonomous animal-<br/>reactive flight control beyond visual line of sight and/or<br/><br/>mission-specific individual and behaviour recognition tasks<br/><br/>rely to some degree on this capability. In response, we in-<br/>troduce WildLive – a near real-time animal detection and<br/><br/>tracking framework for high-resolution imagery running<br/>directly onboard uncrewed aerial vehicles (UAVs). The<br/>system performs multi-animal detection and tracking at<br/><br/>17.81 fps for HD and 7.53 fps on 4K video streams suit-<br/>able for operation during higher altitude flights to minimise<br/><br/>animal disturbance. Our system is optimised for Jetson<br/>Orin AGX onboard hardware. It integrates the efficiency of<br/>sparse optical flow tracking and mission-specific sampling<br/><br/>with device-optimised and proven YOLO-driven object de-<br/>tection and segmentation techniques. Essentially, compu-<br/>tational resource is focused onto spatio-temporal regions<br/><br/>of high uncertainty to significantly improve UAV process-<br/>ing speeds. Alongside, we introduce our WildLive dataset,<br/><br/>which comprises 200K+ annotated animal instances across<br/><br/>19K+ frames from 4K UAV videos collected at the Ol Pe-<br/>jeta Conservancy in Kenya. All frames contain ground truth<br/><br/>bounding boxes, segmentation masks, as well as individ-<br/>ual tracklets and tracking point trajectories. We compare<br/><br/>our system against current object tracking approaches in-<br/>cluding OC-SORT, ByteTrack, and SORT. Our multi-animal<br/><br/>tracking experiments with onboard hardware confirm that<br/>near real-time high-resolution wildlife tracking is possible<br/>on UAVs whilst maintaining high accuracy levels as needed<br/>for future navigational and mission-specific animal-centric<br/>operational autonomy. We publish all source code, weights,<br/>dataset, and labels for easy utilisation by the community.