Evaluating broad area search and classification of heavy construction activity from multisource, multitemporal satellite image sequences

The IARPA Space-Based Machine Automated Recognition Technique (SMART) program recently posed a new challenge for computer vision in satellite imagery - searching for and characterizing anthropogenic activity, i.e. gradual changes to the Earth’s surface caused by humans. While numerous solutions exist for detection and classification of objects in single images, characterizing activity requires spatio-temporal reasoning over long time scales. Given the novelty of this problem, standard performance metrics and benchmarks had not yet been established. This work proposes a suite of application-generic performance metrics for detecting and classifying sites where activities of interest have taken place, and presents initial results of several different algorithms aimed at spatially and temporally localizing and classifying heavy construction activity. An implementation of these performance metrics along with sample system outputs is available with the annotation dataset on Github¹ and IEEE DataPort.²