CollabCam: Collaborative Inference and Mixed-Resolution Imaging for Energy-Efficient Pervasive Vision

While DNN models have dramatically improved the accuracy of machine vision tasks, pervasive deployments of vision sensors for surveillance tasks continue to suffer from high energy consumption and network traffic overhead. To tackle these problems, we introduce CollabCam , an edge-based machine vision system designed for multi-camera deployments that leverages the naturally-occurring overlaps in the field-of-view (FoV) among neighboring cameras. CollabCam synergistically combines two innovative ideas: (a) having each individual camera compose and transmit mixed-resolution frames (MRF) via lightweight down-sampling, where the transmitted images have significantly lower resolution in the shared, overlapping portions , and (b) performing inference, for an exemplar object detection task, for each camera stream using a new collaborative mechanism which utilizes suitably-translated object bounding boxes from a peer “collaborating” camera as an additional input channel. We demonstrate how this collaborative mechanism is generalizable and can be realized by simply retraining off-the-shelf object detector DNNs, such as YOLOv3 and SSD, without modifying their model structures. By emulating the performance of CollabCam using two benchmark outdoor-campus multi-camera datasets, we show that Collab-DNNs can accommodate a 50–60 fold reduction in image size (therefore reducing network transmission overhead), for both high-resolution (1056x1056) and low-resolution (512x512) images, with a modest ≤ 2 - 5% drop in object detection accuracy, compared to a non-collaborative approach that suffers a ∼ 45–60% drop in accuracy. Subsequently, by deploying a Raspberry-Pi based CollabCam prototype on a campus-based test-bed, we demonstrate that CollabCam can reduce the overall energy/image frame overhead by ∼25–35%, with even higher energy savings (∼35–45%) likely with hardware optimization. Finally, additional experiments help demonstrate that CollabCam can prove beneficial for varied (including multi-class) object detection tasks and that CollabCam’s performance benefits may be best realized by ensuring that the number of deployed, collaborating cameras is not excessively high.