Firefly: A Low-Resolution Thermal Trigger for Gated Sensor Fusion in Human–Robot Collaborative Manufacturing

Automated guided vehicles and mobile robots are present on many Intelligent Manufacturing factory floors. Approximately 7.0 million U.S. workers also work on those same floors. During 2011 through 2017, forklifts caused 614 worker fatalities. Currently, no commercially available collision-avoidance sensor can manage the complexity associated with working within a plant. Therefore, fusing multiple modalities allows a system designer to gate the more expensive ones behind some minimal signal that indicates all systems should be active. Such minimal signals are expected to include body heat that even stationary personnel would have difficulty suppressing. In addition, the signal-capturing sensors ought to operate at milliwatt levels of power while keeping workers’ privacy. One type of device that has been found suitable for these requirements is a coarse thermopile array. Such a device can be fused with a rangefinder that determines distance to form a gated system. Yet the measurement capability frontier of the thermopile arrays in practical scenarios has not been explored. In addition, there is currently no figure of merit to quantify the trade-ff between induced errors and system-level power saving. In order to address these gaps, we built the fusion system, Firefly, together with a training-free detector and tested them against a human and a mannequin 1.2 to 7 m away in four thermally distinct environments, one of which is the floor of an operating factory. We found that the thermal array’s detection range varies with the environment but is predictable in a thermally uniform environment. Based on these experiments, we also proposed the Gated-Fusion Efficiency (GFE), a figure of merit that converts a trigger’s error rates into system-level power savings. Together, the capability boundary and the GFE help guide sensor-fusion system design in industrial manufacturing toward safer working environments.