Comparing fast semantic segmentation CNNs for FPGAs with standard methods

Abstract In this contribution small semantic segmentation CNNs are evaluated against traditional segmentation approaches and state-of-the-art segmentation CNNs. The CNNs are optimized for the implementation on frame grabber FPGAs. A dataset of industrial burner flames and a dataset of transparent plastic granules is used to assess the segmentation performance of the models. VisualApplets by Basler AG is used to implement the models on an FPGA. The implemented models reach foreground IoU values of up to 96.8 % on our plastic granule dataset with a resolution of 256 × 256 pixel. In comparison state-of-the-art models evaluated reach values ranging from 94.0 % to 98.0 %. For images upscaled to 512 × 512 pixels, the performance of our models decreases to 91.4 % for the best FPGA implemented one. The inference of a 512 × 512 pixel image takes roughly 1 ms with an FPGA. The latency between the start of an input line to the output of the line is 0.1–1.9 ms for streaming an 8,192 pixel wide image. Using a GPU, the inference times on our models range between 0.59 ms and 1.03 ms, while the fastest state-of-the-art model reaches 3.44 ms.