Enhancements of FLEX Hyperspectral Data Compression Using High-Performance Embedded Space Computing

This paper describes the enhancement of a data compression computing system that operates on imaging spectrometer data. The implementation is part of the focal plane interface electronics-digital (FPIE-D) and includes AMD System-on-Chip (SoC) Zynq-based and KU060-based versions. JPL imaging spectrometers can acquire data with 640, 1280, or 3000 cross-track positions with 328 or 480 bands at up to 216 images per second. Uses for imaging spectrometer data include studying the mineral dust cycle, methane detection, analysis of biodiversity, and fire detection. We have implemented the data compression system on an AMD Zynq-based space-qualified custom FPIE-D SoC commercial-off-the-shelf (COTS) Alpha Data 7z1 hardware, which fits into a 120mm by 190mm by 40mm assembly that uses 9 watts at peak performance. The computing element is a Xilinx Zynq Z7045Q SoC, which includes a Kintex-7 FPGA and two ARM Cortex-A9 processors. We have also implemented the data compression system on the AMD KU060-based FPIE-D FPGA COTS Alpha Data KU1 hardware PCIe board. The changes to the Fast Lossless Extended (FLEX) data compression block (a modified implementation of the CCSDS-123.0-B-2 standard) compared to the implementation used for the Earth Surface Mineral Dust Source Investigation (EMIT) mission were: (1) Band Interleaved by Line (BIL) scan order (instead of Band Interleaved by Pixel (BIP)), (2) narrow local sums, (3) a modification to use ‘stale’ prediction weights, (4) a minor modification to the predictor, and (5) a pipelined predictor-quantizer core that was made possible by the other changes. These changes together do not adversely affect the achieved compression ratio. The motivation for the ‘stale weights' was that with BIL scan order and narrow local sums, the main obstacle to pipelining the prediction calculation in an FPGA implementation is that the weight update calculation requires the sign of the prediction error for the previous sample. By using weights that are a few samples old, it is possible to pipeline the prediction calculation. The enhancement provides an increase in data throughput by a factor of 2.5 compared to the EMIT implementation. Specifically, the FPIE-D SoC 7z1 achieves a data throughput of 60 MSamples/sec (compared to 25 MSamples/sec for EMIT), allowing real-time data compression for a JPL imaging spectrometer similar to EMIT (40 MSamples/sec), using a 3 FLEX BIL core implementation. The FPIE-D FPGA KU1 board achieves a data throughput of 218 MSamples/sec using an 11 FLEX BIL core implementation, which is the maximum number of cores based on the resources available on the KU060. The FPGA implementation was stress-tested on the FPIE-D FPGA KU1 board using different image patterns, image sizes (number of cross-tracks from 32 to 1280 in increments of 32, and number of bands from 32 to 480 in increments of 8), and compression fidelity parameters (maximum error values varying by band and randomly ranging from 0 (lossless) to 1023). Over 20 million images of different sizes, totaling 23 terabytes (TB) of data, were tested. We also demonstrated using Alpha Data performance analysis FPGA IPs, that the limited PCIe data bandwidth (1.4 GBytes/sec) does not affect the compression data throughput, allowing the host processors to perform real-time acquisition and downlink while the FPGA simultaneously performs real-time data compression. Finally, we stress-tested a 1-core implementation on the FPIE-D SoC 7z1 board and verified the functional test on a 3-core implementation.