Towards Vector Encoding Methods on GPUs for Dictionary Search Memory Optimization

Motivation: GPU memory size can be a limiting factor for MRF dictionary search. Accelerated grouped search on GPUs was implemented previously without addressing GPU memory constraints. Goal(s): To investigate the feasibility of GPU memory footprint reduction through vector storage format compression. Approach: We built a CPU-only prototype search engine storing dictionaries as approximations enabling 2x and 4x memory reduction. Additionally, we implemented a post processing step that refines approximate results using the non-approximated vectors. Results: Refinement was effective at mitigating errors from vector compression. Overall, 2x group memory compression resulted in no quality loss and speed gains, while 4x compression resulted in speed loss. Impact: Dictionary search errors from vector storage compression of up to 4x have been found to be well mitigated by an uncompressed refinement step. This indicates the feasibility of implementing compression on GPUs to make efficient use of limited GPU memory.