Quantization of KLT matrices using a GMRF model for image blocks with application to adaptive transform coding

Abstract Forward adaptive transform coding of images requires a codebook of transform matrices from which the best transform can be chosen for each macroblock. Codebook construction is a problem of designing a quantizer for Karhunen–Lóeve transform (KLT) matrices estimated from sample image blocks. We present a novel method for KLT matrix quantization based on a finite-lattice non-causal homogeneous Gauss–Markov random field (GMRF) model with asymmetric Neumann boundary conditions for blocks in natural images. The matrix quantization problem is solved in the GMRF parameter space, simplifying the harder problem of quantizing a large matrix subject to an orthonormality constraint to a low-dimensional vector quantization problem. Typically used GMRF parameter estimation methods such as maximum-likelihood (ML) do not necessarily maximize the coding performance of the resulting transform matrices. To this end we propose a method for GMRF parameter estimation from sample image data, which maximizes the high-rate transform coding gain. We also investigate the application of GMRF-based transforms to variable block-size adaptive transform coding.