Law-Smooth Update Scheme for the Cross-Entropy Optimization Algorithm And Applications

It is well known that the Cross-Entropy (CE) algorithm, based on Gaussian distributions family, is significantly less efficient than successful methods such as CMAES or even particle-based approaches such as particle swarms. Nevertheless, the Gaussian-based CE approach implements similar ingredients as the CMAES approach, but it is significantly penalized by its law updating step after sample selection. Variants of the CE approach have been proposed, featuring a smoothing of the distribution parameters. We show why these approaches by smoothing law parameters may result in wrong convergence, and propose an approach based on smoothing laws. We implement this CE updating approach and show that it gets close to the performance of CMAES on an applicative example. Our application concerns the robust optimization of band selection for anomaly detection in multispectral remote sensing images. As it is important to take into account the variability in the scenes observed, as well as the diversity of objects likely to be encountered, we have drawn on robustness measures, such as quantile, to quantify the variations in anomaly detection criteria over the 100 hyperspectral images of the benchmark dataset. Finally, we also explore a generalization of this CE update to laws of the exponential family.