Next Generation Triggers ML Optimization: Quantization Techniques for Scalable Inference

Quantization is the use of a lower precision for the model parameters. It is a useful technique, as it lowers significantly the size of a model, while trying to preserve the accuracy. Some runtimes like ONNX offer the possibility to do dynamic quantization. The tasks will be to explore and evaluate the different techniques for model quantization to offer users a view of their model metrics under such a compression. The outcomes of this project will provide decision support for the users whether to use quantization for inference and/or training.