Block-gequantiseerd en data-efficiënt deep learning aan de rand: Optimalisatie van implementaties met universele en domeinspecifieke vectorinstructiesets

The demand for Artificial Intelligence (AI) continues to grow, leading to it becoming an integral aspect in our everyday lives. This surge can largely be attributed to the remarkable progress in Deep Learning (DL), which has consistently pushed performance boundaries often even outperforming human capabilities. Recent trends have focused on performance improvements through two major scaling axes: training data and model sizes. However, such scaling is often infeasible for practical DL on edge devices targeting custom and personalized tasks. This has driven research into data-efficient algorithms, which aim to deliver high task performance with limited data, addressing scenarios where traditional DL algorithms may falter. While data-efficient algorithms tackle the challenge of data availability, they often overlook the stringent resource constraints inherent to edge devices. Additionally, model adaptation at the edge is often required to effectively handle data drift, distribution mismatches, etc., further complicating deployments. Block-quantization is used in this thesis to enable efficient computation at the edge, while maintaining algorithmic performance even at reduced-precision. The algorithmic landscape for these workloads is evolving rapidly, motivating the use of instruction-programmable processors that are resilient to obsolescence and also avoid high customization design costs. Moreover, improved performance and efficiency of deployments can be achieved through the use of vector instructions that are suitable for diverse data-parallel workloads predominantly found in DL. This thesis firstly explores the data-efficient DL modality of few-shot image classification task, which overcomes the challenge of learning a new unseen task with very limited training data, as low as 1 or 5 samples per class. To enable efficient DNN adaptation on resource-constrained edge devices, block-quantization is used during the training process in both the forward and backward passes. It is also shown that conventional meta-learning algorithm initializations used for such tasks fail to transfer well for these quantized adaptations and can be improved by a HW-aware meta-learning algorithm that upfront includes the target HW constraints for DNN adaptations. In the next part, the focus is still on data-efficiency but targeting a different modality of continuous control tasks such as robotic control. The objective here is to attain effective task control while not requiring large number of environmental interactions. The usage of block-quantized DNNs as dynamics models is introduced here, leading to more efficient inference deployments, and in turn more HW-efficient planning in a conventional model-based RL algorithm. A systematic study of the impacts of quantization error on the degradation of control performance is performed, which typically exacerbates with any model prediction inconsistencies due to the effect of compounding error in its future predictions. Optimizing such a process while still maintaining effective control is achieved through flexible block configurations and utilizing higher precision only for more sensitive computations. Next to these algorithmic impacts of block-quantization, the exploration of the effect of running them on instruction-programmable processors is also performed. Block-scaled GeMM kernels are designed and optimized across multiple design parameters, with only the use of general-purpose vector instructions. System-level analysis is done to benchmark these kernels against standard datatypes. The impact of increasing block sizes for the same workload leads to decreasing cycle counts due to lower exponent handling overhead. Similarly, 2D blocks are introduced as opposed to conventional 1D blocks, also providing increased performances. To reduce the cost of memory transfers, effective fused requantization has also been designed. These kernels were also evaluated for usage in a DNN training workload. A co-optimized process of adaptive training with vector and square block shapes is proposed to trade off GeMM computational speeds and accuracy. Lastly, the deployments using domain-specific vector instructions were explored for block-quantized DNN computations with a software-defined SIMD HW unit. Such a HW unit enabled high flexibility for runtime variable-precision computations exhibiting different data-level parallelism, that can be effectively utilized for DNNs. Several mixed-precision inference deployments of CNNs were analyzed, providing several HW-SW trade-off optimization potential. The contributions of this thesis help push the optimization boundaries of deployments of data-efficient DL algorithms at the edge, using general to domain-specific vector instruction sets, with the use of block-quantization techniques. The progress of research in this interdisciplinary intersection is crucial to enable futuristic applications of bringing intelligence to everyday devices in a performant and privacy-driven manner.