AutoUMA: Accelerated ML Compiler Support for Hardware Accelerators based on Automatic Kernel Library Generation

The rapid development of domain-specific deep neural network (DNN) accelerators, particularly spatial architectures such as systolic arrays, has become essential for improving power–performance efficiency in edge computing. However, despite advances in hardware design tools, deploying deep learning models on custom spatial accelerators remains a major bottleneck. Existing machine learning (ML) compilers are primarily designed for GPUs and CPUs, making the integration of new accelerators complex and labor-intensive due to diverse programming interfaces and architecture-specific execution models. This lack of a unified deployment interface limits the scalability and generality of current compiler toolchains. In this work, we address this challenge by shifting the abstraction from accelerator-specific interfaces to kernel functions, the fundamental building blocks for executing DNN workloads. We present AutoUMA, an automated compilation framework that externalizes kernel generation and introduces an AutoScheduler to automatically derive optimized kernels from accelerator mappings. The scheduler performs structured transformations, including multi-level loop tiling, memory transfer staging, and instruction emission, and incorporates a guard mechanism to eliminate redundant memory operations. In addition, we introduce AutoUMA ’s compiler retargeting pipeline that leverages kernel functions as a uniform, coarse-grained deployment interface and abstracts low-level modifications through high-level accelerator descriptions. We evaluate our framework on the Gemmini accelerator and demonstrate that it significantly reduces integration effort while generating efficient kernel libraries. Our approach achieves speedups of up to 4.4× over manually developed static libraries in low-level C and delivers performance competitive with expert-optimized kernel implementations using high-level scheduling languages.

Authors

Institutions

Publication Details

Journal
ACM Transactions on Architecture and Code Optimization
Published
2026-11-13
DOI
https://doi.org/10.1145/3862857
Primary Topic
Parallel Computing and Optimization Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

AutoUMA: Accelerated ML Compiler Support for Hardware Accelerators based on Automatic Kernel Library Generation

Daniel Mueller-Gritschneder, Ulf Schlichtmann, Samira Ahmadifarsani
2 links
ACM Transactions on Architecture and Code Optimization
Parallel Computing and Optimization Techniques
article

AutoUMA: Accelerated ML Compiler Support for Hardware Accelerators based on Automatic Kernel Library Generation

Daniel Mueller-Gritschneder, Ulf Schlichtmann, Samira Ahmadifarsani
article en

Abstract

The rapid development of domain-specific deep neural network (DNN) accelerators, particularly spatial architectures such as systolic arrays, has become essential for improving power–performance efficiency in edge computing. However, despite advances in hardware design tools, deploying deep learning models on custom spatial accelerators remains a major bottleneck. Existing machine learning (ML) compilers are primarily designed for GPUs and CPUs, making the integration of new accelerators complex and labor-intensive due to diverse programming interfaces and architecture-specific execution models. This lack of a unified deployment interface limits the scalability and generality of current compiler toolchains. In this work, we address this challenge by shifting the abstraction from accelerator-specific interfaces to kernel functions, the fundamental building blocks for executing DNN workloads. We present AutoUMA, an automated compilation framework that externalizes kernel generation and introduces an AutoScheduler to automatically derive optimized kernels from accelerator mappings. The scheduler performs structured transformations, including multi-level loop tiling, memory transfer staging, and instruction emission, and incorporates a guard mechanism to eliminate redundant memory operations. In addition, we introduce AutoUMA ’s compiler retargeting pipeline that leverages kernel functions as a uniform, coarse-grained deployment interface and abstracts low-level modifications through high-level accelerator descriptions. We evaluate our framework on the Gemmini accelerator and demonstrate that it significantly reduces integration effort while generating efficient kernel libraries. Our approach achieves speedups of up to 4.4× over manually developed static libraries in low-level C and delivers performance competitive with expert-optimized kernel implementations using high-level scheduling languages.

ACM Transactions on Architecture and Code Optimization
TU Wien (AT), Technical University of Munich (DE)
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon