FasTARFlow: Distilling transformer-based autoregressive flows into fast inverse autoregressive models

TARFlow introduces Transformer-based autoregressive flows that overcome the challenges of expressiveness and training in normalizing flows (NFs), achieving state-of-the-art likelihoods and high-quality samples. However, its autoregressive nature results in slow sampling, which limits its practical applicability. In this work, we propose FasTARFlow, a distillation framework that transfers knowledge from a trained TARFlow model, used as the teacher, to a fast and parallelizable inverse autoregressive flow (IAF) model, used as the student. The distilled model maintains competitive sample quality while substantially accelerating inference. We evaluate FasTARFlow on three datasets spanning a broad range of resolutions and content statistics: AFHQ ( ), FFHQ-256, and CIFAR-100. Experiments show that FasTARFlow achieves sample quality comparable to or better than that of the teacher TARFlow across all three datasets while significantly accelerating sampling. Moreover, FasTARFlow is particularly effective in the high-resolution regime, where the autoregressive sampling cost of TARFlow is most severe. These results advance the practicality of NF-based generative modeling.