ACache: An Importance-Based Cache with Augmentation Cutting for Accelerating DNN Training

In deep learning(DL) training, unique IO characteristics are observed. Although many accelerators (GPUs, FPGAs) exist for DL training, these devices are primarily used for neural network computation graphs. For some IO-intensive DL training tasks, IO becomes a bottleneck. Data augmentation is a data processing method, which helps to improve the diversity of data and the generalization of the model, it creates tremendous repeated computations. At the same time, DL frameworks treat each sample equally, even though the sample contribution to the model varis during training. Existing training frameworks do not leverage these features to build cache layers that reduce reading samples from low-speed storage devices. To address these challenges, we propose ACache, a caching system built using importance sampling. The workload caused by data augmentation is optimized to minimize the interference of repeated computations on training. ACache will update the sampling factors periodically, which are updated asynchronously throughout the training process. Additionally, it implements a fine-grained cache replacement strategy to determine whether to replace the cache for each sample. We evaluated ACache on a representative computer vision task and found that, compared to the current state-of-the-art DL Cache system, ACache is capable of training models at a faster rate.