Making AI Training Smaller for Edge Devices

Training large models on edge devices poses significant challenges when the available hardware has limited computational power. Simultaneously, there are situations where the data cannot be centralized due to privacy concerns or bandwidth limitations. This research explores adapter-based training in a Split Federated Learning (SFL) environment, using methods pioneered by LLM research such as LoRA and LoftQ. The experiments, conducted on ResNet models, compare and demonstrate the ability of these methods to match or surpass centralized full fine-tuning of pretrained models while using less than 1% of the full training parameters on-client.