Comparison of Vision Transformer and ResNet under Federated Learning in Human Action Recognition

This study investigates the adaptability of ResNet50 and Vision Transformer (ViT) in Federated Learning (FL) versus centralized training for human action recognition, addressing data privacy and distributed data silos in real-world applications. Employing the Kaggle dataset (about 12,000 images across 15 actions), the thesis standardized preprocessing (bilinear resizing, flip/rotation augmentation, ImageNet normalization) and conducted controlled experiments. Centralized training utilized full-batch gradient descent, while FL simulated non-independent and identically distributed (non-IID) data partitioning with Federated Averaging (FedAvg) aggregation. The results indicate that ViTs achieved 87.30% accuracy in FL, surpassing Residual Network (ResNet) by 8.01%. The train-validation gap was significantly reduced (1.67% vs. 11.35%), revealing improved generalization. Overfitting was limited through phased aggregation, which also enabled late-stage convergence exceeding. ViTs global attention proved more adaptable to heterogeneous data distributions, and FLs distributed optimization enhanced model resilience. These results highlight the complementary strengths of ViT and FL, especially for privacy-sensitive applications like healthcare, where preserving data sovereignty is essential.