Using adaptive learning and momentum to improve generalization

Abstract In this paper, we propose a novel algorithm called adaptive learning & momentum sharpness-aware minimization MAML (AS-MAML) that builds upon the concept of sharpness-aware minimization and adaptive learning and momentum to improve generalization. We prove theoretically by performing convergence analysis and PAC-Bayes analysis that AS-MAML performs better than the state-of-the-art algorithms in model-agnostic meta-learning. We draw the same conclusion through extensive experimental analysis using benchmark datasets.