Modular Methods for Continuous and Efficient Adaptation of Large Language Models

This thesis investigates modular methods as alternatives for achieving continuous and efficient adaptation in Large Language Models (LLMs). The immediate goal of this research is to develop and empirically validate novel modular techniques that overcome the inherent limitations of monolithic LLM architectures. The broader goal is to advance the conceptual understanding and practical realization of modularity as a foundational principle for building a new generation of adaptable and scalable AI systems. The modular methods explored in this dissertation are structured around the core concepts of continual learning, efficiency, and composition. First, this work establishes the foundations of modularity in continual learning. Through ExSSNeT (Exclusive Supermask Subnetwork Training), I demonstrate how modular subnetworks and masking can enable specialization and mitigate catastrophic forgetting in sequential learning scenarios, laying the groundwork for more complex modular architectures. Extending this investigation, the thesis explores continual learning for code generation models, showcasing the versatility of modular prompting techniques, and introducing PP-TF (Prompt Pooling with Teacher Forcing) as a novel adaptation strategy for this domain. Second, the thesis addresses the critical aspect of efficiency in modular LLMs. ComPEFT, a novel compression technique for Parameter-Efficient Fine-tuning (PEFT) updates, is presented as a solution to drastically reduce communication and storage overhead. This method leverages sparsification and quantization to create highly efficient modules, enabling the practical deployment of modular adaptation strategies at scale. Lastly, this research culminates in the exploration of advanced modular composition for LLMs. Ties-Merging, a novel model merging technique, is introduced to resolve interference and enhance the performance of statically composed expert models. Moreover, we study the scalability of model merging methods and find that they can be used to adapt LLMs at scale. Furthermore GLIDER (Global and Local Instruction-Driven Expert Router) is proposed as a dynamic routing mechanism that leverages LLM-generated instructions to enable intelligent and adaptive composition of expert modules at inference time, showcasing a pathway to scalable and generalizable modular LLM systems. In summary, this dissertation advances the field by establishing a comprehensive framework for modular methods, demonstrating their potential to enable continuous and efficient adaptation, and offering a compelling vision for a future where AI systems are inherently more flexible, scalable, and adaptable to the ever-evolving demands of real-world applications.