Multi-Task Learning 1997–2024: Part III Applications

In Parts I <https://doi.org/10.1162/99608f92.462f48a2> and II <> of this article, we introduced the fundamentals of multitask learning (MTL) and the technical foundations of regularization and optimization methods that facilitate information sharing across tasks. This Part III focuses on the practical application of MTL theory in real-world industrial contexts, offering actionable guidance and demonstrating how theoretical concepts are effectively implemented in professional settings. We examine the evolution and application of MTL, tracing its development from the deep learning (DL) phase to the rise of pretrained foundation models (PFMs). In the early DL phase, MTL was achieved by directly sharing features across layers of networks handling different tasks. This approach was later enhanced with attention mechanisms and mixture-of-experts (MoE) architectures to enable more efficient and adaptive feature propagation. However, these methods faced challenges when applied across heterogeneous domains and modalities. Recent advances in PFMs address these challenges by offering a robust foundation for encoding and decoding multimodal data using specialized tokenizers and detokenizers. We explore how MTL has progressed from sharing representations among homogeneous tasks to supporting multimodal, cross-domain learning through pretraining. This part of the survey presents detailed architectural taxonomies and practical insights, showcasing the effectiveness of these models. Furthermore, we highlight the potential of PFMs in task-agnostic pretraining, which promises to redefine the future landscape of MTL.“Multi-Task Learning 1997–2024” is a three-part article. Part I, “Fundamentals” can be read here <https://doi.org/10.1162/99608f92.462f48a2> . Part II, “Regularization and Optimization” can be read here <https://doi.org/10.1162/99608f92.03dd064a> .