A federated approach towards fine-tuning LLMs for abstractive summarization

A decentralized machine learning method known as federated learning (FL) for model-training has been discussed in this paper. FL is a subset of machine learning in which multiple clients(edge devices) train parts of a large dataset in a decentralized manner. This is a different approach to traditional machine learning in which data is centrally stored. The purpose of this study is to implement FL in a critical use case of summarizing legal papers. This study contrasts FL with centralized learning strategies by utilizing the Flower architecture. Flower is an open-source Python framework that simplifies FL tasks on a cluster of machines. The primary aim of this research is to achieve competitive summary quality metrics in legal document summarization. This work highlights the promise of FL as a scalable method to protect privacy in decentralized contexts during deep learning model training. The outcomes are on par with centralized procedures, and the least amount of money needs to be spent on model training.