Private linear equation solving: An application to federated learning and extreme learning machines

In federated learning, multiple devices compute each a part of a common machine learning model using their own private data. These partial models (or their parameters) are then exchanged in a central server that builds an aggregated model. This sharing process may leak information about the data used to train them. This problem intensifies as the machine learning model becomes simpler, indicating a higher risk for single-hidden-layer feedforward neural networks, such as extreme learning machines. In this paper, we establish a mechanism to disguise the input data to a system of linear equations while guaranteeing that the modifications do not alter the solutions, and propose two possible approaches to apply these techniques to federated learning. Our findings show that extreme learning machines can be used in federated learning with an extra security layer, making them attractive in learning schemes with limited computational resources. • We develop a general method (see Theorem 2 ) to disguise the input data, that is, H and Y , to a general linear equation system H ⋅ B = Y . The line of research is started by investigating additive changes to H (later in Lemma 9 : S = H + R ) and, following the mathematical implications, leads to orthogonal projections onto proper subspaces. In essence, instead of H we obtain a matrix consisting of projected columns of H . • We apply this new method of disguising input data to linear equation systems to the setting of federated learning of extreme learning machines in Theorem 5 and Theorem 6 • A beneficial side effect of the orthogonal projections involved in our method is that the amount of data transmitted from clients to the server is reduced. • The leakage of information of the traditional scheme of sending the data to compute the normal equations, that is, H T H and H T Y , is characterized in Theorem 7 and compared exemplarily to our method in Section 4 .