Development of federated learning neural networks with combined horizontal and vertical data partitioning

Federated learning trains a global machine learning model without moving data that is distributed geographically, preserving privacy. Most of the existing works are on either horizontal- or vertical-federated learning. This paper proposes two new federated learning frameworks, addressing different data set-ups, for combined horizontal and vertical data-partitioned scenarios based on neural networks. The data is horizontally partitioned for a subset of clients, referred to as horizontal clients (H-clients), it is further vertically partitioned with another client, labelled as a vertical client (V-client). In the first proposed framework, referred to as ‘Horizontal-OutputFed’, the output feature is stored at clients with horizontal partitioning, while in the second proposed framework, referred to as ‘Vertical-OutputFed’, the output feature is stored at the V-client. The accuracy, loss and convergence rate of the Horizontal-OutFed are the same as achieved with a horizontal partitioned federated learning framework. Further, the impact of non-identically and independently distributed data is the same in both the Horizontal-OutFed and purely horizontal federated learning setup for combined features. Compared to a centrally trained model, the globally trained model in the Vertical-OutFed has the same performance metrics such as accuracy and convergence rate. Moreover, there is no effect of imbalanced data on the performance of the Vertical-OutFed, contrary to some of the previous works on individual horizontal- or vertical-federated learning. Privacy attack analysis were conducted for active, passive, white-box, and black-box attacks demonstrating no risk of input and output feature leakage, nor any data inference from the resulting models or while sharing hidden layers’ outputs for either of the combined federated approaches. • Two novel FL frameworks, H-OutFed and V-OutFed, are developed for scenarios involving both horizontally and vertically partitioned data. To the best of our knowledge, these frameworks are the first to specifically address combined data partitioning in this manner. • The H-OutputFed framework ensures that the resulting mathematical model is equivalent to that generated in a purely horizontal FL setup, while the V-OutputFed framework achieves the same accuracy and convergence rate of the loss as a centralized model, which is not typically attainable with existing FL methods. • Extensive privacy analyses are conducted on the proposed FL frameworks to validate their security. These analyses demonstrate that the exchange of hidden layer outputs, as facilitated by our frameworks, does not result in any data leakage.