Benchmarking Aggregation Free Federated Learning using Data Condensation: Comparison with Federated Averaging

This project investigates the performance of Federated Learning Aggregation-Free (FedAF) compared to traditional federated learning methods under non-independent and identically distributed (non-IID) data conditions, characterized by Dirichlet distribution parameters (alpha = 0.02, 0.05, 0.1). Utilizing the MNIST and CIFAR-10 datasets, the study benchmarks FedAF against Federated Averaging (FedAVG) in terms of accuracy, convergence speed, communication efficiency, and robustness to label and feature skews.  

Traditional federated learning approaches like FedAVG aggregate locally trained models at a central server to form a global model. However, these methods often encounter challenges such as client drift in heterogeneous data environments, which can adversely affect model accuracy and convergence rates. FedAF introduces an innovative aggregation-free strategy wherein clients collaboratively generate a compact set of condensed synthetic data. This data, augmented by soft labels from the clients, is transmitted to the server, which then uses it to train the global model. This approach effectively reduces client drift and enhances resilience to data heterogeneity. Additionally, by compressing the representation of real data into condensed synthetic data, FedAF improves privacy by minimizing the transfer of raw data.  

The experimental results indicate that while FedAF converges faster, it struggles to stabilize under highly heterogenous environments due to limited real data representation capacity of condensed synthetic data.