近年来，联邦学习因其隐私保护特性和在医疗保健和金融等隐私敏感领域的潜力而备受关注。然而，在分布式设备节点和中央服务器之间传输高维模型需要消耗大量通信资源，造成了联邦学习系统实际部署的关键瓶颈。因此，如何优化通信效率、降低通信成本成为当前联邦学习领域研究的一个关键瓶颈。为解决这一问题，先前的研究工作曾尝试调整分布式节点间的通信频率或通信传输量，但这些工作大多忽视了这两种调整方式之间的相互作用，这导致联邦学习系统的通信效率问题仍有进一步提升的空间，同时也揭示了同时对通信频率和通信传输量进行分析优化的必要性。为了提高联邦学习系统的实际性能和可部署性，本文提出了一种基于自适应通信压缩的高效通信联邦学习框架，其中针对传输量的压缩策略能够与针对通信频率的调整策略进行交互，从而系统性地降低传输开销。本文通过对联邦学习系统中的惰性聚合框架进行深入研究，从理论上对惰性聚合和通信压缩可能引起的聚合偏差进行分析，从而将联邦学习系统中的动态通信压缩策略转化为针对聚合偏差的优化问题，实现了通过最小化因降低通信频率和通信传输量而产生的信息损失来动态调整通信资源的分配。因此，本文提出的自适应通信压缩机制能够在保证训练收敛速度不受影响的同时，动态调整通信频率和通信传输量两个因素，显著缓解联邦学习系统的通信瓶颈。实验结果表明，本文所提出的联邦学习框架可以减少大约60%的通信传输量。此外，该框架在具有异构数据分布的联邦学习场景中也能够保持优越性能，具有良好的通用性和可扩展性。
In recent years, Federated Learning (FL) has gained significant research interest due to its privacy-preserving properties and its huge potential in privacy-sensitive distributed areas such as healthcare and finance. Nevertheless, the advancement and implementation of FL have been hindered by the significant communication overhead involved in transmitting high-dimensional models between the distributed device nodes and the central server. Therefore, how to optimize communication efficiency and reduce communication costs has become a key question in implementing FL systems. To tackle this challenge, previous attempts have been made to adjust either the communication frequency or the transmission amount for each communication, but few of them have investigated the interplay between the two kinds of adjustment. Consequently, it leaves room for further improvement and reveals an urgent need for a comprehensive analysis that simultaneously considers both communication factors.In this context, this thesis proposes a communication-efficient FL framework with adaptive compression that is able to interact with the adjustment of communication frequency and reduces unnecessary transmissions systematically. It first conducts an in-depth study of the lazy aggregation framework in FL systems and theoretically analyzes the aggregation bias that can be caused by lazy aggregation and communication compression. Based on this theoretical building block, it adaptively adjusts the compression level and communication frequency by minimizing the aggregation bias. The proposed adaptive compression strategy significantly reduces communication burden without negatively affecting training convergence. Experimental results indicate that the proposed framework can reduce overall transmitted bits by approximately 60%. Moreover, it maintains model performance in a variety of FL scenarios with heterogeneity, including heterogeneous data distribution and local model structure.
