About: Federated Learning's Blessing: FedAvg has Linear Speedup

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: Federated Learning's Blessing: FedAvg has Linear Speedup Goto Sponge NotDistinct Permalink

An Entity of Type : schema:ScholarlyArticle, within Data Space : covidontheweb.inria.fr associated with source document(s)

Attributes	Values
type	Academic Article research paper schema:ScholarlyArticle
isDefinedBy	Covid-on-the-Web dataset
title	Federated Learning's Blessing: FedAvg has Linear Speedup
Creator	Kalagnanam, Jayant Li, Zhaojian Lin, Kaixiang Qu, Zhaonan Zhou, Jiayu »more»
source	ArXiv
abstract	Federated learning (FL) learns a model jointly from a set of participating devices without sharing each other's privately held data. The characteristics of non-iid data across the network, low device participation, and the mandate that data remain private bring challenges in understanding the convergence of FL algorithms, particularly in regards to how convergence scales with the number of participating devices. In this paper, we focus on Federated Averaging (FedAvg)--the most widely used and effective FL algorithm in use today--and provide a comprehensive study of its convergence rate. Although FedAvg has recently been studied by an emerging line of literature, it remains open as to how FedAvg's convergence scales with the number of participating devices in the FL setting--a crucial question whose answer would shed light on the performance of FedAvg in large FL systems. We fill this gap by establishing convergence guarantees for FedAvg under three classes of problems: strongly convex smooth, convex smooth, and overparameterized strongly convex smooth problems. We show that FedAvg enjoys linear speedup in each case, although with different convergence rates. For each class, we also characterize the corresponding convergence rates for the Nesterov accelerated FedAvg algorithm in the FL setting: to the best of our knowledge, these are the first linear speedup guarantees for FedAvg when Nesterov acceleration is used. To accelerate FedAvg, we also design a new momentum-based FL algorithm that further improves the convergence rate in overparameterized linear regression problems. Empirical studies of the algorithms in various settings have supported our theoretical results.
has issue date	2020-07-11(xsd:dateTime)
has license	arxiv
sha1sum (hex)	e1f2588ee4a7503aa2cf37f338ed50a92b0d0e65
resource representing a document's title	Federated Learning's Blessing: FedAvg has Linear Speedup
resource representing a document's body	covid:e1f2588ee4a7503aa2cf37f338ed50a92b0d0e65#body_text
is schema:about of	named entity 'remains' named entity 'gap' named entity 'large' named entity 'emerging' named entity 'literature' »more»

Faceted Search & Find service v1.13.91 as of Mar 24 2020

Alternative Linked Data Documents: Sponger | ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3229 as of Jul 10 2020, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (94 GB total memory)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2025 OpenLink Software