New rules for decentralized learning during device outages
Self-inclusive Freshness-weighted Gossip helps devices aggregate models after being offline.
Decentralised federated learning (DFL) trains personalised models on devices. These devices average models with neighbours instead of using a central server.
Real devices often go offline for hours or days. A client returning from an outage must aggregate models held by its neighbourhood.
The stalest model is often the client's own model. Existing rules discount received models but do not address the client's own staleness.
Self-inclusive Freshness-weighted Gossip (SFG) uses three principles. The receiver must be discounted like a neighbour. Staleness must be measured as inherited lag rather than age. The discount must be steep.
Effective lag tracks inherited lag at the cost of one scalar per message. SFG-L and SFG-Cut are lag-based members of the SFG family.
Tests on three image benchmarks and eight outage processes showed SFG-L and SFG-Cut outperform decentralised federated averaging. These tests included real smartphone availability traces.
Key facts
- SFG-L and SFG-Cut consistently outperform decentralised federated averaging and recent age-of-information rules.
- The receiver must be discounted like any neighbour to keep error bounded.
- Effective lag tracks inherited lag using one scalar per message.