Publication Radar · Source metadata

Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression

Boao Kong, Xu Huang, Yuqi Xu, Yixuan Liang, Bin Wang, Kun Yuan

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2025-09-23 · Indexed by Pub-AI: 2026-10-31 UTC

preprint

Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression

Boao Kong, Xu Huang, Yuqi Xu, Yixuan Liang, Bin Wang, Kun Yuan

Preprint posted: 2025-09-23 · Original record ↗

arxiv
2509.19029
Explicit source versions (2)
  • v2 · 2026-10-28
  • v3 · 2026-11-02
  1. Explicit source version detected · v2

    Source date: 2026-10-28 · First detected: 2026-10-31 UTC · Historical baseline

    Source evidence (1)
  2. Preprint available

    Source date: 2025-09-23 · First detected: 2026-10-31 UTC · Historical baseline

    Source evidence (2)
  3. Explicit source version detected · v3

    Source date: 2026-11-02 · First detected: 2026-11-13 UTC

    Source evidence (1)
Abstract · arxiv

Pipeline-parallel distributed optimization is essential for large-scale machine learning but is challenged by significant communication overhead from transmitting high-dimensional activations and gradients between workers. Existing approaches often depend on impractical unbiased gradient assumptions or incur sample-size memory overhead. This paper introduces Clapping, a Communication compression algorithm with LAzy samPling for Pipeline-parallel learnING. Clapping adopts a lazy sampling strategy that reuses data samples across steps, breaking sample-wise memory barrier and supporting convergence in few-epoch or online regimes. Clapping comprises two variants including Clapping-FC and Clapping-FU, both of which achieve convergence without unbiased assumption for compressed gradient, effectively addressing compression error propagation in multi-worker settings. Numerical experiments validate the performance of Clapping across different tasks.