acceptodds
Under review as a conference paper at ICLR 2027

COMPASS: Decentralized Language Model Post-training

Abstract

Pipeline-parallel post-training over Internet links must reduce communication while adapting open-weight models without architectural changes. The two-circuit design of anon2026twocircuit addresses this problem with a fast circuit that uses lossy compression in the forward and backward passes, and an anchor circuit that runs uncompressed passes on delayed weights. The anchor trades staleness for uncompressed computation and uses its gradients to guide the fast circuit. We propose COMPASS, which improves both pipeline compression and communication between the circuits. We propose to learn a basis from the anchor activations, reducing payloads at compressed boundaries by more than two orders of magnitude relative to dense communication in supervised fine-tuning (SFT). Across SFT and reinforcement learning with verifiable rewards (RLVR) experiments, we find that anchor-gradient signs are sufficient to provide useful guidance, reducing the cost of returning gradients. For RLVR, we propose a weight forecasting mechanism that accounts for anchor delay as the policy and its training data change. We also derive a convergence bound for the system and show that it achieves state-of-the-art results across various SFT and RLVR benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Decentralized training, post training, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_8u6VOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

COMPASS makes communication cheaper for distribution post-training through 3 components:

  1. it compresses the dimensions of the activations of fast circuit at pipeline boundaries (alternating ones in SFT) by projecting onto a low-rank basis that the anchor estimates from its uncompressed activations via power iteration

  2. the anchor maintains a running average of its gradients and sends the signs of that average (directional information) to the fast circuit. These signs are then used correct the fast circuit gradients, flipping the direction of its gradients if the anchor and fast circuit gradient directions don't align (\alpha < 0.5)

  3. For SFT, only top-10% blocks of weight change between the fast circuit weights and the anchor weights, quantized to int8, are used to update the anchor's weights, with error feedback. For RLVR, COMPASS additionally does anchor weight forecasting, which linearly extrapolates weights from the two most recent available snapshots and uses them to compute the basis (1) and signs (2) with an aim to reduce their staleness

Critical strengths
  1. full-parameter distributed post-training of existing open-weight models without architectural changes is a timely and under-studied problem

  2. the proposed learned activation basis is shown empirically to outperform random mask used in prior work at the same compression ratio

  3. the anchor sign correction yield large gains in accuracy for RL

Critical weaknesses

my main concerns are:

  1. Lines 784-792, the authors mentioned that their main result in Table 2 are "synthetic draft placeholders, with matched measurements still pending". In other words, apart from NarrativeQA, code, and SQL metrics, the rest of the numbers in the table appear to be placeholders. Furthermore, in the same paragraph, the authors wrote that "earlier unspecified Code result was 0.475 for Dense, 0.510 for the previous method, and 0.497 for COMPASS" but this matches nothing in Table 2.

  2. Line 712 and Line 725, the authors claim to run their experiments with multiple seeds and report mean and standard deviation but their results only report a single number. Also, Appendix A is written using future tense, e.g. "will use", "will differ", "will be disabled" making the reader confused what has been run.

  3. Table 9 shows SQL increasing from 0.67 to 0.79 when K drops from 80 to 20 while Table 2 reports 0.79 at K=165. Furthermore, Appendix F timing model gives K~=132.7 and describes K=35 as unverified (Line 919). Which K was actually used?

  4. The 128x applies only per compressed boundary but only alternating boundaries are compressed in SFT (Lines 772-773), does the abstract's claim of "more than two orders of magnitude" still hold for end-to-end reduction or is it overclaimed?

  5. Table 10 Qwen3-4B-Base shows that COMPASS performance is below Dense in all cases but is written as "on par" in the conclusion

Given these inconsistencies between the claims and reported results, and the integrity and reliability of the main results, I recommend rejection. I am also open to raising my score if the authors can provide complete mean and standard deviation results as described in the Appendix, clarify the anchor delay used in the main experiments, and release anonymized code so I can independently verify their numbers.

Questions for authors

See weaknesses

1 comment

Thank you for the detailed review. The inconsistencies you list are fair.

The Appendix C sentence was written while we were drafting the paper. The results in the table are derived from our actual empirical results. We have now included an anonymised code in the supplementary materials for reproducibility. In the future version we will revise the manuscript carefully. This was an honest mistake on our part, and we apologise for it,

All our training runs used a single seed, so we cannot provide means and standard deviations.

The supervised runs used an anchor delay of K = 20 fast steps, and Table 9 varies it from 20 to 80. The Appendix F values are estimates from a simulated timing model, not measurements.

The abstract's figure refers to payloads at compressed boundaries. Only alternating boundaries are compressed, so it does not hold end to end.

On scale, we agree that "on par" does not hold at the larger scale. At 1.5B, after 200 updates, COMPASS and Dense have similar ten-benchmark means (30.03 and 29.36), and the differences go both ways across benchmarks, so we think "on par" is fair at that scale, from one run each. At 4B, after 100 updates, COMPASS is above the base model on the mean but below Dense on every benchmark (34.05 against 38.64). We tuned the settings at 1.5B and used them at 4B without retuning. The conclusion should have reported each scale separately.

In future we will come with a strong submission addressing all the reviewer concerns.

Reviewer_z1LZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces COMPASS, a communication-efficient method for pipeline-parallel post-training over slow links without changing the model architecture. It uses learned activation compression and gradient-sign guidance to reduce communication costs while maintaining model performance. The method also introduces weight forecasting to handle delayed updates during reinforcement learning. Experiments cover SFT on Llama-3.2-1B and RLVR on Qwen2.5-Math-1.5B and Qwen3-4B-Base.

Critical strengths
  1. Practical and underexplored problem: Addresses communication bottlenecks in pipeline-parallel post-training over slow Internet links.
  2. Architecture preservation: Enables efficient post-training without modifying the original model architecture.
  3. Extension to reinforcement learning: Extends communication-efficient post-training to reinforcement learning with verifiable rewards (RLVR).
  4. Theoretical analysis: Provides a convergence bound for sign-guided optimization under specific assumptions.
  5. Extensive evaluation and reproducibility: Evaluates multiple model families, sizes, and diverse benchmarks, with detailed appendices.
Critical weaknesses

W1. Placeholder benchmark results:

Appendix C explicitly states that "the additional benchmark entries are synthetic draft placeholders, with matched measurements still pending." Moreover, several reported values raise concerns: in Table 2, the 128× control achieves 0.366 accuracy on WinoGrande (below the 0.5 random-guessing baseline for a binary task) and 0.156 on MedQA (below the 0.25 random-guessing baseline for a four-choice task).

W2. Missing statistical significance:

Although Appendices A and B describe reporting means and standard deviations across multiple seeds, the main results tables do not provide standard deviations, error bars, or statistical significance tests, making it difficult to assess the reliability of the reported improvements.

W3. Limited theoretical guarantees:

The convergence analysis considers a simplified sign-guided SGD update rather than the full COMPASS training procedure. Moreover, it relies on strong assumptions about gradient alignment and anchor-sign accuracy that are not empirically verified.

Questions for authors
  1. What do “earlier manuscript” and “synthetic draft placeholders” refer to in Appendix C, and why do some results in Table 2 fall below random-chance accuracy?
  2. Which results in Tables 2–5 are based on actual measurements, and which are placeholders or estimates? Could the authors provide the measured results, including means and standard deviations across multiple seeds?
  3. Is the reported training throughput directly measured or estimated?
1 comment

Thank you for the careful review. Your concerns are fair.

The Appendix C sentence was written while we were drafting the paper. The results in the table are derived from our actual empirical results. We have now included an anonymised code in the supplementary materials for reproducibility. In the future version we will revise the manuscript carefully. This was an honest mistake on our part, and we apologise for it,

All our training runs used a single seed, so we cannot report means and standard deviations.

We agree that the convergence analysis covers a simplified sign-guided SGD step on a fixed objective, not AdamW or the full training procedure, and that its assumptions are not verified empirically. The throughput in Table 5 comes from a timing model, not a measurement.

In future we will come with a strong submission addressing all the reviewer concerns.

Reviewer_w529OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

COMPASS improves an existing two-circuit framework for decentralized language-model post-training through anchor-learned activation projections, sign-based gradient correction, and compressed checkpoint updates. It additionally introduces weight forecasting for RLVR and provides a conditional SGD convergence bound. Evaluation covers SFT and RLVR, but some central SFT results are explicitly identified as synthetic placeholders, substantially weakening the empirical support.

Critical strengths

The sign-based correction is simple and avoids the per-matrix SVDs required by the previous method. Learning the activation basis on the dense anchor also keeps this computation off the fast trainer. Table 11(b) provides useful evidence for sign correction: validation accuracy increases from 50.30% to 63.33% when signs are retained with the adaptive basis.

Critical weaknesses
  1. Appendix C states that additional entries in Table 2 are “synthetic draft placeholders,” but these entries are not identified in the table and are included in the 14-benchmark mean. This makes the main SFT comparison difficult to trust and is my main reason for recommending rejection.
  2. The RLVR experiments do not establish the benefit of forecasting because there is no matched zero-gain control. The frozen basis also outperforms the adaptive basis. At 4B, COMPASS scores below Dense on every benchmark, with a mean of 34.05% versus 38.64%, so the conclusion’s broad claim of comparable performance is not supported.
Questions for authors
  1. Which entries are measured and which are placeholders? Please provide corrected tables and averages using completed measurements, along with variability across seeds.
  2. Does forecasting improve performance over an otherwise identical (\gamma=0) control? What evidence supports updating the basis rather than keeping it frozen?
1 comment

Thank you for the review. Both weaknesses are fair.

The Appendix C sentence was written while we were drafting the paper. The results in the table are derived from actual experiments. We have now included an anonymized code for reproducibility in supplementary materials. In the future version we will revise the manuscript fully to fix those. This was an honest mistake on our part, and we apologise for it,

All our training runs used a single seed, so we cannot report variability across seeds.

On forecasting, we agree that the paper does not show a benefit. We compared forecast gains of 1, 0.5 and 0.25 at matched delays, but never a gain of 0, so no run isolates forecasting. Section 6.4 notes this, but the abstract, introduction and conclusion describe forecasting without that caveat, which we will fix. The gains we tested were at most three points apart in validation accuracy, and with forecasting on, scores stayed in a similar range as the delay grew from 10 to 80 updates. With one run per setting, this shows only that the scores did not change much with the gain or the delay. It does not show that forecasting helps, because we cannot tell how the runs would have done without it. In the released code, the forecast gain is a single setting, so the zero-gain control needs no other change.

On scale, we agree that "on par" does not hold at the larger scale. At 1.5B, after 200 updates, COMPASS and Dense have similar ten-benchmark means (30.03 and 29.36), and the differences go both ways across benchmarks, so we think "on par" is fair at that scale, from one run each. At 4B, after 100 updates, COMPASS is above the base model on the mean but below Dense on every benchmark (34.05 against 38.64). We tuned the settings at 1.5B and used them at 4B without retuning. The conclusion should have reported each scale separately.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.