- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors propose Dual Routing, a multi-LoRA training interface for solver–verifier–reviser workflows where downstream roles generate from a 0.5/0.5 factor-wise merge of their adapters while each role's gradient trains only its own adapter. The key mechanism is that the merged read pool reduces the effective adapter scale during training, preventing downstream roles from destructively overwriting the solver's correct answers — the paper's own controls show that training role isolation at this reduced scale recovers most of the gain (34.66% vs. 38.99%). Experiments on 1,472 synthetic program-reasoning items with Llama-3.1-8B across 7 seeds show Dual Routing beats role isolation by 26.7 points and is the only method with positive system gain, but on Qwen2.5-7B all methods are within 2 points. The statistical analysis is thorough (sign tests, Wilcoxon, bootstrap, Holm correction), though the underlying evaluation scope is narrow — four deterministic task templates with no natural-language reasoning benchmarks.
Critical strengths
Exemplary experimental controls. The preregistered route factorial (Table 6, Appendix E–G) systematically decomposes the contribution of reads, writes, and scale into separate controls — this is the level of ablation design more multi-role papers should aspire to.
Unusual transparency. The paper explicitly proves PCGrad-A/B reduce to role isolation under disjoint adapters (line 228–236), reports that Qwen results do not transfer (Appendix J), discloses that 3/7 Dual seeds were re-run after scoring (Appendix K), and provides first-run sensitivity analysis — the honesty test is passed decisively.
Rigorous statistical methodology. Seed-level inference with four complementary procedures (sign, Wilcoxon, t, bootstrap) with Holm correction, plus explicit separation of record-level vs. seed-level uncertainty (Table 13), sets a high standard for RL training papers that typically report a single-seed pooled mean.
Critical weaknesses
-
Mechanism undermines motivation. The paper's title and narrative emphasize "collaborative reads," but the own-half control (Table 17) shows that simply halving the adapter scale — with no cross-role reading at all — recovers 86% of the gap (34.66% vs. 38.99% for full Dual, from 7.64% isolated). The actual cross-role reading contributes only +4.33 points, making the headline contribution primarily an implicit regularization effect, not a collaboration effect.
-
Evaluation scope is critically narrow. All main results come from 4 synthetic task templates on a single backbone (Llama-3.1-8B); the one additional backbone tested (Qwen2.5-7B) shows the final-accuracy result does not transfer (Table 22, all methods within 1.6 points). No real-world benchmarks (math, code, open-domain QA), no model scale variation, no variation of the fixed 0.5/0.5 merge ratio or topology (e.g., solver reading downstream adapters) — the generality claimed in the conclusion far exceeds what the experiments support.
-
Effective baseline count is two. PCGrad-A and PCGrad-B are confirmed no-ops (inner product is exactly zero, projection never fires), collapsing five methods to three (Shared, Isolated, Dual). The Shared comparison is "unresolved" at seed level (CI [−5.82, +14.32]). So the paper's reliable claim is against one real baseline (role isolation), on one backbone, on synthetic tasks.
Questions for authors
-
The Qwen result shows no final-accuracy transfer. Have you investigated why, beyond "the solver is already strong"?
-
Why are no real-world benchmarks included?
6 comments
Response to Reviewer Spg3 (Part 2/4)
W1 (+ Presentation and clarity): Rollout Readout Scale Under Role-Owned Writes
"The paper's title and narrative emphasize 'collaborative reads,' but the own-half control (Table 17) shows that simply halving the adapter scale ... with no cross-role reading at all ... recovers 86% of the gap (34.66% vs. 38.99%)... making the headline contribution primarily an implicit regularization effect, not a collaboration effect."
Table 17 pointed directly to readout scale as the candidate driver. We tested the competing mechanistic hypotheses and centered the revised paper's explanation on forward readout scale under role-owned writes (Abstract, §1, §2.3, §5, §7, Limitations), tracing how rollout-scale attenuation prevents downstream entropy collapse:
- Factor-merge algebra and cross-term test (§2.3 Eqs. 4–5, p. 3; Main Tables 3–4, p. 6): Expanding (0.5 B_v + 0.5 B_r)(0.5 A_v + 0.5 A_r) = ¼(B_v A_v + B_r A_r + B_v A_r + B_r A_v) weights each downstream role's own adapter by s = 0.25. Scoring Dual checkpoints with exact rank-128 linear readout 0.25 B_v A_v + 0.25 B_r A_r (zero cross-terms) yields 31.60% vs. 32.07% on the 1,472-item main panel, 38.80% vs. 38.48% on Llama GSM8K (n = 500), and 72.08% vs. 71.96% on Qwen GSM8K (Appendix N, Tables 27 & 29), bounding the bilinear cross-term effect below 0.5 pp. Main Tables 3–4 also include Untrained workflow (0 updates, ref.; 41.75%) and label PCGrad-A/B as "(isolated rep. A/B)".
- 6-point readout-scale ladder (Table 24, p. 21) and 5-pair zero-rerun confirmation (Table 25, p. 21): On single seed 52, own-only readout collapses at s = 1.00 (0.61%) and stays in a stable regime below it: 33.15% at s = 0.75, 39.54% at s = 0.50, 43.95% at s = 0.25, 36.01% at s = 0.10, and 42.66% for the untrained reference (s = 0), with variations over s ∈ [0.25, 0.50] lying within single-seed variance. Across 10 fresh zero-rerun runs on seeds 53–57 (bitwise-identical 1,344-tensor LoRA initialization per pair), cross-scale s = 0.25 reaches 36.64% Final and Dual Routing 32.07% (Dual − cross-scale s = 0.25: −4.58 pp, t95 [−13.86,+4.70]; Always-revise +0.12 pp, [−3.38,+3.62]; 7-seed own-half at s = 0.5 gives −0.67 pp). Dual Routing and explicit s = 0.25 own-only reads perform within seed variance (paired SD 7.47 pp).
- Optimizer learning-rate control (Appendix N, Figure 6a, p. 22): Testing whether s = 0.25 behaves like a 4× smaller step size, we cut AdamW LR 4× to 2.5×10⁻⁶ while keeping s_train = 1.0 across 3 fresh runs (s53–55). Downstream collapse persists (21.33% final vs. 33.22% solver = −11.89 pp gain, 4.80% stop; behind both Dual and cross-scale s = 0.25 on 3/3 paired seeds), whereas attenuating only the evaluation readout to s_eval = 0.25 restores a 41.51% stop rate and moves GSM8K gain from −3.27 pp to +2.27 pp. Separating read (R_τ) and write (W_τ) routes thus isolates forward rollout scale under disjoint role-owned writes as the governing stability lever, and with solver drift removed (Base Solver + Trained VR, Figure 6b), Dual VR (29.52%) finishes ahead of cross-scale s = 0.25 VR (28.15%) on 5/5 paired seeds (+1.37 pp, t95 [+0.24,+2.50]).
Response to Reviewer Spg3 (Part 3/4)
W2, W3 & Q2: Real-World Math (GSM8K) & Code (APPS), Topologies, and Baselines
"Why are no real-world benchmarks included? ... No real-world benchmarks (math, code, open-domain QA), no variation of the fixed 0.5/0.5 merge ratio or topology ... Effective baseline count is two (Shared, Isolated)."
-
54-Checkpoint Cross-Domain and Cross-Family Evaluations (Appendix O, Tables 27–29, Figure 6c, pp. 22–24): We evaluated six Llama-3.1-8B arms (23 checkpoints, including same-seed s45–48 Dual and fixed-solver transplants) and seven Qwen2.5-7B arms (31 checkpoints, matched seeds 42–46) on held-out GSM8K (n = 500) natural-language math (zero prose math in training) and held-out APPS (n = 500) competitive programming (100 intro / 300 interview / 100 competition, container-executed against official unit tests):
Workflow Arm / Transplant (n = 500 items/benchmark) GSM8K Strict Final (Lenient) GSM8K Gain GSM8K APPS Strict Final (Test-avg) APPS Gain APPS Llama Untrained (0 updates, ref.) 65.60% (68.00%) -9.20 pp 24.1% 11.40% (24.83%) -1.00 pp 30.6% Llama Shared (, s45–48) 26.15% (26.25%) -9.35 pp 85.1% 5.85% (11.14%) -3.70 pp 84.2% Llama Role-Isolated (, s45–48) 13.35% (13.45%) -9.40 pp 99.3% 0.85% (1.72%) -4.40 pp 100.0% Llama Dual Routing ( merge, s45–48) 27.45% (29.90%) +5.20 pp 36.5% 7.75% (13.39%) -0.35 pp 28.5% Llama Cross-scale (s53–57) 26.44% (28.04%) +4.12 pp 39.8% 7.60% (13.47%) -0.28 pp 32.5% Llama Dual Routing ( merge, s53–57) 38.48% (40.44%) +1.68 pp 31.8% 6.76% (12.03%) +0.40 pp 38.5% Llama Fixed Base Solver (74.80%) + Dual VR 67.12% (69.44%) -7.68 pp 28.9% -- -- -- Qwen2.5-7B Untrained (0 updates, ref.) 80.20% (solver 20.80%) +59.40 pp 14.4% 12.40% (26.17%) (Table 29) 46.1% Qwen2.5-7B Shared (, s42–46) 54.04% (solver 28.00%) +26.04 pp 51.3% 13.72% (29.23%) (Table 29) 51.5% Qwen2.5-7B Role-Isolated (, s42–46) 33.44% (77.40% at ) +12.80 pp 85.7% 15.12% (31.09%) (Table 29) 66.8% Qwen2.5-7B dualconcat(, s42–46)0.84% (solver 20.64%) -19.80 pp 98.1% 1.60% (2.71%) (Table 29) 94.1% Qwen2.5-7B Dual Routing ( merge, s42–46) 71.96% (Base S + Dual VR: 80.56%) +49.20 pp 14.2% 13.52% (29.15%) (Table 29) 40.3% For Qwen rows the parenthetical gives strict-format solver accuracy, except Role-Isolated (accuracy at s_eval = 0.25) and Dual (Base S + Dual VR). Zero-shot Qwen2.5-7B often omits the required
<answer>tag on first-pass prose math (20.80% strict solver), and the reviser stage formats and repairs answers to 80.20% strict Final for the untrained workflow (Table 29 caption); the identical strict-tag scorer applies across all arms.- Expanded baseline suite (W3): Alongside Shared and Role-Isolated (), the revised paper compares against zero-shot Untrained (Main Tables 3–4, Tables 27–29), scale-matched cross-scale and
own-half(Tables 24–25, 27–28), Low-LR () Role-Isolated (Appendix N, Figure 6a), shared-downstream-writes (Table 16), unattenuateddualconcat(, Table 29), and fixed-solver transplants (Tables 27 & 29), and contrasts single-backbone LoRA routing with 3-checkpoint MALT in §6 (p. 9). Same-seed Llama (s45–48) Dual beats Role-Isolated on 4/4 seeds on both GSM8K (+14.10 pp, item , +5.20 pp system gain) and APPS (+6.90 pp, seed ), while fixed-solver Base S + Dual VR maintains untouched base-solver accuracy across all three settings (29.52% vs. 29.42% on the 1,472-item main panel; 67.12% vs. 65.60% Llama GSM8K; 80.56% vs. 80.20% Qwen GSM8K), compared with 0.29%, 44.75%, and 33.48% for Role-Iso VR.
- Expanded baseline suite (W3): Alongside Shared and Role-Isolated (), the revised paper compares against zero-shot Untrained (Main Tables 3–4, Tables 27–29), scale-matched cross-scale and
-
Merge-scale and operator variations (W2): Together with the 2×2 route factorial, solver-read, and shared-downstream-write topologies (Table 6 p. 8; Appendices E–G pp. 14–18), Table 24 (p. 21) sweeps , and Tables 27 & 29 (pp. 23–24) compare exact rank-128 linear readout, concatenation (
dualconcat), and factor merging.
Response to Reviewer Spg3 (Part 1/4)
We thank Reviewer Spg3 for the constructive review and for noting our controlled route ablations, statistical analysis, and transparency. Taking up your comments on presentation, mechanism, and scope (W1–W3, Q1–Q2), we reorganized the Revised Manuscript around the algebraic rollout-scale mechanism (Abstract, §1, §2.3, §4.1, §5–§7, Limitations), added Untrained (0 updates, ref.) as row 1 of Main Tables 3–4, labeled PCGrad-A/B as "(isolated rep. A/B)", and ran 19 fresh 100-step H200 RL runs alongside fixed-solver transplants, exact rank-128 zero-cross-term readouts, and 54-checkpoint evaluations (>60,000 trajectories) on GSM8K (n = 500) math and container-tested APPS (n = 500) code.
The Dual Routing training and inference procedure is unchanged. Because the 0.5/0.5 merge sets s = 0.25 by construction on each role's own delta, Dual Routing keeps the LoRA rollout perturbation in the stable regime while role-owned writes prevent cross-role gradient interference.
Summary of Revisions