- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the spectral structure of RLVR updates and finds that the base model's singular values can be reused while adapting the singular vectors. Based on this observation, the authors propose ISO, including ISO-Merger for data-free expert merging and ISO-Optimizer for online RLVR training. Experiments show improvements in merging performance, reasoning accuracy, and training efficiency.
Critical strengths
-
Well-motivated empirical finding. The paper goes beyond observing limited spectral drift in RLVR. The spectrum restoration and fixed-spectrum training experiments (Section 2) provide useful evidence that the base spectra can be reused without losing most of the learned capabilities. This gives a reasonable motivation for the proposed methods.
-
Consistent empirical improvements. ISO-Optimizer outperforms the tested AdamW and Muon baselines on math and coding tasks. On Qwen3-4B, it reaches AdamW's final accuracy with 34% less wall-clock time. ISO-Merger also achieves the best aggregate scores among the evaluated data-free merging methods. The comparisons with Pion and the additional ablations strengthen the empirical evaluation.
Critical weaknesses
-
Limited novelty in the core parameterization. Spectrum-preserving optimization has already been explored in POET and Pion, while Stiefel-manifold parameterizations have also appeared in prior work. ISO differs in how it optimizes the singular frames and applies this structure to RLVR, but the underlying geometric idea is not new. The methodological advance over these approaches is therefore somewhat incremental.
-
The source of the optimization gains remains unclear. The trainable-spectrum ablation (Appendix Fig. 18) performs similarly to the fixed-spectrum variant. This supports the claim that spectral changes are unnecessary, but does not establish whether fixing the spectrum is responsible for the improvement over conventional optimization. The gains may instead come from the factorized parameterization or retraction dynamics. Also, the main online experiments are limited to relatively small models, with limited multi-seed evidence.
Questions for authors
-
Can the authors better isolate the contributions of fixed-spectrum constraints, singular-frame parameterization, and orthogonal retraction to the observed optimization gains?
-
How robust are the improvements at larger model scales and across random seeds? In particular, would the advantage over Pion remain with more extensive tuning on Qwen3-4B?
5 comments
Rebuttal 2/3
Q2. Attribution of the optimization gains.
The source of the optimization gains remains unclear...This supports the claim that spectral changes are unnecessary, but does not establish whether fixing the spectrum is responsible for the improvement... The gains may instead come from the factorized parameterization or retraction dynamics.
Can the authors better isolate the contributions of fixed-spectrum constraints, singular-frame parameterization, and orthogonal retraction to the observed optimization gains?
Response: We appreciate this question and would first respectfully clarify that we do not attribute ISO's gains over AdamW to spectrum freezing. As discussed in Q1, spectral inheritance naturally motivates optimizing the singular frames while reusing the source spectrum. Frame-coordinate updates and orthonormality restoration through polar retraction are both integral parts of this finding-grounded design. Our submission explicitly states that “The factor parameterization and optimizer-state dynamics are both part of the method” (Section 4.2, Lines 320–323).
Additional attribution controls. Following your suggestion, we add five Qwen3-4B-Base/16K ablation arms to examine the roles of spectrum fixing, frame-coordinate optimization, and polar retraction. The full design, learning rates, accuracy curves, and gradient-norm table are reported in our response to Reviewer YDHU, Q3. We report accuracy to the latest 210 steps for each arm. We show that:
- Spectrum projection alone does not reproduce ISO's gains (A/B/C). At update 210, ordinary and projected AdamW score 42.6 and 41.9, respectively.
- Frame-based optimization adds value beyond the spectral constraint (B/C). Both B and C enforce the source spectrum, but ISO reaches 44.0, 2.1 points higher than projected AdamW. It also exceeds ordinary AdamW by 1.4 points.
- Orthogonal retraction helps stabilize optimization (D/E). The variant without retraction (E) learns faster initially, but its actor gradient norm rises sharply after about 190 updates. In the 10-update window labeled 210, its mean / maximum norms reach 0.23 / 0.55, versus 0.04 / 0.05 for both the variant with retraction (D) and AdamW. This is accompanied by a decline in accuracy from 43.2 at update 160 to 41.9 at update 210, when D scores 44.3—a gap of 2.4 points.
Overall, spectral inheritance motivates optimizing the singular frames while maintaining their orthonormality. Both components (frame parameterization and orthogonal retraction) are integral to ISO-Optimizer, a simple instantiation of our design principle, and the added controls support their respective roles.
Our paper does not attribute ISO’s performance gains to spectrum fixing. Figure 18 instead shows that, once optimization is shifted to the singular frames, making the spectrum trainable provides little additional performance benefit. We emphasize that establishing source-spectrum reusability is itself a key contribution, even if spectrum fixing alone offers little direct performance benefit. This finding provides the empirical basis for shifting adaptation to the singular frames, grounds our principle, “reuse the spectrum, optimize the frames,” and enables the broader ISO framework for online learning and offline expert composition.
Rebuttal 3/3
Q3. Larger models, independent seeds, and tuned Pion.
How robust are the improvements at larger model scales and across random seeds? In particular, would the advantage over Pion remain with more extensive tuning on Qwen3-4B?
Response: Full results for the seed and Pion experiments are reported in our response to Reviewer YDHU, Q1.a and Q2; we summarize them below alongside the 8B comparison.
(1) Independent training seed in the main setting. We have added one independent training seed per method in the main Qwen3-4B/16K setting, giving two runs per method at the same 400-update endpoint:
| Method | Accuracy at update 400 (original; additional) | Mean ± sample SD |
|---|---|---|
| AdamW () | 44.82; 44.50 | 44.66 ± 0.23 |
| ISO-AdamW () | 46.98; 46.24 | 46.61 ± 0.52 |
ISO-AdamW remains ahead in both repetitions, with a mean advantage of 1.95 percentage points.
(2) Additional Pion tuning on 4B. Using the official Pion implementation, we sweep its learning rate over , extending the comparison beyond the published setting. Higher learning rates lead to late-stage performance collapse, and Pion's highest reported score remains below ISO-AdamW's final accuracy: 44.32 versus 46.98, a gap of 2.66 percentage points.
(3) Larger-model validation: Qwen3-8B-Base. We use one training seed per method and a batch of 512 prompts, twice the batch size in the main experiments, to accelerate the experiment. The runs reach 120 steps now, while the results through the 120 steps are already informative, as this budget covers roughly two epochs over the training datasets. AdamW uses , its best tested learning rate on 4B.
| Update | AdamW | ISO-AdamW | Difference (points) |
|---|---|---|---|
| 0 (shared base) | 20.96 | 20.96 | — |
| 20 | 36.94 | 39.51 | +2.57 |
| 40 | 41.76 | 43.61 | +1.85 |
| 60 | 45.43 | 47.52 | +2.09 |
| 80 | 47.76 | 48.61 | +0.85 |
| 100 | 47.98 | 49.76 | +1.78 |
| 120 | 48.84 | 50.35 | +1.51 |
- Higher accuracy. ISO-AdamW leads at every evaluated checkpoint from updates 20–120, by 0.85–2.57 points, and is 1.51 points higher at update 120.
- Faster time to target. Using AdamW's update-120 accuracy, 48.84, as the common target on the shared evaluation schedule, ISO first reaches it at update 100, versus update 120 for AdamW, using about 17% fewer updates and 18% less training time.
We will include these results in the revised manuscript.
Thank you again for your review and patience, and apologies for the slightly delayed response while we complete the requested ablations. We hope these additional results and clarifications address your concerns, and we would be grateful if you would consider raising your overall assessment. Please let us know if you have any further questions.
Rebuttal 1/3
Reviewer 3dnf
Thank you for recognizing the functional evidence for spectral inheritance and the consistent improvements of both ISO applications. We address your concerns about novelty, attribution, and robustness below.
Q1. Novelty and relationship to prior work.
Response: We appreciate the opportunity to clarify the sources of novelty in our work. We agree that ISO shares geometric structure with prior methods such as POET, Pion, and Stiefel-manifold optimizers. Sharing a geometric constraint does not imply identical scientific findings or algorithms. We believe that assessing ISO's novelty primarily through its shared spectral constraint overlooks its combined scientific and methodological contribution, which extends beyond the reparameterization alone.
(1) Identifying functional spectral inheritance not established by prior methods. We establish a functional property of RLVR: adaptation can be shifted to the singular frames while inheriting the source spectrum. Successful training with a spectrum-preserving optimizer alone, despite the shared geometry, does not establish that the source spectrum remains reusable after unconstrained RLVR. This is a distinct scientific contribution that those prior methods have not established: we move beyond observations of spectral stability to functionally validate source-spectrum reuse after unconstrained RLVR.
(2) Proposing a shared adaptation space that supports an RLVR optimization framework beyond a single optimizer design. ISO establishes an RLVR optimization framework, rather than another individual optimizer design under the same geometric constraint. Grounded in a functionally validated property of RLVR, ISO provides a shared adaptation space that supports multiple methods beyond the reparameterization itself: online learning directly optimizes the frames, while offline composition combines specialists' learned frame changes. This joint construction is a contribution at the framework level that prior work on similar reparameterizations has not established.
(3) Distinct learning and composition algorithms.
ISO-Optimizer provides a generic transformation of base optimizers. It applies the chosen base optimizer—AdamW or Muon in our experiments—directly to the singular frames, maintains optimizer states in those coordinates, and restores orthonormality through polar retraction. As you also noted, its concrete update rule differs from those of prior methods despite the shared weight geometry. Both instantiations improve over their corresponding baselines.
ISO-Merger introduces a novel formulation of expert merging: it composes changes in both singular frames relative to a shared base, using retention-aware weighting and mode selection under the inherited spectrum. It requires no post-merge data, rollouts, or distillation, and the experts need not have been trained with ISO. The shared parameterization does not specify this composition algorithm; that construction is a methodological contribution of ISO-Merger.
ISO therefore contributes an RLVR optimization stack grounded in a functional finding. Online policy learning and offline expert composition are two complementary instantiations of the same empirically supported principle. We believe that the functional finding, unified RLVR optimization framework, and concrete algorithms together constitute a substantive contribution beyond the shared geometric constraint.