- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper studies the parameter placement problem in LoRA. Given a fixed budget of trainable entries in the matrix of a LoRA adapter, with frozen and at initialization, the authors ask whether the choice of entries affects downstream performance. Entries are scored at initialization from 50 forward-backward passes of the supervised cross-entropy loss, using either the absolute mean gradient or the diagonal Fisher , and the global top- entries are trained with . The authors report that under SFT random and informed subsets perform comparably (Table 2), whereas under GRPO on Qwen2.5 base models (1.5B, 3B, 7B) random subsets remain close to the base model and or circuits of 10K to 50K entries match full LoRA on GSM8K and MATH-500 (Table 3). A similar ordering is reported for text-to-SQL (Table 36). The proposed explanation rests on gradient structure measured at initialization. SFT gradients are reported to be low-rank and aligned across consecutive steps, whereas GRPO gradients are higher-rank and nearly orthogonal, so that only coordinates with consistently signed gradients accumulate displacement (Table 1). A synthetic two-layer MLP experiment is presented as reproducing this distinction (Fig. 1). The remaining analyses characterize the selected circuits. Discovery is stable and takes under 10 seconds (Tables 6 to 8), random placement remains at base accuracy for between 32 and 2048 (Table 13), circuits overrepresent the V projection and concentrate in early layers (Table 4), removing high-scoring entries from a trained full-LoRA adapter degrades accuracy faster than random removal (Fig. 3), and circuits induce larger logit-lens distribution shifts than random subsets at comparable weight norm (Fig. 4).
Critical strengths
-
The central empirical observation under GRPO is clear and is controlled for the most obvious alternative explanation. On Qwen2.5 base models, random sparse placement remains at or near base accuracy (on 1.5B GSM8K, 9.5 for the base model, for random, for and for full LoRA, Table 3), while the informed circuits recover full-LoRA accuracy with well under 1% of its trainable parameters. A natural concern is that a random mask receives too small an effective update. Table 13 addresses this directly, since at K random placement stays between 9.1% and 9.9% for every from 32 to 2048, while the circuits improve monotonically and saturate near . The same ordering is observed on text-to-SQL with an execution reward across four Qwen2.5-7B variants (Table 36). This result is the main reason I do not recommend a clear rejection. It is of practical relevance, as it indicates that under GRPO on base models the trainable entries of a sparse adapter cannot be chosen arbitrarily, and that a short gradient pass at initialization suffices to identify a working subset. It also qualifies the conclusion of Donoway et al. (2025) that randomly chosen subsets are sufficient.
-
The appendix reports several results that are unfavorable to the main narrative, which increases confidence in the reported measurements. The authors show that random placement fails under SFT on GSM8K (Table 31), that circuits scored with cross-entropy and with GRPO share only 11 to 13% of their entries (Table 20), and that normalizing scores within each tensor renders the knockout curve indistinguishable from random (§D.8). These results are reported transparently. As the weaknesses below indicate, they are also central to evaluating the paper's claims, and much of the evidence needed to revise the interpretation is already present in the submission.
Critical weaknesses
- The main explanatory claim, that placement matters under GRPO but not under SFT because GRPO gradients are noisy and cancel across steps, is not supported by the paper's controlled evidence, and the proposed mechanism is not tested by intervention. The SFT evidence in Table 2 is obtained on Alpaca and evaluated with log-likelihood multiple-choice benchmarks, on which full LoRA itself yields little improvement over the base model (65.6 to 66.7 for Qwen2.5-3B and 70.9 to 71.5 for 7B). With roughly one point of headroom, all placements are expected to appear comparable. The GRPO evidence, in contrast, is obtained on generative mathematical reasoning with exact-match grading. The training regime is therefore confounded with the task and the evaluation metric. The only same-task comparison in the paper, Table 31, does not support the claimed distinction. Under SFT on GSM8K with Qwen2.5-1.5B, random placement reaches 13.2% against a base accuracy of 9.5%, while , and full LoRA reach 62.2%, 64.3% and 61.4%. Random placement thus fails under SFT in the same way as under GRPO (11.6%, Table 3). Qwen2.5-3B exhibits the same pattern (random 62.9% against a base of 62.1%), and random placement falls below the base model on IFEval after SFT (Table 33). This failure occurs in precisely the setting for which Table 17 reports favorable SFT gradient structure (effective rank 2.3, consecutive cosine similarity 0.87, net-to-step norm ratio 0.93), so the proposed mechanism predicts the opposite outcome. The converse case is also inconsistent with the mechanism. For Qwen2.5-7B on GSM8K the GRPO gradient is comparatively well aligned (cosine 0.41, net-to-step ratio 0.78, Table 17), yet random placement remains near the base model (80.9% against 79.3%, Table 3). The SFT comparison is further complicated by an inconsistency in the reported scaling factor. Section 4 states that is used for every sparse run, random placement included, whereas §E.4 states that the random baselines in Table 2 use . The latter would give random placement a 32-fold smaller effective step, which Table 13 shows is sufficient to keep even the informed circuits at base accuracy under GRPO.
More fundamentally, gradient noise is proposed as the cause, but it is never varied experimentally. All gradient statistics are computed at initialization, where and the parameters do not change between the compared steps. Table 1 therefore measures agreement between minibatch gradients at a single point rather than cancellation along a training trajectory. If the batching in Table 27 follows the TRL convention, each GRPO update contains approximately 2 prompts with 4 completions each, and at 9.5% base accuracy most groups receive zero advantage. Near-orthogonal consecutive gradients may thus be a consequence of the small effective batch rather than a property of the RL objective. The experiment that would test the mechanism directly is absent. Such an experiment would increase the number of prompts or generations per GRPO step and examine whether random placement recovers as gradient noise decreases, and would conversely reduce the batch size or inject gradient noise under SFT and examine whether random placement begins to fail. The synthetic experiment does not substitute for this test. Between its two settings it simultaneously changes the target (a dense rank-2 residual versus a with 5% large entries), the optimizer (Adam versus SGD), the batch size (128 versus 4) and the noise level (none versus ) (§C.6). Because the RL-like target is sparse in the coordinate basis of by construction, the advantage of coordinate selection in that setting is built into the design. It also models the GRPO signal as concentrated, whereas Section 3.3 describes the GRPO gradient as spread over more directions. In addition, Fig. 1 does not agree with the accompanying text. In Fig. 1(a), informed placement achieves lower relative MSE than random placement by approximately 0.15 to 0.25 at every budget (approximately 0.57 versus 0.87 at 10%), contrary to the statement that the two are "indistinguishable" (line 208). In Fig. 1(b), random placement at 50% reaches a relative MSE of approximately 0.38, contrary to the caption's statement that it "barely improves even at 50%". Since the title, abstract, Section 3.3 and conclusion all rely on this regime distinction, this is the weakness that most strongly determines my rating.
- Whereas the first weakness concerns when placement matters, the second concerns what the selection procedure identifies. The evidence in the paper is consistent with explanations at a coarser granularity than the individual entries emphasized in the title, and several supporting analyses are circular in a way that obscures this possibility. The knockout experiment ranks the entries of a trained full-LoRA adapter by their initialization scores and interprets faster degradation as causal evidence that the scored parameters mediate the adapter's effect (line 424). However, §D.8 shows that z-scoring the scores within each tensor, which preserves the within-module ranking and removes only the cross-module scale, makes the knockout curve indistinguishable from random removal. The knockout curve therefore indicates that some modules matter more than others, which the global top- selection already encodes. At fractions comparable to the circuit size (0.1% to 1%) the accuracy drop is small, and the gap emerges only between 20% and 75% removal. The interpretation of selected modules as attention readers and residual writers (§D.9) is introduced post hoc to account for the observed selection and does not yield a prediction that is subsequently tested. Because §D.8 indicates that selection depends on cross-module gradient scale, the overrepresentation of V and K, whose matrices have far fewer rows (256 in Qwen2.5-3B), may reflect the dependence of per-entry gradient magnitude on module shape rather than a functional role. The analysis in Fig. 4 raises a similar concern. Random and informed subsets with comparable weight norm but located in different modules produce different behavioral changes, and attributing this to the "leverage of the selected parameters" restates their location rather than explaining it. In addition, dense LoRA restricted to the two top modules matches full LoRA in all six GRPO configurations (Table 35), and element-level circuits change substantially with the seed or the scoring objective while module and layer allocations remain stable (Table 7 at , Tables 20 and 21). The control that would resolve this question is not reported, namely random entries sampled to match the circuit's allocation across modules and layers at the same and . If such a baseline performs comparably to and , the finding would reduce to the statement that sparse GRPO adapters should be placed in early-layer V, K and O projections. That result would remain useful, but it would constitute a different contribution, and it would overlap with prior findings that V is the most effective projection to adapt (SMT, He et al., ICLR 2025) and that the attention output projection carries much of the reasoning behavior (Shao and Wu, 2025).
The mechanism described in the first weakness is also not connected to the scoring rule. The paper argues that circuits select coordinates whose GRPO gradients have a consistent sign, but the circuits are scored with the supervised cross-entropy loss on reference solutions (§E.5), the sign-consistency analysis in §D.9 appears to use the same loss, and the only direct comparison shows that a cross-entropy circuit shares 11 to 13% of its entries with a GRPO-scored circuit (Table 20). The statement in the conclusion that the procedure "identifies a sparse subset of B elements with consistently signed gradients" is therefore not established for the GRPO gradient, and the statement at line 258 that RL objectives select "similar circuits" overstates Table 20. The use of reference solutions for scoring also admits a simpler account of the GRPO results. Under the zero-shot prompts used in the paper, base Qwen2.5-1.5B attains 9.5% on GSM8K and 17.4% on MATH-500, considerably below published few-shot results for models of this size (the Qwen2 technical report lists 58.5% on GSM8K for Qwen2-1.5B), whereas full LoRA after GRPO reaches 63.7%. A substantial part of the improvement may therefore be attributable to output-format compliance, a form of elicitation that RLVR is known to produce on Qwen2.5 models even with weak or spurious rewards (Shao et al., 2025). A cross-entropy gradient computed on reference answers encodes the expected format directly, so informed placement may succeed in part because it incorporates supervised information about the output format that is unavailable to random placement or to full LoRA under GRPO. Finally, the comparison between base and instruction-tuned models is confounded with model family and training setup. All base models are Qwen models trained with TRL in bf16, all instruction-tuned models are Llama models trained with Unsloth in 4-bit precision with the BNPO loss (§E.3), and Qwen2.5-7B-Instruct appears in the gradient analysis (Table 17) but not in the mathematical GRPO results. The observation that random placement performs better on instruction-tuned models may therefore reflect the model family or training configuration rather than instruction tuning.
In summary, the empirical observation described in the first strength appears reliable, but the explanatory account built around it, which constitutes the majority of the claimed contribution, is not established and is in several places inconsistent with the paper's own results. Most of the experiments needed to resolve these issues are inexpensive, and they are listed in the questions below.
Questions for authors
-
Which value of was used for the random SFT runs in Tables 2 and 31, as stated in Section 4 or 32 as stated in §E.4? How do the authors reconcile Table 31, in which random placement fails under SFT on GSM8K, with the claim that placement has limited effect under SFT? A matched comparison on GSM8K and MATH, using the same model, task, metric, and for SFT and GRPO, would address the first weakness directly. If random placement succeeds under SFT and fails under GRPO in that setting, I would raise my score.
-
Can the authors provide an interventional test of the noise mechanism? Specifically, does random placement under GRPO improve as the number of prompts or generations per step increases, and does random placement under SFT fail when the batch size is reduced or gradient noise is added? Please also specify what constitutes a step in Tables 1 and 17 for each regime (number of prompts, completions and tokens), the value of , and the fraction of GRPO groups with zero advantage at initialization for the 1.5B model. Given the comparatively aligned GRPO gradient for 7B on GSM8K in Table 17, why does random placement still fail in that setting? Do the gradient statistics remain similar when measured during training rather than only at ?
-
Please report random placement with entries sampled to match the circuit's allocation across modules and layers, at the same and , under GRPO for at least 1.5B on GSM8K and 3B on MATH. Results for random placement restricted to V and K, random placement restricted to the early layers, and top- selection within each module would also be informative. A clear advantage of and over the stratified baseline would support the element-level claim in the title and would improve my assessment.
-
The sign-consistency account and the distinction between and are not separated by the current experiments, since both scores perform similarly and select the same modules. What is the outcome of training on entries with high but low , that is, entries with large gradients of inconsistent sign? The proposed mechanism predicts that such entries should fail under GRPO.
-
Do the GRPO-scored and Dr.GRPO-scored circuits of Table 20 perform well when trained with GRPO? In addition, what is the per-coordinate consistency of the GRPO gradient ( across steps) on the cross-entropy circuit, compared with random entries in the same modules? This measurement would connect the scoring rule to the proposed mechanism.
-
For the base model, random placement, the circuits and full LoRA, what fraction of GSM8K and MATH-500 outputs contain an extractable answer (#### or \boxed{})? How much of the improvement remains under few-shot prompting or lenient answer extraction? Relatedly, does scoring with the cross-entropy of the model's own correct rollouts, instead of reference solutions, still yield effective circuits? This would distinguish informed placement from the injection of supervised format information.
-
Was the learning rate tuned for the full-LoRA baseline? Full LoRA is trained with a learning rate of and , whereas sparse runs use (Tables 26 and 27), and LoRA typically requires a larger learning rate than full fine-tuning. Since and the learning rate are nearly interchangeable under Adam, do circuits trained with and a correspondingly larger learning rate reproduce the results in Table 3?
-
Can the comparison between base and instruction-tuned models be performed within a single model family and training setup, for example Qwen2.5-7B against Qwen2.5-7B-Instruct on GSM8K and MATH using the TRL configuration of §E.2?
-
How do LoRA-XS, VeRA, rank-1 LoRA on V, or the TinyLoRA configuration of Morris et al. (2026) perform under GRPO on these base models at 10K to 50K parameters? Please also discuss the relation to Mukherjee et al. (2025), "Reinforcement Learning Finetunes Small Subnetworks in Large Language Models", who report that RL updates a sparse subnetwork and that training only that subnetwork recovers full performance. The relation to Yin et al. (2026), which the paper cites only in support of freezing , also requires discussion. That study reports that extreme parameter reduction (VeRA, rank-1 adapters, IA3 and LayerNorm tuning) fails under RLVR, and concludes that RLVR exhibits a strict lower bound on expressivity. The present submission reports that 1K to 50K entries of match full LoRA under GRPO, a budget well below those found insufficient by Yin et al. Can the authors explain what distinguishes their setting, for example the base model family, the task, or the scaling factor , and whether the vector-only methods of Yin et al. would also succeed under the authors' training configuration?
The remaining points are requests for clarification or suggestions. They do not individually determine my rating, but taken together they affect the reliability of the paper in its current form.
-
Several design choices are justified only under SFT or are not ablated under GRPO. The comparison between training A and B jointly and training B alone (§C.1) uses Qwen3-0.6B with SFT and perplexity rather than GRPO. The runs use both and (Table 26), so the same corresponds to different fractions of across configurations, and no rank ablation is reported under GRPO. The mask is fixed at initialization, and periodic re-scoring during training is not compared. The synthetic experiment would be more informative if the target structure, optimizer, batch size and noise level were varied one at a time.
-
The text-to-SQL, GPQA, AIME and AMC evaluations, Tables 11 to 13, and the gradient-structure tables appear to be based on single runs. With 30 AIME and 40 AMC problems, a single problem corresponds to 3.3 and 2.5 points respectively, so the claims in §4.4 and in the caption of Fig. 2 ("matches or exceeds Full LoRA across all settings") should be qualified. In Fig. 2(a), random placement at 50K reaches the same AIME 2024 pass@64 as and full LoRA, and in Fig. 2(b) it matches at pass@64. §B.2.2 also states that full LoRA performs best on AIME at 7B. The text-to-SQL statement that circuits remain within half a point of full LoRA holds only when the better of and is taken for each row, since is 2.4 points below full LoRA on Spider for the base model.
-
On which training run are the displacement ratios in Table 16 computed? If they are computed on circuit-trained runs, the result follows trivially from the mask. If they are computed on full-LoRA runs, why do the top-scored entries move only 0.24 to 0.41 times as much as random entries under SFT, given that SFT gradients are described as directionally stable? A knockout ordered by the magnitude of the trained entries would also be a useful control for Fig. 3.
-
Why does the GPQA row of Table 2 use rank 32, a setting in which full LoRA (36.6%) is below random placement (37.4%)? At rank 128 (Table 12), full LoRA reaches 50.7% and the circuits recover only 20% to 61% of its improvement. Random placement is reported as 37.4% at both 5K and 50K, which should be verified. With 198 questions in GPQA-Diamond, the binomial standard error is approximately 3.4 points, so this comparison is not conclusive in either direction.
-
The abstract states that the selected parameters are "concentrated in the attention value projection matrices across model families and scales". In Table 4, the O projection receives 59.9% of the budget for 7B on MATH, and in Table 9 Llama-3.2-3B allocates 91.5% to Down and 6.6% to V. The V projection is overrepresented relative to its parameter share, but the budget is not concentrated there. Similarly, the statement that random placement "fails to improve over the base model" does not hold for 7B on MATH (57.4% against 53.6%, Table 3), and §B.2 states that random placement at 7B "did not learn at this scale", which is inconsistent with that entry.
-
Several internal inconsistencies should be corrected. The caption of Fig. 3 identifies circuit knockout as red and random knockout as blue, whereas the legend and the plot show the reverse. Fig. 9 is captioned as Alpaca SFT, but §F.3 states that the 1.5B sweep uses OpenHermes. The 1.5B random curve in Fig. 9 (approximately 55% to 56% up to 50K) is consistent with neither Table 2 (61.6% at 50K) nor §F.3 (58.3% at 10K), and in the 3B panel random placement at 10K lies approximately 4 points below the base model. Line 241 refers to a 128-fold sweep of , whereas Table 13 covers a 64-fold range. The base and full-LoRA accuracies in Table 22 differ from those in Table 3 (for example 85.2% against 87.4% for full LoRA on 7B GSM8K), and Fig. 6 implies a 3B MATH base accuracy of 49.0% against 50.6% in Table 3.
-
Several smaller methodological points also require clarification.
- Table 6 measures overlap with a reference circuit computed from samples. If the samples are a subset of the reference samples, the reported overlap of approximately 98% is inflated, and overlap between disjoint draws would be more informative.
- The figure of 0.49% of SFT cost in Table 8 does not follow from §E.4, where 1,000 steps with 4 gradient accumulation steps imply that 50 passes correspond to approximately 1.25% of SFT. Please state the denominator used.
- Checkpoints for MATH are selected by test accuracy (§E.2). This matters for small margins such as at 1K against full LoRA (62.4% against , Table 11).
- The absolute gradient norms in Table 18 are compared across objectives with different loss scaling, so the statement that GRPO takes larger steps is not well defined, although the scale-free ratios are appropriate.
- The reproducibility statement indicates that code will be released upon acceptance. An anonymized repository during the discussion period would help resolve several of the questions above.
8 comments
part 2/n
Given the comparatively aligned GRPO gradient for 7B on GSM8K in Table 17, why does random placement still fail in that setting?
7B GSM8K is the only base-model GRPO cell where the gradient is partly aligned (cosine 0.41, net/step 0.78, against net/step 0.18 to 0.34 in the other five; Tables 1 and 17), and it has the smallest gain of informed over random placement (6.0 points; Table 3). On the same model, MATH-500 has an unaligned GRPO gradient (cosine 0.00 ± 0.08, net/step 0.20 ± 0.05) and a gain of 16.8. SFT on 7B GSM8K is more aligned still (cosine 0.97 ± 0.01, net/step 0.98 ± 0.01).
Please report random placement with entries sampled to match the circuit's allocation across modules and layers ... for at least 1.5B on GSM8K and 3B on MATH. Results for random placement restricted to V and K, random placement restricted to the early layers, and top-k selection within each module
Matched: in every B tensor, as many random entries as the circuit has there. Z-scored top-k: z-score the scores within each tensor, then take the global top-k, which keeps each tensor's own ranking at roughly parameter-share allocation (at 3B MATH, V's share of the budget falls from 22.5% to 0.5%).
| GRPO | Pretrained | Random | Matched random | Z-scored top-k | Ŝ | F̂ |
|---|---|---|---|---|---|---|
| 1.5B, GSM8K | 9.5 | 11.6 ± 3.1 | 27.7 ± 9.4 | 10.6 ± 0.3 | 65.2 ± 1.7 | 63.8 ± 2.6 |
| 3B, MATH-500 | 50.6 | 52.2 ± 1.4 | 54.1 ± 0.5 | 55.1 ± 0.3 | 62.2 ± 0.8 | 63.4 ± 0.9 |
| 7B, MATH-500 | 53.6 | 57.4 ± 0.9 | 65.7 ± 1.2 | 68.7 ± 0.3 | 74.2 ± 0.9 | 75.3 ± 0.7 |
| 7B, GSM8K | 79.3 | 80.9 ± 1.2 | 81.4 ± 0.4 | 82.2 ± 0.4 | 86.9 ± 1.0 | 87.2 ± 1.7 |
Random and the circuits are Table 3. At 3B MATH-500, random placement restricted to V and K reaches 54.6 ± 0.4, and restricted to layers 0–4, 50.9 ± 0.2.
Neither the allocation nor the within-tensor ranking alone recovers the circuit. At 1.5B GSM8K the within-tensor ranking alone stays at the random level (10.6 vs 11.6); at 7B GSM8K, matching the allocation adds nothing over random (81.4 vs 80.9).
The knockout curve therefore indicates that some modules matter more than others, which the global top-k selection already encodes.
Agreed; the knockout alone cannot separate allocation from entry identity. The training controls above do. We have revised §D.8 accordingly.
What is the outcome of training on entries with high F̂ but low Ŝ
Under the cross-entropy objective such entries barely exist: among F̂'s top 10K entries the median sign consistency (Σg)²/(NΣg²) is 0.90 (random signs: 1/N = 0.02; 1.5B GSM8K), so Ŝ and F̂ share 96% of their top-10K entries and train alike (Table 3).
Do the GRPO-scored and Dr.GRPO-scored circuits of Table 20 perform well when trained with GRPO? Relatedly, does scoring with the cross-entropy of the model's own correct rollouts, instead of reference solutions, still yield effective circuits?
Yes. Every circuit of Table 20 works under GRPO (1.5B GSM8K; random placement):
| Scoring objective | Ŝ | F̂ | Random |
|---|---|---|---|
| Cross-entropy (Table 3) | 65.2 ± 1.7 | 63.8 ± 2.6 | 11.6 ± 3.1 |
| GRPO | 60.1 | 62.6 | 10.6 |
| Dr.GRPO | 64.6 | 61.3 | 11.1 |
We score with cross-entropy because it is cheaper: 50 forward-backward passes and no sampling. A circuit scored on the 3B model's own correct rollouts reaches 58.8 on MATH-500 under GRPO, vs 52.2 random and 62.2 for the reference-scored circuit (Table 3); the two share 90.6% of their top-50K entries.
what fraction of GSM8K and MATH-500 outputs contain an extractable answer? ... the Qwen2 technical report lists 58.5% on GSM8K
Published base-model numbers are few-shot; every comparison here uses the same zero-shot prompt for every experiment, including the pretrained model. The graders do not reward format: MATH-500 is read with math_verify (the pretrained 3B / 7B models already use \boxed{} in 81% / 94% of outputs), and GSM8K parses the whole output with math_verify, using #### only to truncate. Where the pretrained models already score 50.6 to 79.3 (3B, 7B; Table 3), random placement gains 1.4 to 3.8 and the circuits 7.6 to 21.7. The self-rollout circuit above uses no reference solutions.
Can the comparison between base and instruction-tuned models be performed within a single model family and training setup
Table 36 is that comparison: four Qwen2.5-7B variants, one setup, GRPO with an execution reward.
| Spider | Pretrained | Random | Ŝ | F̂ | Full LoRA |
|---|---|---|---|---|---|
| Qwen2.5-7B | 57.8 | 56.0 | 69.2 | 66.6 | 69.0 |
| Qwen2.5-7B-Inst | 64.9 | 66.7 | 71.7 | 71.5 | 72.1 |
| Qwen2.5-Coder-7B | 63.4 | 65.4 | 70.7 | 70.2 | 70.1 |
| Qwen2.5-Coder-7B-Inst | 73.3 | 74.7 | 77.9 | 77.7 | 78.0 |
The gain of informed over random placement shrinks as the model moves toward the domain (Ŝ minus random: 13.2 on the base model, 5.0 and 5.3 on Instruct and Coder, 3.2 on Coder-Instruct), and informed placement tracks full LoRA throughout.
part 3/n
Since α and the learning rate are nearly interchangeable under Adam, do circuits trained with α = 32 and a correspondingly larger learning rate reproduce the results in Table 3?
As the reviewer notes, under Adam α = 32r at one learning rate and α = 32 at an r-times larger one give nearly the same update, so that run reparameterizes Table 3 rather than tests it. The question it stands in for, whether random placement fails at every step size, is answered by Table 13: random stays at the pretrained level across a 64× range of α, while the circuits improve with α.
How do LoRA-XS, VeRA, rank-1 LoRA on V, or the TinyLoRA configuration ... perform under GRPO ... the relation to Yin et al. (2026) ... Mukherjee et al. (2025)
Rank-1 LoRA on V (3B, MATH-500, 83K trainable parameters) reaches 52.0 under GRPO, no better than random placement (52.2), while the 50K-entry Ŝ circuit reaches 62.2 (Table 3). LoRA-XS, VeRA and TinyLoRA change the parameterization itself, training a few coefficients over fixed or random bases; we keep LoRA's parameterization and choose which entries of B to train, so we did not run them. Placement could be applied inside them, for example to choose which entries of the LoRA-XS core to train.
Yin et al. report that under RLVR rank-1 LoRA gains nothing over the base model, as our rank-1 V result does, and that LoRA-FA, which trains only B, matches LoRA, which supports training B alone. Mukherjee et al. find the subnetwork after RL fine-tuning, as the 5 to 30% of weights that changed; we find ours before training, from 50 forward-backward passes at initialization. We have added both to the related work.
The comparison between training A and B jointly and training B alone (§C.1) uses Qwen3-0.6B with SFT and perplexity rather than GRPO.
Training B with A frozen is the LoRA-FA configuration (Zhang et al., 2023). Under GRPO it gives up nothing in our settings: B-only circuits of 10K to 50K entries match full LoRA, which trains both A and B (Table 3). LoRA-FA has been studied in RLVR by this recent work (Yin et al. 2026) which shows that it can sometimes outperform LoRA.
no rank ablation is reported under GRPO.
At fixed rank, Tables 11 and 3 vary the budget tenfold (k = 1K, 5K and 10K, 0.005% to 0.05% of B at Qwen2.5-1.5B): random placement stays at the pretrained level at every budget while the circuits improve with k. Table 3 shows the same pattern at both ranks (r = 8 at 3B and 7B MATH and 7B GSM8K; r = 32 at 1.5B and 3B GSM8K).
The mask is fixed at initialization, and periodic re-scoring during training is not compared
The entries chosen at initialization stay useful throughout training: on the trained runs they keep moving in one direction (displacement ratio |B_T| / Σ_t |ΔB_t| of 0.82, where 1 means no back-and-forth, vs 0.59 for uniform random entries in the same tensors; 1.5B GSM8K), and the circuit matches full LoRA (Table 3). Periodic re-selection is a natural extension, and is how GaLore maintains its gradient subspace in pre-training and SFT. Under GRPO, consecutive gradients are nearly orthogonal (Table 1), so a criterion recomputed from a few steps risks switching between subsets on noise rather than tracking a real shift in importance, and it adds a recurring cost and a period hyperparameter.
Why does the GPQA row of Table 2 use rank 32, a setting in which full LoRA (36.6%) is below random placement (37.4%)? At rank 128 (Table 12), full LoRA reaches 50.7% and the circuits recover only 20% to 61% of its improvement. Random placement is reported as 37.4% at both 5K and 50K, which should be verified.
Thanks for catching this. We had a transcription issue when copying the table into the paper, and we now updated the paper text accordingly. In short, the corrected r32 results confirm your point; we now report r128 at k=5K. There, Ŝ and F̂ beat random by 12.1 and 9.9 points (paired z = 5.4 and 4.4 over the same 198 questions), while random stays at the base model. They recover 53% and 43% of full LoRA's gain (full LoRA corrected to 53.4 after the same fix), so they don't match full LoRA on this task.
Table 6 measures overlap with a reference circuit computed from N samples.
Recomputed on disjoint draws (Ŝ, GSM8K, top-10K): two disjoint 50-example draws overlap 96.5% (1.5B) and 95.0% (3B); against a disjoint 100-example reference, 96.8% and 95.6%. Table 6 now reports these.
part 1/n
We thank the reviewer for a careful review. We ran the experiments requested.
Same circuits, k, α and evaluation as the GRPO runs in Table 3. Mean and std over 3 seeds.
Random placement succeeds under SFT and fails under GRPO in every setting.
MATH SFT uses NuminaMath-CoT (GSM8K and MATH sources removed): SFT on the MATH train split scores below NumiaMATH for every method. We have added the table to the paper.
Table 31's SFT rows use a learning rate ten times below the default SFT learning rate of §E.4 (2 × 10⁻⁵ vs 2 × 10⁻⁴, α = 32). At that step size, random placement barely moves in 1,000 steps. With a larger effective step it succeeds: 62.1 ± 0.3 in the matched comparison above. Under GRPO, no step size helps: random placement stays at the pretrained level across a 64× range of α (Table 13).
For SFT, α was chosen per model and method by a sweep. The values used:
The α = 32r scaling in §4 applies to GRPO; we have corrected §4 and §E.4 to list these values.
Yes. Gaussian noise on the trainable-entry gradient, 3× its norm, once per step; nothing else changed:
Noise lowers random placement by 2.3 to 6.8 points; Ŝ moves by at most 1.1 and F̂ improves.
No. With 4× the completions per step (same total completions), random placement stays at the pretrained level: 49.4 at 3B MATH-500 (pretrained 50.6; 4 prompts × 8 or 1 × 32) and 54.6 at 7B (pretrained 53.6).
One training example for SFT; one prompt with 8 completions for GRPO. Zero-advantage prompts carry no gradient and are excluded, so every GRPO step in the tables carries signal; at initialization they are 37% of 1.5B GSM8K prompts and 61% of 1.5B MATH prompts. We have added these definitions to the captions of Tables 1 and 17.
They remain near-orthogonal during training: at trained checkpoints (3B MATH, steps 400 and 3000) the consecutive-step cosine of the GRPO gradient is −0.003 to 0.031 and net/step 0.16 to 0.20, against 0.61 to 0.96 and 0.81 to 0.97 for SFT at initialization (Tables 1 and 17).