acceptodds
Under review as a conference paper at ICLR 2027

Post-Hoc Paired-Coordinate Reduction in LoRA-RL Reasoning Adapters

Abstract

Parameter-efficient reinforcement learning makes reasoning post-training practical, yet completed low-rank adapters are typically retained at the rank chosen before or during optimization. Existing adaptive-rank methods mainly modify training itself, leaving open whether a completed LoRA-RL factorization contains removable capacity. We study this question through post-hoc paired-coordinate selection: checkpoint-derived scores rank shared LoRA factor coordinates, a fixed Top-k set is sliced consistently across modules, and the original scaling is preserved without retraining or using downstream task outcomes for selection. On a rank-32 LoRA-RL adapter, evaluations across MATH500, AIME25, and AMC23 at reveal substantial but task- and budget-dependent retention relative to random selection and per-module SVD reconstruction. Validity diagnostics further show that successful endpoints do not imply intrinsic or representation-invariant coordinate importance. As a cross-family test, applying the Energy selector to a separately trained DoRA-r32 adapter retains MATH500 and AMC23 performance at k = 16 while AIME25 decreases, showing that post-hoc reduction transfers beyond the original LoRA parameterization but remains task dependent. These results identify usable post-hoc redundancy while bounding its interpretation to the stored factorization and evaluation protocol.

Primary area:
reinforcement learning
Keywords:
ML: Other Foundations of Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oFk4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies post-hoc rank reduction of trained LoRA-RL reasoning adapters without retraining. It selects paired LoRA coordinates using checkpoint-derived scores and evaluates the reduced adapters on three mathematical reasoning benchmarks. The results show substantial but task-dependent redundancy, while additional analyses suggest that coordinate importance depends on the stored factorization. A DoRA experiment provides limited evidence of transfer to another adapter type.

Critical strengths
  1. The proposed post-hoc reduction retains most of the original performance after removing up to 50% of the LoRA parameters, demonstrating meaningful redundancy in trained reasoning adapters.

  2. The paper explicitly distinguishes factorization-specific coordinate selection from intrinsic coordinate importance and reports validity checks designed to test this distinction.

Critical weaknesses

The effectiveness of the proposed method is not convincingly demonstrated. The coordinate-scoring method does not consistently outperform random selection or SVD, and the paper’s own analysis shows that the scores do not reliably reflect the actual contribution of individual coordinates. Therefore, the experiments mainly demonstrate redundancy in LoRA adapters rather than the effectiveness of the proposed selection method.

Questions for authors
  1. Can the authors clarify when and why the proposed scoring method performs better than random selection or SVD? Is the improvement statistically significant in any setting?

  2. Do the LoRA and DoRA experiments use the same DeepSeek-R1-Distill-Qwen-1.5B backbone? If so, what evidence suggests that the findings generalize to other model sizes or model families?

2 comments
AuthorsOpenReview ↗

Response to Reviewer oFk4

Thank you for asking when Energy performs better than Random or SVD, and whether the observed differences support a statistical claim. We also clarify what the DoRA comparison does and does not establish about backbone generalization.

Question 1: Can the authors clarify when and why the proposed scoring method performs better than random selection or SVD? Is the improvement statistically significant in any setting?

On AMC23 at r20r20, with Random coordinate-subset seed 42, Energy scores 25/40 versus Random's 23/40 at generation base seed 47 (+5.0+5.0 percentage points) and 28/40 versus 25/40 at seed 89 (+7.5+7.5 points; Supplement Table 16). Against SVD-DeltaW at r20r20, Energy scores 28/40 versus 25/40, but only 20/40 Energy--SVD question pairs have verified matching physical-GPU provenance (Supplement Table 17). It does not lead at seed 23 (25/40 versus 26/40), and the fixed-generation-seed comparisons in Supplement Table 15 are mixed: four Energy wins, one tie, and one loss across the six r16/r20r16/r20 comparisons.

Energy measures the magnitudes of the stored, unscaled rank-one terms; it does not measure their marginal accuracy contributions, so the observed differences do not establish a mechanism. The reported values are per-question paired observed differences. Statistical significance has not been established. The archived bootstrap intervals cannot be exactly reproduced because the original bootstrap implementation is not bound to the available files. These point estimates are descriptive.

Question 2: Do the LoRA and DoRA experiments use the same DeepSeek-R1-Distill-Qwen-1.5B backbone? If so, what evidence suggests that the findings generalize to other model sizes or model families?

The archived LoRA and DoRA configurations identify the same named DeepSeek-R1-Distill-Qwen-1.5B backbone. The DoRA evaluation contract records base-file fingerprints matching the recovered base-model manifest, but the available records do not fully bind that immutable base-weight version to the original M1 LoRA training invocation. The DoRA experiment tests another adapter parameterization on the recorded named backbone; it does not establish generalization to other model sizes or model families, and the independent training runs do not isolate a causal adapter-parameterization effect. See main-text Section 4.5 and Table 3, and Supplement Section J.1 and Table 14 for the DoRA protocol and results.

Thank you for the careful and transparent clarification. The response confirms that the advantage of Energy over Random or SVD is mixed and not statistically established, while the DoRA experiment does not demonstrate generalization across model sizes or families. These remain central concerns regarding the proposed selection method, so I maintain my current rating.

Reviewer_X4R7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies post-hoc paired-coordinate reduction of a completed rank-32 LoRA-RL reasoning adapter. Checkpoint-derived Energy or growth-minus-energy scores select one shared Top-k coordinate set across modules, with the original scaling preserved and no retraining. On one DeepSeek-R1-Distill-Qwen-1.5B adapter, the authors evaluate retained ranks 16, 20, and 24 against the full adapter and per-module SVD-DeltaW, with a Random control at rank 24, on MATH500, AIME25, and AMC23. A separate DoRA-r32 test and validity diagnostics limit the interpretation to the stored factorization and evaluated tasks.

Critical strengths

The paper precisely defines shared paired-coordinate slicing and preserves a_k / k = 4. This makes the post-hoc intervention and its comparison with SVD-DeltaW reproducible in principle.

The SVD-DeltaW and Random comparisons distinguish paired-coordinate selection from per-module reconstruction. The authors explicitly avoid claiming intrinsic or representation-invariant coordinate importance.

At k=16, the LoRA adapter has 50% fewer adapter parameters while MATH500 remains 83.96% versus 84.50% for the full adapter. The separate DoRA test adds limited cross-family evidence.

Critical weaknesses

Table 1 reports Random selection only at k=24. Matched Random controls at k=16 and k=20 would clarify whether retention at stronger reduction depends on checkpoint-derived score order or also occurs with random subsets of the same stored coordinates.

A minor reporting concern is that Table 1 uses unequal generation-seed counts and pools pruning-subset and generation-seed variation for Random. Summarizing the existing paired comparisons with Full would make the magnitude and uncertainty of performance changes easier to assess.

Questions for authors

On the same source adapter and evaluation protocol, how does Energy compare with Random at k=16 and k=20?

Can the authors summarize the existing paired comparisons with Full using matched generation seeds and distinguish pruning-subset variation from generation-seed variation for Random?

1 comment
AuthorsOpenReview ↗

Response to Reviewer X4R7

Thank you for the specific requests for same-rank Energy-Random comparisons at k=16k=16 and k=20k=20, and for matched comparisons with Full. We distinguish variation across Random coordinate subsets from variation across generation base seeds below.

Question 1: On the same source adapter and evaluation protocol, how does Energy compare with Random at k=16k=16 and k=20k=20?

At generation base seed 89, Energy-r16 scores 26/40 against Random-r16 scores of 25/40, 22/40, and 27/40 for coordinate-subset seeds 42, 2025, and 2026, respectively; the paired differences are +2.5+2.5, +10.0+10.0, and −2.5-2.5 percentage points. Energy-r20 scores 28/40 against Random-r20 scores of 25/40, 24/40, and 28/40, for +7.5+7.5, +10.0+10.0, and 0.00.0 points. Supplement Table 15 reports all six matched comparisons and paired correctness counts; Energy leads in four, ties in one, and trails in one.

Question 2: Can the authors summarize the existing paired comparisons with Full using matched generation seeds and distinguish pruning-subset variation from generation-seed variation for Random?

At generation base seed 89, Energy-r16 and Full-r32 both score 26/40, with four exclusive correct answers on each side. Energy-r20 scores 28/40 versus Full-r32's 26/40, with three Energy-only and one Full-only correct answers. In the historical MATH500 runs, Energy-r16 scored 84.8% versus 83.2% for Full-r32 at generation seed 89 (+1.6 pp), and 85.4% versus 85.8% at seed 167 (-0.4 pp). These are matched-seed run-level differences. Supplement Table 15 varies the Random coordinate-subset seed while fixing the generation base seed at 89; Supplement Table 16 fixes the Random coordinate seed at 42 while varying the generation base seed. In Table 16, Energy-r20 scores 25/40 versus Random's 23/40 at seed 47 (+5.0+5.0 points) and 28/40 versus 25/40 at seed 89 (+7.5+7.5 points); the additional seed-23 result is 25/40 versus 26/40 (−2.5-2.5 points). The paired conditions use the same 40 question IDs, and the recorded item-level manifests verify matching physical-GPU UUIDs for all 40 Energy--Random pairs within each batch. Reuse of the question set means these are descriptive per-seed outcomes, not additional independent questions or evidence of seed robustness.

The original Table 1 pools Random coordinate-subset and generation-seed variation, so its SD does not isolate either source. Section 3 describes SVD-DeltaW as independent rank-kk reconstruction of each module's dense update, whereas Energy and Random select shared stored coordinates. The matched AMC23 panels are supplementary diagnostics and do not establish statistical superiority.

Reviewer_aXjtOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the problem of pruning a trained LoRA adapter. Focus seems to be placed on the particular setting of RL-trained reasoning models; experiments use DeepSeek-R1-Distill-Qwen-1.5B. The paper introduces a particular technique for pruning adapters, based on certain energy scores.

Critical strengths

None

Critical weaknesses

Writing: I found the paper's writing style to be very bizarre and difficult to follow, with many sentences being nearly impossible to parse. These issues are noticeable as early as the abstract and the introduction: the problem of pruning LoRA adapters is barely motivated, no numerical results are stated in the introduction, and technical terms are used before being properly introduced. (E.g., what is "the Energy selector" of lines 22?.) I have no choice but to suspect that the paper is virtually entirely LLM generated. I believe that this issue alone renders the paper unfit for publication.

Some examples of LLM-ey writing:

  • Line 59: "Component energy, random subsets, and SVD-DeltaW provide matched post-hoc comparisons, but they do not define a universal coordinate semantics."
  • Line 62: "Our procedure reads saved adapter eights only after GRPO training. It introduces neither a policy update nor a new RL optimizer; in particular, rewards, verifiers, and task outputs are not pruning-score inputs."
  • Line 85: "Training-time direct-rank, AdaLoRA, and GeoRA results remain methodological context rather than matched numerical baselines."

Results: The energy-based technique introduced in the paper appears to be fairly convoluted, and I struggled to understand it. Then, the experimental results seem to be extremely weak. In Table 1, random selection and the paper's energy-based methods are statistically indistinguishable for the 3 datasets considered. (Also, the table is structured very awkwardly and is difficult to read.) This seems to suggest that the method does not outperform even the simplest baseline.

Questions for authors
  • The AI use statement reads: "All ... final manuscript content were reviewed and verified by the authors..". Is this true? Do the authors understand the technical content in the paper?
  • Tables 1 seems to suggest that the energy-based method introduced by the paper does not outperform a simple random baseline, correct? Doesn't this undermine the paper's claimed contribution?
2 comments
AuthorsOpenReview ↗

Response to Reviewer aXjt

Thank you for asking directly about our AI-use statement and the technical meaning of the method. We also clarify what the original Table 1 establishes about Random and how the new matched AMC23 panel relates to the contribution.

Question 1: AI disclosure and technical content

The manuscript's AI Use Statement reads: “We used large language model tools to assist with code and workflow debugging, manuscript language editing, and consistency and audit checks. All reported experiments, numerical results, citations, scientific interpretations, and final manuscript content were reviewed and verified by the authors, who take full responsibility for the work.” For example, coordinate ii pairs column ii of BB with row ii of AA, and retaining the same ordered indices slices those paired factors. With r=32r=32, α=128\alpha=128, and k=16k=16, setting αk=64\alpha_k=64 preserves αk/k=4\alpha_k/k=4 while halving only the LoRA A/BA/B parameters for fixed target modules and widths. The frozen backbone is unchanged, while the dense update generally changes because coordinates are removed.

Local CPU scripts recalculated the 15 numbered formulas and checked the source adapter's Energy-selected coordinates at retained ranks 16, 20, and 24 against saved manifests. These were script-based checks, not author manual calculations or verification of which target weights a formal evaluation worker loaded.

Question 2: Random baseline and contribution

On the matched AMC23 panel, Energy-r16 retained the Full-r32 score of 26/40 while using half as many LoRA A/B parameters, with no retraining. Each condition is exclusively correct on four questions, so equal total scores do not imply identical item-level correctness. At generation base seed 89, Energy-r16 also scores 26/40 versus 25/40 for Random-r16 with coordinate-subset seed 42 (Supplement Table 15). At r20r20 with the Random coordinate seed fixed at 42, Energy minus Random is −2.5-2.5, +5.0+5.0, and +7.5+7.5 percentage points at generation base seeds 23, 47, and 89, respectively (Supplement Table 16 and its additional seed-23 result). With generation seed 89 fixed, the six r16/r20r16/r20 comparisons across three Random subsets include four wins, one tie, and one loss (Supplement Table 15). The original main-text Table 1 reports Random only at k = 24 and does not establish a consistent Energy advantage over Random. The new matched AMC23 panel shows that rank-16 post-hoc compression can preserve the full adapter's total score in this setting.

Reviewer_aXjtOpenReview ↗

Response

I thank the authors for their response.

I maintain my score; I believe the paper has serious presentation issues and its method often fails to outperform a simple random baseline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.