acceptodds
Under review as a conference paper at ICLR 2027

PALORA: PARTIAL LOW-RANK ADAPTATION FOR LANGUAGE MODEL POST-TRAINING

Abstract

Low-rank adaptation (LoRA) reduces trainable parameters in language model post-training but applies the same update parameterization across target matrices. We introduce Partial Low-Rank Adaptation (PALORA), which directly trains selected Transformer projection matrices and retains LoRA elsewhere. Its probe–select–reinitialize procedure begins with short LoRA-only runs of Group Relative Policy Optimization (GRPO). Complementary binary masks yield absolute adapter-gate gradients of the GRPO loss on fixed held-out rollout groups. For each matrix, we normalize mean sensitivity by its standard deviation across probe questions plus a numerical stabilizer, then average scores across probe seeds. The lowest-scoring matrices receive direct updates: an empirical allocation rule rather than a diagnosis of insufficient adapter rank. After discarding probe weights, final GRPO training restarts from the original checkpoint with fresh adapters and a fixed allocation. We evaluate instruction-tuned Qwen2.5 models at 1.5B, 3B, and 7B parameters on five mathematical reasoning benchmarks. The 1.5B configuration directly trains 14 of 196 candidate matrices, retaining rank-16 LoRA at the remaining 182. At validation-selected checkpoints, PALORA improves mean accuracy across the five benchmarks over the reported LoRA baselines by 2.15, 3.18, and 4.26 percentage points, respectively. These single-seed comparisons use nonidentical parameter budgets and do not isolate placement effects.

Primary area:
foundation or frontier models, including LLMs
Keywords:
parameter-efficient fine-tuning, low-rank adaptation, LoRA, mixed-parameterization adaptation, language model post-training, reinforcement learning, GRPO, mathematical reasoning, module selection, sensitivity analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nN6iOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PaLoRA combines direct training of selected Transformer projection matrices with LoRA elsewhere. Short GRPO probes estimate adapter-gate sensitivity, and matrices with low variability-normalized scores receive direct updates after reinitialization. On five mathematics benchmarks, the reported mean gains over LoRA are 2.15, 3.18, and 4.26 percentage points for Qwen2.5-1.5B, 3B, and 7B, respectively.

Critical strengths

Table 2 reports improvements over LoRA across all five benchmarks at all three model sizes. At 3B and 7B, mean accuracy is within approximately one percentage point of Full-FT while training only a small fraction of the model parameters. The probe–select–reinitialize procedure is clearly defined, and the paper appropriately acknowledges that adapter sensitivity does not diagnose insufficient rank.

Critical weaknesses

W1. The current evidence does not isolate the benefit of the allocation rule. Table 2 compares methods with unequal parameter budgets, while Table 3 omits the parameter costs of alternative placements. Matching the number of selected matrices does not control for differences in trainable parameter counts. Moreover, all final training runs use a single seed (42); results from three probe seeds do not establish the reproducibility of the reported gains. Budget-matched comparisons against LoRA and random/high-score placements, together with repeated final training runs for the key comparisons, are needed to establish a reliable benefit from the allocation rule. W2. Important experimental details remain insufficiently documented. Table 1 specifies only the 1.5B PaLoRA configuration, while the 3B/7B settings, baseline ranks and learning rates, and selection procedures for the hybrid baselines are insufficiently specified in Sections 4.2–4.3. Appendix A provides little additional experimental detail beyond Figure 1, and the promised separate final-checkpoint results are absent. Reproducibility requires complete configurations and lists of selected matrices. The paper should also report final-checkpoint results and runtime and peak-memory measurements that include probing costs. Validation curves and an accuracy–cost plot would make the performance and efficiency comparisons easier to assess.

Questions for authors
  1. Can you provide matched-budget allocation comparisons and verify whether the main gains persist across independent final-training seeds (W1)?
  2. Can you supply the missing configurations, final-checkpoint results, and complete-pipeline cost measurements, with validation curves and an accuracy–cost visualization (W2)? These would materially affect my assessment.
Reviewer_87X6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PaLoRA, which combines direct training of the selected Transformer matrix with LoRA of the remaining matrix. It uses gradient based sensitivity scores from GRPO probes to select matrices. Experiments on the Qwen2.5 model have shown improvements over the standard LoRA on mathematical reasoning benchmarks.

Critical strengths
  1. This paper is generally well-organized. The methods and training procedures have been clearly described.
  2. The experiment covered three model sizes and five mathematical reasoning benchmarks. The results indicate that compared to LoRA, its performance continues to improve.
  3. This paper includes several baseline methods and ablation studies to test the effectiveness of their design choices.
Critical weaknesses
  1. The novelty is somewhat limited. The main idea of combining selective full parameter training with LoRA is relatively simple.
  2. The comparisons do not use exactly matched trainable-parameter budgets. Therefore, it is difficult to determine how much improvement the proposed selection strategy has brought.
  3. All final training experiments use only one seed. This makes it difficult to determine the stability of the improvement.
  4. This method requires additional GRPO detection, but the paper did not report the end-to-end computational cost.
Questions for authors
  1. Why do matrices with lower adapter sensitivity benefit more from direct training? Is there stronger evidence to support this selection rule?
  2. Can the author compare under matching parameter budget and multiple training seeds?
  3. How much additional computational cost does the detection phase introduce compared to standard LoRA training?
Reviewer_JdD9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PALORA directly trains selected projection matrices and uses LoRA elsewhere. Short LoRA-only GRPO probes produce gate-gradient scores under complementary masks, and low-scoring matrices are assigned direct updates. The probe weights are then discarded before final training. Experiments report math gains across three Qwen2.5 model sizes.

Critical strengths
  • The mixed adaptation scheme is straightforward, and the reported gains over LoRA cover all five benchmarks at each model size.
  • Discarding the probe weights separates matrix selection from a training warm start. Checkpoint selection uses internal validation rather than the external benchmarks.
Critical weaknesses
  • I cannot tell how much of the gain comes from choosing better locations. Table 3 omits parameter counts, although selecting the same number of matrices can give different budgets. Table 1 documents only one 1.5B configuration; the baseline ranks and 3B/7B settings are needed to interpret the main results.
  • The score in Eq. (5) needs a stronger check. Each probe uses only 16 questions, with a different mask for each, so its standard deviation mixes question and mask variation. Clipping and KL gradients in Eq. (4) could also affect the allocation, but their contributions are not examined.
  • All final-training results use seed 42. The normalization gains in Table 3 are only 0.64–1.03 points, so repeated runs could change the conclusions about the scoring details. The cost of the three probe runs and scoring is also missing.
Questions for authors
  • What is the exact C(ξ)C(\xi) for each Table 3 row, and what LoRA ranks are used in Table 2? Does low-score placement still beat high-score and random placement at a fixed parameter budget?
  • How many of the 14 selected matrices change when masks are resampled on the same probe questions? In Eq. (4), how much of the gate gradient comes from the policy surrogate versus KL?
  • Do the main gains persist across several final-training seeds? Please also report total GPU-hours, including the three 300-step probes and scoring, and compare with LoRA given the same total compute budget.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.