acceptodds
Withdrawn from ICLR 2027

Representation-Correct Multi-Objective LoRA

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Modern post-training often requires balancing multiple objectives, such as helpfulness and safety, while low-rank adaptation (LoRA) provides a parameter-efficient mechanism for model adaptation. However, LoRA factors are non-identifiable: infinitely many factor pairs represent the same effective update. We show that this redundancy makes multi-objective optimization representation-dependent. Equivalent factorizations of the same adapter can change cross-objective gradient geometry, even reversing conflict signs and altering the Pareto weights selected by gradient-based methods. Moreover, even when the same effective combined gradient is fixed, updating equivalent raw factors can produce different next effective models. Thus, representation dependence arises separately in both multi-objective decision making and update execution. We formalize representation correctness for the complete optimizer and give conditions under which equivalent LoRA representations induce identical effective trajectories. Based on this principle, we develop Quotient Pareto Adaptation (QPA), which performs Pareto optimization directly in the effective fixed-rank geometry. We further derive Stiefel Pareto Adaptation (SPA) as a complementary representation-correct construction under a distinct geometry. Both extend to transported first-moment momentum with pathwise guarantees. We evaluate QPA and SPA on helpfulness-safety Pareto frontiers across Qwen and Llama models and on external safety benchmarks.

Primary area:
optimization
Keywords:
LoRA, Multi-Objective Optimization, Parameterization Invariance, Pareto Optimization, Representation-Correct Optimization, Large Language Models, Parameter-Efficient Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.80 over 5 reviews, above 27% of the others.
Reviewer_1uU3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper focuses on training LoRA adapters for multi-objective fine-tuning. It identifies an issue arising from the fact that the LoRA adapters are non-identifiable as infinitely many factor pairs correspond to the same adapter. Such non-identifiable factorization affects the gradients of each objective and thus the multi-objective optimization procedure as a whole. Focusing on the multiple-gradient descent algorithm (MGDA) for multi-objective optimization, two simple examples show that different factorizations result in either different objective weights or identical objective weights but different gradient updates. The paper then introduces two update rules for the LoRA factors ensuring identical optimization trajectories of distinct but equivalent parametrizations. This is achieved by re-casting optimization domain as the manifold of matrices with a fixed rank instead of the two LoRA factors, performing a gradient descent step on the tangent space of this manifold computed at the current LoRA adapter, and projecting the result back to the manifold via Frobenius norm minimization. Experiments show that the proposed methods are robust to equivalent perturbations of LoRA factors, in the sense that the perturbed factors do not result in contradictory gradients, as well as that the effective perturbed and unperturbed LoRA adapters remain very close during optimization when the proposed algorithms are used. Finally, a real-world example shows that DPO finetuning while balancing helpfulness and safety show that the methods achieves higher scores on these two metrics while improving hypervolume of the pareto frontier, HarmBench and XSTest scores.

Critical strengths

The problem is well motivated with clear examples, and the proposed solution solution elegantly extend the insight of BaLoRA, to perform optimization on the set of low-rank matrices, to multi-objective optimization on a manifold. The paper also extends the methods to stateful optimizers.

Critical weaknesses

The key motivating issue raised by the paper, namely "if two LoRA factorizations represent exactly the same effective model, they should induce the same multi-objective optimization behavior" (L040) is not solved by the proposed algorithms. In a narrow sense, each of the proposed algorithms satisfies this requirement in isolation (Theorems 5 and 6), however, "QPA and SPA can nevertheless select different Pareto weights, showing that representation correctness does not uniquely determine the optimization geometry" (L314) and "Representation correctness does not require QPA and SPA to choose the same optimization geometry." (L1259). This directly contradicts the "same multi-objective optimization behavior" desiderata, and since different algorithms result in different optimization trajectories, it is not clear why representation correctness is a desirable property at all.

The usefulness of such property is also questionable in light of the fact that "the simplex set producing an exactly zero combined factor gradient is identical across all equivalent full-rank factorizations" (L173) and "exact first-order Pareto stationarity is representation-invariant" (L176). Adapter re-parameterization thus only changes convergence properties of the optimization algorithm, and it is not clear why this is valuable from the paper. In fact, BaLoRA already showed that the convergence speed depends on adapter parameterization, but the experiments in the paper used the same number of steps and step size for all methods without showing convergence; it is thus unclear whether performance gain comes from faster convergence speed. If this is not the case, since "exact first-order Pareto stationarity is representation-invariant", the demonstrated performance gains are rather surprising and need to be explained.

The exposition is very hard to follow and exhibits typical LLM-isms such as name-dropping and talking about things without introducing them first. A few rounds of editing and proof-reading are necessary to clarify key aspects of the paper. Critically, the motivation for SPA, namely "SPA removes the non-orthogonal factor ambiguity through product-preserving Stiefelization" (L259) is highly unintelligible: what is the non-orthogonal factor, why is it ambiguous, why is it a problem, and why/how does "Stiefelization" solve this problem? It sounds like this ambiguity is a flaw of QPA that stands to be corrected, but experiments show these algorithms to be qualitatively different rather than SPA being the superior adaptation. What is then the need for SPA? The notation O\inO(r)O\inO(r) (L264) is ambiguous: does the big-oh notation refer to a matrix norm (which one?) or to individual entries of the matrix O? Another example is "a strict rank-r spectral cutoff gap for every accepted trial" (L252): what are accepted trials? What is the acceptance criterion? What matrix needs to have its spectrum altered, and why?

Questions for authors
  • Please clarify why representation correctness for an algorithm is a desirable property, even when (1) different algorithms can still exhibit different optimization trajectories and (2) stationary points in the loss landsape are not changed by different parameterizations.
  • If the performance gains in Table 2 are due to improved optimization speed, provide evidence in form of loss curves and hyper-parameter optimization; using the same hyper-parameters does not automatically establish superiority of the proposed algorithms.
  • If the performance gains in Table 2 are not due to improved optimization speed, please provide evidence that all models converged and that the best hyper-parameters were used for each of them, and explain why improved performance of converged models is to be expected even though stationary points are representation-independent.
Reviewer_zKm4OpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

LoRA factorizations (B, A) and (BQ, Q⁻¹A) give the same effective update, but they generally give different Euclidean gradient geometry. The paper argues that this makes gradient-based multi-objective optimization (MGDA in particular) representation-dependent at two stages: the decision (the Gram matrix and Pareto weights, Theorem 1 and Corollary 2) and the execution (the raw-factor update, Theorem 4). It proposes a “representation-correct” criterion for the complete optimizer, including optimizer state. It then gives two constructions: QPA, which works on the fixed-rank manifold, and SPA, which uses a Stiefel parameterization. Both have momentum variants. Experiments on Qwen2.5-7B and Llama-3.2-3B with PKU-SafeRLHF report conflict-flip rates under factor transformations, a decision/executor drift study, and helpfulness-safety trade-offs against five baselines.

Critical strengths

I am not an expert in LoRA or Riemannian optimization. I did not verify the proofs or assess novelty relative to prior LoRA optimizers (e.g., LoRA-RITE, BaLoRA). My comments concern presentation and the experimental evidence, and my confidence is low. For this reason, I cannot provide the strengths of this paper confidently.

Critical weaknesses

My overall assessment is that the paper maybe propose something useful or interesting, but the current structure suggests that the paper is not review ready and should enhance the information density to reduce readers' time. Without such reformatting, it's hard to recommend this paper for acceptance even if the paper proposes something novel because the burden would be imposed on readers. Plus, appendices G,I are not referenced if my check is correct.

The main text is not self-contained

It is fine to defer to appendices but my question is why the manuscript has so many white spaces, when the authors force readers to check their appendices. I will list what could be reduced to reserve spaces for appendix items to move into the main body:

  • Use inline equations for most equations. The main body refers to Eqs. 2, 3, 10, 21 only.
  • Figs. 1,3 could be compacted to get more spaces.
  • Algorithms 1,2 waste the half textwidth. They could be listed side by side with some refinement.
  • Table 1 is also wasting the side spaces.

Then, move the hidden contents to the main body,

  • The central notion, representation correctness of the complete optimizer.
  • proof sketches for Theorems 5 and 6 and Corollary 7.
  • The example showing that QPA and SPA select different Pareto weights supports the claim that correctness does not determine the geometry.
  • extended related work.

Of course, I don't think simply moving these contents to the main body is sufficient. Significant revision is necessary to make the manuscript coherent.

The statistical support for the task-performance claims is thin

The differences are small. On Qwen, HV is 0.02926 for SPA, 0.02861 for MGDA+AdamW, and 0.02850 for weighted DPO. On Llama, stateful QPA (0.01827) and QPA (0.01815) are nearly tied. Without uncertainty estimates I cannot tell whether the rankings are meaningful.

Questions for authors

Due to my review scope, I don't have methodological or novelty questions for authors. Please see Critical Weknesses for suggestions.

The ratings of presentation, informativeness, and soundness is based on my major revision assessment, and it may not necessarily reflect the novelty of this paper. That said, the novelty should be assessed after reformatting this paper and I will delegate such assessment to the other reviewers. Please note that since my concern is the formatting issue and the authors need to significantly revise the manuscript, I don't think the rebuttal would resolve the issue, but I am open to hearing any comments from the authors.

Reviewer_HuBkOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how equivalent LoRA factorizations can change gradient-based multi-objective decisions and subsequent updates. It proposes QPA and SPA, which use fixed-rank and Stiefel geometries to maintain representation consistency, including with transported momentum. Experiments examine representation sensitivity and helpfulness–safety trade-offs on Qwen2.5-7B and Llama-3.2-3B, with additional evaluation on HarmBench and XSTest.

Critical strengths
  1. The paper identifies a concrete issue: equivalent LoRA factorizations can change gradient-conflict judgments and objective weights despite representing the same effective model. Separating this effect from representation-dependent update execution provides a useful diagnosis for multi-objective LoRA optimization.

  2. The proposed optimizers have representation-consistency guarantees under explicit assumptions, including for transported momentum states. QPA also computes its geometric updates using ordinary LoRA gradients and compact linear algebra, making the construction applicable without forming dense effective gradients.

Critical weaknesses
  1. The technical advance over existing representation-aware LoRA optimization is limited. QPA and SPA largely combine established fixed-rank or Stiefel geometry with MGDA. The related-work discussion also omits Panacea [1] and PaLoRA [2], which already study multi-objective trade-offs using low-rank adaptation. These methods address different aspects of the problem, but discussing them is necessary to position the contribution. The representation-consistency analysis is useful, yet the experiments do not sufficiently isolate how much this property improves task performance.
  2. The experimental comparisons do not convincingly establish the method’s practical advantages. The weighted-DPO baseline uses equal objective weights throughout formal training, so its seven warm starts do not constitute a standard scalarization sweep. This limits the comparison of attainable Pareto fronts. Evaluation also covers only two backbones with different families and sizes, one training dataset, and a fixed LoRA rank. The gains vary across methods and backbones, leaving their consistency and scope insufficiently established.

[1] Zhong et al. Panacea: Pareto Alignment via Preference Adaptation. NeurIPS, 2024.

[2] Dimitriadis et al. Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences. ICLR, 2025.

Questions for authors
  1. How does the work relate to Panacea [1] and PaLoRA [2]? Please clarify its contribution within existing multi-objective LoRA research, as well as its technical advance over combining MGDA with fixed-rank or Stiefel LoRA optimization.
  2. Why does weighted DPO use equal weights during all 2,000 formal training steps? How do the results change when each run retains its respective scalarization weight throughout training?
  3. Figure 2 separates the decision and execution components and reports representation drift. How do these variants compare in task performance? This would help clarify the practical benefit of correcting each component.
  4. How sensitive are the reported performance differences to the learning rate? Do the main conclusions remain consistent across a reasonable range of learning rates?
  5. How do the results change across model sizes within the same family and across LoRA ranks? The different winning methods on Qwen and Llama make the scope of the gains unclear. If the authors can adequately address these concerns, I am open to revising my score.

[1] Zhong et al. Panacea: Pareto Alignment via Preference Adaptation. NeurIPS, 2024.

[2] Dimitriadis et al. Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences. ICLR, 2025.

Reviewer_aTgcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies representation dependence in multi-objective LoRA optimization. It shows that equivalent LoRA factorizations can alter gradient conflict assessments and MGDA weights, and that representation-invariant decisions alone do not ensure identical effective model updates. The authors formulate representation correctness for the complete optimization procedure and introduce QPA and SPA, based on fixed-rank and Stiefel geometries, respectively, with guarantees under stated assumptions and extensions using transported momentum. Experiments on Qwen2.5-7B and Llama-3.2-3B examine representation sensitivity and helpfulness–safety trade-offs on PKU-SafeRLHF, alongside external evaluations on HarmBench and XSTest. The results support the proposed invariance properties and show distinct trade-offs between the two geometries, while performance gains vary across backbones and metrics.

Critical strengths

The paper is clearly structured, and separating multi-objective decision making from update execution makes the problem easy to follow. The invariance claims are supported by explicit derivations under stated assumptions. The experiments also cover several relevant baselines and include both controlled representation-sensitivity tests and external safety evaluations, providing evidence beyond preference accuracy alone.

Critical weaknesses
  1. My main concern is that the task-level benefit of correcting representation dependence is not clearly established. QPA and SPA also change the optimization geometry and effective updates. Figure 2 shows that the correction reduces representation drift, but does not show how much it contributes to final performance. It is therefore difficult to attribute the reported gains to the specific problem the paper sets out to address.

  2. The Pareto-front comparison is not yet convincing. The seven points come from different warm starts with a single formal-training seed, while weighted DPO uses equal weights throughout formal training rather than maintaining different scalarization weights. Using the same nominal learning rate across optimizers also leaves tuning sensitivity unresolved. A stronger scalarization baseline and independent repetitions would help establish whether the hypervolume gains are robust.

  3. The results provide limited guidance on choosing between the proposed geometries. SPA performs better in hypervolume on Qwen, whereas Stateful QPA leads on Llama, and higher safety preference accuracy does not consistently improve external safety. Appendix G confirms that the geometries produce different updates, but does not explain these task-level differences. What determines when either geometry is beneficial remains unclear.

Questions for authors
  1. How much of the performance gain comes from fixing representation dependence? Could it instead come from the changes in optimization geometry and update rules?

  2. Do the hypervolume gains hold against a tuned weighted-DPO baseline using different fixed objective weights throughout training? Are these gains consistent across independent training seeds?

  3. Why does SPA perform better on Qwen, while Stateful QPA leads on Llama? What can we learn from these results about when to choose each method?

Reviewer_bz1ROpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper shows that equivalent LoRA factorizations can change gradient conflict judgments, MGDA weights, and effective updates. It introduces representation correctness for the complete optimizer and proposes two constructions: QPA on the fixed-rank matrix manifold and SPA with Stiefel-constrained factors. Both extend to consistently transported first-moment momentum. Experiments on Qwen and Llama examine representation sensitivity and helpfulness-safety trade-offs.

Critical strengths
  1. The paper identifies a concrete issue: equivalent LoRA factors can change gradient inner products and even reverse conflict signs.
  2. It usefully separates decision from execution. Matching objective weights alone does not guarantee matching effective updates.
  3. The analysis covers optimizer state, transport, and control flow, with both stateless and momentum variants.
  4. The claims are carefully scoped. Exact Pareto stationarity remains invariant, while update geometry can change. The paper also shows that representation correctness does not determine a unique geometry.
Critical weaknesses
  1. Novelty relative to existing manifold methods

QPA largely applies MGDA on a fixed-rank manifold with truncated-SVD retraction; SPA also uses standard geometric tools. The paper should clarify what is new beyond these constructions. A simpler baseline using balanced factors, raw MGDA, and SGD would help. AdamW + BaLoRA does not replace this comparison because its adaptive state introduces additional differences.

  1. The source of practical gains remains unclear

The conflict-flip and trajectory experiments establish representation sensitivity, but sensitivity alone does not imply poor task performance. QPA and SPA change the metric, execution, and state handling together, so Table 2 does not isolate the benefit of representation correctness. The impact of plausible factor scales and condition numbers on final performance also remains unclear. Since conflict judgments depend on the metric, representation invariance alone does not establish a preferred trade-off.

  1. Limited scalarization baseline and cost analysis

According to Appendix F.1, Weighted DPO uses different warm-start weights but equal objective weights throughout formal training. Its seven endpoints therefore do not represent a full weight sweep. Retaining different weights during formal training would provide a stronger comparison of hypervolume and trade-off coverage under matched budgets. Training time, peak memory, and rejection or backtracking rates would also help assess practical cost.

Questions for authors
  1. What does QPA add beyond standard Riemannian MGDA on the fixed-rank manifold, particularly in gradient recovery, state handling, and implementation?
  2. Could you compare against MGDA + SGD with balanced factors and rebalancing after each step?
  3. How much do equivalent initial representations affect final helpfulness, safety, and hypervolume under plausible factor scales and condition numbers? Can this be separated from the effect of changing geometry?
  4. Could you include a Weighted DPO weight sweep during formal training and report each method's hyperparameter search budget and selection rule?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.