acceptodds
Under review as a conference paper at ICLR 2027

Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptation

Abstract

Personalized large language models often require a complete adaptation state for each user. However, this paradigm scales poorly as the user population grows. We revisit this design through the lens of personalization capacity allocation: how much adaptation capacity can be shared across users, how that shared capacity should be composed, and how much adaptation must remain user-specific. We answer these questions through three complementary empirical analyses. We find that independent user adapters contain substantial cross-user reusable structure, that the utility of reusable directions reflects both user relevance and variation across queries, and that user histories provide transferable signals for compact individual correction. Motivated by these findings, we propose LINEUP. It learns a bank of reusable low-rank personalization factors, composes them through user-conditioned recall and query-dependent calibration, and restricts target-user adaptation to a tiny user code over a shared correction space. This design decouples expressive personalization capacity from per-user trainable state. Each target user optimizes only eight scalars, while all shared components remain fixed. By comparison, the evaluated private-LoRA configuration uses 4.19 million per-user parameters. Our theoretical analysis gives a finite-step, finite-history risk bound and sufficient conditions for user-code refinement to improve on history initialization. Across six tasks spanning personalized classification, prediction, and generation, LINEUP leads on all 12 metrics, each averaged over three independent runs (e.g., reducing LaMP-3 RMSE by 11.4% relative to the strongest baseline). It maintains advantages under limited history. These results show that rich personalization can be supported primarily by reusable, conditionally composed shared capacity, while independent user adaptation remains confined to a tiny correction state.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Personalization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_F2p9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

This paper asks whether personalized LLMs need a separate adapter for every user and then conducts several prelimilary experiments. It proposes LINEUP, which combines shared low-rank factors using the user’s history and current query, then adapts an eight-dimensional code for each user. Experiments on LaMP and LongLaMP report better performance than the evaluated baselines with much less per-user parameter storage.

Critical strengths
  1. Compelling perspective. Prior work has explored how to balance shared and user-specific LoRA components, but this paper offers a useful capacity-allocation perspective: what should be shared, conditionally composed, and adapted for each user.

  2. Clear empirical motivation. The analyses in Section 3 provide a coherent motivation for the design in Section 4. Cross-user reuse motivates the shared factors, variation across users and queries motivates conditional composition, and transferable corrections from user history motivate the compact user code. These experiments do not constitute a formal derivation, but they provide reasonable support for the design choices.

Critical weaknesses
  1. Reproducibility of the experimental results. Although the appendix provides some implementation details and the authors state that the code will be released after de-anonymization, code and a complete reproduction package are not currently available. Given that LINEUP reports the best mean result on every evaluated metric, it would be helpful to provide an anonymous implementation, exact data splits, and evaluation scripts so that these results can be independently verified during review.

  2. Limited evidence of generality. My major concern is the breadth of the evaluation. The LongLaMP results in the appendix are welcome, but LaMP and LongLaMP belong to a closely related family of history-based text tasks. It remains unclear whether the proposed capacity-allocation strategy transfers to other personalization settings, such as evolving user preferences or multi-turn assistance. Evaluation on at least one benchmark outside this family would substantially strengthen the paper’s generality claim.

Questions for authors

Please see the weaknesses above for my main concerns. Overall, I enjoyed reading this paper and found its perspective interesting. As someone familiar with this area, I look forward to the public release of the code.

Reviewer_PzCoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LINEUP, a lightweight framework for personalizing large language models. It moves most of the personalization capacity into shared parameters through a shared bank of low-rank factors, history-guided factor retrieval, and query-dependent dynamic composition. Each user then needs only an 8-dimensional user code for individual correction. The authors support the design with both empirical and theoretical analysis, and evaluate it on six tasks and it achieves the best average performance on all 12 metrics while cutting the extra per-user storage.

Critical strengths
  1. The problem is practically relevant, and the method design is grounded in empirical observations. The paper targets the storage cost of the many user-specific parameters in personalized models. Its experiments examine the structure shared across users, how useful each factor is for a given user and query, and the role of user history in individual correction. Together these provide a sound basis for LINEUP’s design.
  2. The experiments are quite comprehensive, and the ablations support design of the method . The method is evaluated on tasks of many scenarios and compared with many personalization baselines. The ablations show that history-guided factor retrieval and query-dependent factor composition clearly affect performance, and that the user-code correction adds a further gain.
Critical weaknesses

The novelty of the method, and what’s the difference about prior work, are not quite clear. Sharing and reusing personalized parameters is not new (e.g., MTA: A Merge-then-Adapt Framework for Personalized Large Language Models (Li et al., ACL 2026)). LINEUP’s factor learning, dynamic composition, and low-dimensional user code do have difference from these, but it doesn’t explain how significant the differences are or which account for the improvements.

The personalization correction module contributes relatively little on its own. In the ablations, S1 alone already comes close to the full model’s performance, and adding S2 improves results only slightly. This weakens the paper’s emphasis on a compact user-specific correction mechanism, and leaves the necessity of that module, along with the practical value of its theoretical analysis, still to be demonstrated.

Questions for authors

How are source users and target users split? LINEUP learns the shared factor bank from source users and then applies it to target users who took no part in training the shared components. In real deployments, though, it is usually impossible to know in advance which users will be used to train the shared components and which will be future targets. Could the authors explain the rationale for the current split, and say whether the method still works when the user population’s distribution shifts? The experiments account only for randomness in training, not for randomness in how users are split into source and target groups.

How robust are the results to the user split? The paper reports results across multiple random seeds, but randomness in model training is different from randomness in the source/target split. How much do the current results depend on one particular split? If the models were retrained and evaluated on several different source/target splits, would LINEUP’s advantage over the baselines remain stable?

How much does S2 actually contribute, and how does it relate to the paper’s core motivation? The ablations suggest that most of the gain comes from the shared-factor composition module S1, while the low-dimensional user-code correction S2 adds only a small improvement. Since the paper stresses keeping a lightweight individual adaptation on top of shared personalization capacity, could the authors analyze the tasks or user conditions where S2 is most valuable? Are its gains enough to justify the extra adaptation procedure and the accompanying theoretical analysis?

Reviewer_tL1gOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how much of a personalized adapter is actually personal. It presents three empirical analyses: independently trained OPPU adapters share a small subspace that captures much of their update energy; direction utility has both user-associated structure and query-specific variation; and user histories provide transferable correction signals that compact spaces retain. It then proposes LINEUP, which learns a bank of 32 rank-4 shared LoRA factors, composes them through history-conditioned recall and query-dependent calibration, and confines target-user adaptation to an eight-dimensional code refined on support examples. The evaluation covers six LaMP tasks with Llama-2-7B, three seeds, eight baselines, ablations, cold-start and storage analyses, supplementary LongLaMP results, and a finite-step risk bound.

Critical strengths
  • The paper provides a clear motivation for moving beyond private LoRA adapters, supported by empirical evidence of reusable structures across users.

  • LINEUP combines history-based factor retrieval, query-dependent composition, and lightweight user-specific correction in a coherent framework that balances personalization quality and storage efficiency.

  • The experiments cover six LaMP tasks and multiple baselines, with LINEUP achieving the best mean performance on all 12 metrics. The ablation and cold-start studies further support the effectiveness of the proposed design.

Critical weaknesses
  • LINEUP performs well overall, but the evidence for some smaller gains is limited. The improvement on LaMP-4 and LaMP-2N Accuracy are small. Besides, the evaluation is limited to Llama-2-7B and LaMP benchmarks, so it remains unclear whether LINEUP generalizes to other model architectures and settings.

  • The comparison does not clearly isolate the benefit of shared capacity from the effect of adaptation rank. OPPU uses a rank 8 per-user LoRA, while LINEUP uses four rank-4 factors, giving an effective rank of up to 16. Thus, comparing S1-only with OPPU in a shared rank 16 LoRA and a rank 16 OPPU baseline, or other fair comparasion, would help establish solid evidence of this paper.

  • The ablations do not fully establish the robustness of the chosen configuration. The user-code dimension is varied, but the number of shared factors (32), factor rank (4), and active-set sizes are not systematically tested. The storage advantage also depends on the number of users. At 1,024 users, LINEUP requires substantially less storage than PerFit and OPPU, but at LaMP-2N’s actual scale of 49 users, its estimated storage (151.2 MB) exceeds PerFit’s (about 79.3 MB). Showing both regimes more clearly would help establish the practical scope of the scalability claim.

Questions for authors

Please see Weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.