- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PersJEPA takes ideas from JEPA to personalize a frozen LM while keeping the user’s profile out of the prompt at inference time. But training still uses user information. The authors pass each answer through the LM twice, once with that information and once without, and use the difference between the hidden states as a training target.
They also group users into different clusters and train a small sparse model for each cluster to predict this difference from the request alone. These smaller models get another round of tuning to make the user’s answers more likely under the frozen LM. The authors also introduce SynPer, their own LLM-generated benchmark with ten personas, to evaluate the method.
Critical strengths
The paper offers a practical way to steer a frozen language model without including the user’s profile in each inference prompt. It learns small predictors and applies their outputs throughout generation. This supports the feasibility of the approach for controlled persona generation.
Critical weaknesses
-
PerFit [1] already studies how adding user information changes hidden states. It separates this change into shared and user-specific parts, learns adjustments for both, and evaluates them on the same LaMP tasks.
-
L2S [2] trains a small network to predict the hidden-state change caused by contrastive prompts using only the input. It applies the predicted change at each generated token. This is very close to the first stage of PersJEPA.
The ablations do not show what the proposed components add - A dense predictor without residual loss performs as well as the full model (94.0% vs. 93.2%).
Fixed group vectors obtained by averaging the trained predictor’s outputs still reach 87.4%. This raises questions about whether the separately trained group-vector baseline (62.6%) was optimized well enough.
Relatively limited evidence for personalization .
On LaMP, a global predictor without user information recovers 88–98% of the gains. Only LaMP-2 shows a notable user-specific gain (+3.2 points). With just 20 groups for thousands of users, personalization largely happens at the group level. It is unclear how the method captures differences between users within a group.
The large gains come from SynPer, where the true labels assign ten synthetic personas to experts. Its classifier is trained on the same synthetic answers the method learns to reproduce, yet the paper reports neither its accuracy on reference answers nor a human evaluation. Increasing the steering scale also lowers F1.
[1] J. Liu et al. PerFit: Exploring Personalization Shifts in Representation Space of LLMs. ICLR 2026.
[2] J. Parekh et al. Learning to Steer: Input-dependent Steering for Multimodal LLMs. NeurIPS 2025.
Questions for authors
Without clarification, the following points limit my confidence in the reported results.
-
Table 2 numbers evaluation. - If each percentage is calculated from 500 examples in a single run, the values should be multiples of 0.2%. The other values in the 9B and Llama blocks follow this pattern, but the SteerX and BiPO rows include 2.7%, 2.3%, 15.9%, 10.9%, and 21.3% - Please clarify how these scores were calculated or averaged.
-
What does “cold start” mean in Figure 3d if these users’ histories were used to train the predictor? And how many history entries were actually used at k=20 when each user has at most eight? Why does the unsteered baseline differ from Table 1 (36.1% vs. 36.3%)?
The following points need to be addressed to make the paper more solid and convincing.
- Explain what PersJEPA adds beyond PerFit, L2S, and other steering methods. What do residual prediction and sparsity contribute when a dense predictor performs similarly?
2.Resolve the inconsistencies listed in Part I above, and document the settings used to produce the reported results.
Response to Reviewer CKuf Part2/2
Weakness 2 & Question 2: W2: The results on the synthetic benchmark and LaMP-2 are strong, while the gains over strong baselines on the other datasets are limited or inconsistent. Q2: Do the authors have an explanation?
Answer to W2 & Q2:
The differences from Trained vectors are statistically significant under a user-level paired bootstrap. PersJEPA is ahead on four of the five datasets against both baselines. Amazon is the exception on this backbone: it is the one set where a user's signal is individual rather than shared by a group.
Weakness 3: The presentation is occasionally hard to follow due to the density of technical details.
Answer to W3: We will add a one-table summary of the four stages (calibration, residual pre-training, likelihood fine-tuning, inference) with the inputs and trained parameters of each, add a notation table, and organise Section 5.1 around the roles listed in our answer to W1 & Q1.
Thank you again for the constructive feedback. We hope that these clarifications and the added experiments address your concerns.
References
[1] Huben et al. "Sparse Autoencoders Find Highly Interpretable Features in Language Models." ICLR, 2024.
[2] Liu et al. "PerFit: Exploring Personalization Shifts in Representation Space of LLMs." ICLR, 2026.
[3] Assran et al. "Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture." CVPR, 2023.
[4] Huang et al. "LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures." ICLR, 2026.
[5] Cao et al. "Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization." NeurIPS, 2024.
[6] Wu et al. "ReFT: Representation Finetuning for Language Models." NeurIPS, 2024.
[7] Tan et al. "Analyzing the Generalization and Reliability of Steering Vectors." NeurIPS, 2024.
[8] Salemi et al. "ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generation." Findings of ACL, 2025.
[9] Zhang et al. "Personalized Text Generation with Contrastive Activation Steering." ACL, 2025.