acceptodds
Under review as a conference paper at ICLR 2027

MultiUS : Unsupervised Persona Mixture and Norm-Bounded Latent Steering for User Simulation

Abstract

Obtaining a faithful User Simulator is a precondition for effective evaluation and training of multi-turn conversational systems. Prompt-based user simulators, however, reproduce human utterances and behavior poorly, while fine-tuning approaches require substantial computational resources. Recently, steering vectors have emerged as a promising and cost-effective mechanism for inducing target behaviors in Large Language Models, and have been successfully applied to persona and behavior control. However, computing steering vectors typically relies on curated contrastive samples and therefore requires prior knowledge of the relevant behavioral dimensions and their intensities. In this work, we propose to infer the distribution of user behaviors directly from real conversations, using unsupervised Gaussian-Mixture models in a latent behavioral space. Samples from these distributions are mapped to steering vectors through lightweight adapter layers applied to a frozen Language Model. The method allows to learn a single User Simulator, trained once, and subsequently generate distinct user behaviors by sampling from the learned distributions, without requiring explicit persona labels or retraining. To preserve the base model's activation geometry, we constrain the steering operation so that, at every token, the steered activation norm remains within (1±) of the original norm. We study two adapter families: a per-channel affine map in the spirit of AdaLN-Zero, and a parameter-efficient variant operating entirely in a low-dimensional subspace. On WildChat, a single simulator adding 7.9–76.3M trainable parameters (0.2–1.8% of the frozen Qwen3.5-4B) writes far more human-like messages than a fully fine-tuned 8B user simulator and than prompted open-weight and frontier models (measured through relative MAUVE scores), is closest to human register, and stops conversations when humans do. On PRISM, a population it was never trained on, it is about as close to real users as real WildChat users are. We name our approach MultiUS.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Unsupervised Learning, User Simulation, Generative AI, Deep Learning, Activation Steering, Steering Vectors

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_h2bCOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces an approach to learn activation steering adapters for frozen language models to better simulate a diverse set of users. The paper describes this approach, which is based on Gaussian Mixture models. The paper then compares their approach to baseline user simulators introduced in past work on two datasets, focusing on simulator realism.

Critical strengths

The paper addresses an important topic in user simulation: how to represent a diverse set of users, compared to fine-tuned simulators on large datasets of interaction logs, which end up learning a general behavior of the average user. Little work has focused on this angle in the past, and introducing methods to achieve this is an important contribution.

Critical weaknesses
  • Issues with evaluation metrics: I am not sure what the motivation of the introduced Register (log ppl) metric is, I find it a bit problematic that this is being computed with an external model like Mistral. Could you instead just report the PPL metric directly by the simulator models and baselines themselves ? This is much more interpretable and in-line with past work. I also have the same concern about the MAUVE metric and feel it is not the right one to use here. The behavioral statistics measures are interesting, if you could combine them into a single metric of alignment with these metrics, it would be much more interpretable. I highly suggest revisiting this and checking closely related work for more reliable ways to assess simulator realism (which is what the experiments are trying to measure, as far as I understand).

  • There is a mismatch between the model used in the introduced approach (Qwen3.5) and the baseline UserLM-8b, which is based on a much older Llama3 model. Since the llama model is open-source, it would be much more sensible to implement the approach on the same model for a more direct and fair comparison with the baseline. Otherwise, it is unclear whether improvements are due to Qwen3.5 being a more recent and capable model. It would also be helpful to perform fine-tuning to Qwen3.5 in a similar way to how the baseline UserLM-8b model was trained. In general, I felt the experimental results lack some rigor and could be further strengthened with more depth. The approach could also be better clarified with a clearer description and more telling figure than the current Figure 1, which is not very reader-friendly.

Questions for authors

Please see questions above in the weaknesses section regarding evaluation metrics.

2 comments

We thank you for your constructive feedback and apologize for the late reply. We thank you in particular for your evaluation metrics suggestions. We actually learned something new about our approaches behavior.

First, we would like to clarify our thinking for our choice of the Δlog⁡PPL\Delta \log \mathrm{PPL} metric scored by Mistral-7B. It measures the average difference in log perplexity, as computed by an external model, between real user utterances and utterances generated by the simulator. We introduced this metric because it directly compares samples generated by each simulator, allowing us to avoid bias introduced by teacher-forcing and to directly measure the quality of outputs. The choice of Mistral, therefore, was made to have a “neutral” model. If we have chosen Qwen, it would naturally assign more probability to its own utterances as opposed to human ones, penalizing MultiUS. And similarly, choosing Llama3-8B would penalize UserLM-8B.

Self-perplexity. As you suggested, we measured how likely each simulator finds real users' messages: the teacher-forced negative log-likelihood of the user turns of real conversations (per byte of user text so that different tokenizers compare) on 1,000 WildChat evaluation conversations and the 8,001 PRISM conversations. MultiUS's is a variational model, so we decided to marginalize its latent variable over 64 prior samples. Values are differences from the frozen Qwen3.5-4B backbone (so that we compare it against a weak simulator, as a floor), next to first-message MAUVE:

systemself-ppl WildChatself-ppl PRISMMAUVE WildChatMAUVE PRISM
MultiUS (τ = 0.6)+0.025 [+0.020, +0.030]+0.017 [+0.013, +0.019]0.6260.083
UserLM-8b−0.157−0.0660.0660.012
UserLM-8b + intent−0.122−0.0020.7360.088
LoRA SFT, e0−0.097−0.1050.2910.056
Qwen3.5-4B prompted+0.006−0.0010.0080.013

We can see that UserLM-8b assigns real user text clearly higher likelihood than MultiUS. We interpret that it covers the distribution of user text better than a bounded perturbation of a frozen 4B model.

How we think it compares to MAUVE and register Self-perplexity measures how well a model covers real users' text when given it, while MAUVE and register measure the text a simulator samples, which is how a simulator is actually used. In our measurements, the metrics rank systems in nearly opposite order. Without an intent, UserLM-8b has the best likelihood on WildChat and a first-message MAUVE of 0.066, most of its first turns running into repetition loops, while LoRA e0 has the best likelihood on PRISM and MAUVE 0.29. Running with an intent, UserLM-8b's two numbers in opposite directions: MAUVE rises elevenfold and its likelihood advantage shrinks. In MultiUS’ side, the steering strength τ shows the same trade-off: at τ = 0.3 it assigns real text higher likelihood than its backbone (−0.025 [−0.031, −0.020] at l₀ = 15, −0.064 [−0.070, −0.058] at l₀ = 7) at the “price” of having a register further from human. Additionally, across all eight grid configurations, self-perplexity follows the order of the held-out CE margin exactly, and the reverse of the register criterion with one exception.

The UserLM-8b setup that matches MultiUS on MAUVE is the one with an intent. Its likelihood lead is large on WildChat (−0.122 vs +0.025) (which is its training corpus), and small on PRISM, which neither model was trained on (−0.002 vs +0.017). The two also tie on first-message MAUVE (0.088 vs 0.083) there. It gets there with a model twice the size, fully fine-tuned, an intent written by a frontier model for every conversation, and sampling guardrails.

Where MultiUS loses likelihood : We admit we were surprised by the overall metrics (MultiUS performing worse than Qwen3.5-4B, while the differences between both models were very obvious when chatting with them), but after looking at length, it revealed our approach predicts short messages well (−0.135 nats/byte under 40 bytes on WildChat, −0.090 on PRISM) and long, well-formed ones very badly. This result is not surprising, as activation steering likely degrades the model’s capabilities beyond the training objective, and we have mostly removed long messages from our training dataset (so predicting well-formed prose, code, etc. gets degraded). Additionally, long messages dominate a per-byte average. MultiUS is better than its backbone in 53% of conversations on both corpora. We have created a “Self-Perplexity of Real User Text” annex with the full tables and analysis.

In this comment, we would like to share the results of your suggested experiment of training MultiUS on the Llama-3-8B backbone (same one as UserLM-8B). We used the same 4,000 conversations, depth and number of epochs, and no re-tuning (121.8 M trainable parameters at this width, imposed by the backbone’s hidden size / embeddings size). We report the checkpoint chosen by our selection rule and, because that one fails our held-out CE constraint, the best checkpoint that passes it.

first turn, WildChatall turns, WildChatfirst turn, PRISMall turns, PRISM
MultiUS, Qwen3.5-4B0.626 [0.556, 0.707]0.518 [0.425, 0.590]0.083 [0.050, 0.113]0.075 [0.057, 0.096]
MultiUS, Llama-3-8B, τ = 0.3 (passes)0.605 [0.491, 0.681]0.479 [0.418, 0.562]0.087 [0.051, 0.125]0.077 [0.053, 0.101]
MultiUS, Llama-3-8B, τ = 0.6 (selection rule)0.624 [0.527, 0.705]0.589 [0.522, 0.668]0.108 [0.071, 0.159]0.094 [0.068, 0.126]
UserLM-8b, no intent0.0660.0440.0120.011
UserLM-8b + intent + guardrails0.764 [0.704, 0.822]0.548 [0.459, 0.617]0.087 [0.056, 0.124]0.066 [0.054, 0.082]

Every Qwen–Llama difference is within its CI. On UserLM-8b's own backbone, MultiUS shows the same picture as on Qwen: far ahead of UserLM-8b without an intent, behind it with an intent and guardrails on WildChat first turns (−0.159 [−0.293, −0.061] for τ = 0.3), and within the CIs over whole conversations and on PRISM (τ = 0.6 is ahead on PRISM over whole conversations, +0.028 [+0.005, +0.060]). The results are not due to Qwen3.5 being a newer model.

Reviewer_cwbWOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes MultiUS, an unsupervised user simulation framework that aims to capture diverse user behaviors without requiring predefined persona labels or expensive full-model fine-tuning. The authors introduce a Gaussian Mixture Variational Autoencoder (GM-VAE) to learn a latent distribution of user profiles from real conversations. Latent representations sampled from this distribution are subsequently mapped to steering vectors through lightweight adapters installed in a frozen language model. To maintain the stability of the underlying model, the authors propose a norm-bounded activation steering mechanism that constrains the magnitude of the modified activations. Both full-dimensional and low-rank variants are investigated to balance modeling flexibility and parameter efficiency. Experiments on WildChat and PRISM demonstrate that MultiUS substantially outperforms the evaluated prompting-based simulators and UserLM-8B in terms of text-distribution similarity, measured by MAUVE. The proposed approach also achieves more human-like linguistic characteristics and conversation lengths while introducing relatively few trainable parameters.

Critical strengths

1. The paper addresses a meaningful problem in user simulation. Learning diverse user behaviors without relying on predefined persona labels is an interesting research direction, particularly for developing scalable user simulators.

2. The authors attempt to integrate Gaussian mixture modeling with activation steering to construct a parameter-efficient user simulator. Although the individual components are largely based on existing techniques, their combination provides a potentially useful approach to modeling user utterance distributions.

3. The paper reports substantial improvements in MAUVE compared with the evaluated baselines. The authors also provide additional analyses of steering strength, adapter configurations, and conversation-level statistics. While I have concerns about the fairness of the comparisons and the interpretation of the results, these experiments offer some empirical observations worth further investigation.

Critical weaknesses

1. The methodological novelty and primary contribution of the paper are unclear. The proposed framework mainly integrates existing techniques, including Gaussian mixture VAEs, activation steering, and parameter-efficient adapters. Although the authors claim that the framework enables unsupervised persona discovery, it is unclear what constitutes the main methodological innovation beyond combining these established techniques. More importantly, the paper does not provide convincing evidence that the learned latent representations correspond to meaningful user personas. The examples in Appendix L appear to primarily capture differences in topics and task categories rather than stable user behavioral characteristics. I therefore find the central claim of unsupervised persona discovery insufficiently supported.

2. The technical presentation is unclear and lacks a complete explanation of the proposed framework. The overall organization of the methodology makes it difficult to understand how the different components interact. Although Figure 1 illustrates the installation of steering adapters within Transformer layers, the paper lacks a clear end-to-end architecture illustrating the entire training and inference pipeline. In particular, the paper does not clearly explain how the learned implicit user representations can be practically used to simulate a specific user or how behavioral consistency is maintained across multiple turns.

3. The experimental results are difficult to interpret, and the fairness of the comparisons is questionable. The reported improvements over existing user simulators are remarkably large. However, the experimental settings raise concerns about whether these improvements can be attributed to the proposed method. In particular, UserLM-8B is evaluated without its original intent conditioning, and the resulting generations exhibit severe degeneration, including extremely long and repetitive outputs. Such behavior can significantly affect distribution-based metrics such as MAUVE.

4. The paper's organization and writing do not effectively communicate its contributions. The paper introduces several methodological elements without clearly distinguishing established techniques from the proposed innovations. The presentation of the framework, the claimed contributions, and the corresponding experimental evidence are not well aligned. The paper would benefit from a substantial restructuring that clearly identifies the primary technical contribution and directly connects it to the experimental validation.

Questions for authors
  1. What is the primary methodological novelty of MultiUS? Given that Gaussian mixture VAEs, activation steering, and parameter-efficient adapters have already been extensively studied, what is the specific new technical contribution beyond their integration?

  2. How can the learned latent representations be used to simulate a particular user? At inference time, the paper samples latent vectors from the learned prior. How would the method infer an appropriate latent representation from a target user's historical interactions, and how would the resulting persona remain consistent throughout a multi-turn conversation?

  3. How can the authors justify the large performance improvements given the apparent degeneration of the UserLM-8B baseline? Could the authors provide more controlled comparisons using identical backbones, training data, and reasonable generation configurations?

1 comment

Thank you for the careful review and apologize for the late reply. We address your questions:

Q1 / W1 / W4. Clarification on novelty We agree that the ingredients exist separately (the VaDE / GM-VAE prior, the AdaLN-Zero-style affine map, low-rank adapters) and that the paper did not state the contribution clearly. It lies in what their combination makes possible:

  1. Steering learned is without behavior demonstrations Steering vectors are usually computed from contrastive examples of a behavior chosen in advance. In MultiUS the steering signal is the decoder of a latent-variable model trained only to reconstruct real user turns. A benefit of that is that the behaviors need not be named, described or demonstrated (and a limitation is that they are harder to interpret).
  2. A latent-conditioned, norm-bounded adapter. At every token the steered activation stays within (1 ± τ) of its original norm (proof in Appendix B). We show that the bound τ controls a trade-off between human register against overfitting held-out users (Section 4), and removing the bound at matched parameter count fails our held-out constraint and doubles verbatim copying of training prompts (reply to P1Ag).
  3. A cheap simulator. One frozen backbone and 0.2–1.8% trainable parameters, and since the backbone is frozen, the activations below the first steered layer are computed once and cached, and one trained model samples many users.

W1. On the Persona claim. We agree with you that the components are not personas in a strong sense. We cite this in our Limitations: the latent does not separate who the user is from what they want, and components differ by task as much as by style. Measured on first messages: messages drawn from the same component share style (intraclass correlation 0.164 [0.078, 0.224]) and topic (TF-IDF cosine, same minus different component, +0.023 [+0.003, +0.051]), against ≈ 0 for shuffled labels. Four messages from one sampled latent share style about half as much as four conversations of one real user (0.193 against 0.371) and topic about a quarter as much (0.038 against 0.140). A sampled latent is closer to a stable user type, but not an individual. We also learned from our K = 1 (reply to P1Ag) experiment that the mixture does not make the simulator more realistic, but its role is to segment users into behaviors that can be sampled separately. We have therefore decided to weaken our “persona” modeling claim : we think it is more accurate to state that we model “sub-populations”.

Q2. Consistency over turns, and simulating a particular user.

  • Consistency. The latent is drawn once per conversation and used throughout the entirety of a conversation with the simulator. However, we cannot claim any formal guarantees of persona consistency. As an empirical first assessment we reran the same model with the same seeds, identical first turns, and the latent redrawn before every later turn: the share of style variance between conversations falls from 0.596 to 0.501 (paired difference +0.094 [+0.061, +0.131], positive on all six style features). With the latent held, MultiUS users are slightly more consistent than real users (0.48), and less rigid than UserLM-8b (0.70–0.75).
  • Is it possible to reproduce a particular user ? Our encoder q(z | conversation) maps a user's conversation to a latent, and conditioning generation on its posterior mean instead of a prior draw is the mechanism for simulating that user. This is the training mechanism, but we have not evaluated it as an inference mechanism for simulating the exact user behind a given conversation. The numbers above suggest why it would be limited: the latent is trained to explain one conversation, not to identify a person across conversations.

Q3 / W3. UserLM-8b's degeneration, and controlled comparisons. We ran all three:

  • UserLM-8b as in its paper, with a per-conversation intent and with and without its decoding guardrails. It no longer degenerates, and it beats MultiUS on WildChat first turns (0.764 against 0.626). Over whole conversations and on PRISM the two are within each other's CIs (Table C). We have revised our claims accordingly.
  • Same backbone: We ask you kindly to refer to the response to reviewer h2bc.
  • Same backbone and data: Against the LoRA SFT of Qwen3.5-4B on the same 4,000 conversations (54.8 M parameters), MultiUS is ahead on every WildChat column (first turns +0.143 [+0.015, +0.253] against the best epoch).
Reviewer_P1AgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MultiUS learns a population of user utterances through a variational model with a learned Gaussian-mixture prior. A sampled latent controls adapters inserted into a frozen Qwen3.5-4B, with a token-level relative perturbation bound. The paper studies full-dimensional affine and low-rank adapters, trained without explicit persona labels on 4,000 filtered WildChat conversations. Evaluation compares generated conversations with held-out WildChat and PRISM using MAUVE, language-model register scores, termination and repetition statistics. The main reported gains are substantial distributional improvements over prompted assistants and intent-free UserLM-8B.

Critical strengths
  1. A useful approach to population simulation with a frozen backbone. The combination of an unlabeled mixture prior and latent-conditioned activation adapters offers a practical way to sample diverse interactions without specifying a persona schema. The low-rank decoder adapters use 7.9M parameters, and the local perturbation bound is straightforward to verify. This is a plausible contribution at the intersection of latent text generation and representation adaptation, even though its individual ingredients have established precedents.

  2. Large and carefully qualified distributional gains. Table 2 reports first-message WildChat MAUVE of 0.676 for full-dimensional MultiUS versus 0.066 for UserLM-8B, and Table 3 reports mean conversation length of 3.90 versus 3.66 for the human reference. First-message evaluation, human-versus-human controls, a cross-corpus reference and the appendix length controls help interpret these gains. The paper also reports repetition and memorization failures. Together these results provide meaningful evidence that the trained simulator better matches this filtered population than the particular baselines tested.

Critical weaknesses
  1. The training procedure is not yet consistent with the claimed validation and objective. In the supplied multius/train.py, Trainer combines train and held-out conversations, warm_up constructs features for both, init_prior fits the mixture and initializes the category head from the combined features, and _population samples from this same combined pool. Its outputs enter gradient-bearing pretraining and training regularizers. Therefore the released procedure uses held-out examples for learning, beyond checkpoint selection. This weakens the independence of the held-out CE constraint used to select admissible configurations. It does not establish leakage from the separate evaluation pool or PRISM, and it is not evidence of misconduct; the issue is the validity and description of the validation protocol.

  2. The comparisons do not isolate the proposed mechanisms or establish behavioral personas. The main comparison mixes a model trained on the target filtered population with prompted models and a separately trained UserLM checkpoint evaluated without intent. The authors justify intent-free evaluation and investigate UserLM’s length failures, so this is not an inherently invalid setting. However, it cannot determine whether the benefit comes from the proposed mixture and bound or simply from adapting the backbone to these data. A Qwen3.5-4B LoRA/SFT baseline trained on the identical 4,000 conversations is particularly important. K = 1, a fixed-latent trained adapter, and a matched alternative to the norm bound would isolate the main mechanisms. A sampled-profile USP comparator [2] would also address the most relevant existing multi-profile approach.

Questions for authors
  1. Clarification affecting validity. Did the reported checkpoints use the released Trainer, including held-out features in prior fitting, category-head initialization and population regularization? If so, can you provide a split restricted to training for all parameter learning and recompute checkpoint selection, the CE constraint and the main results? If not, please identify the exact code version used.

  2. Clarification affecting the method. Which objective generated the reported checkpoints: the expected per-example categorical KL in Equation 10, or KL of mean assignments in losses.py? Were the entropy terms annealed with beta, and how were the token reconstruction and latent KL terms normalized? Please reconcile the implementation with the equations.

  3. Evidence affecting novelty and significance. Can a matched Qwen3.5-4B LoRA/SFT model, K = 1 model and fixed-latent trained adapter recover the distributional gains? Does a norm-preserving or otherwise bounded intervention perform similarly at matched capacity? A compact comparison would materially clarify what the mixture and proposed steering mechanism contribute.

3 comments

Thank you very much for the detailed review, and in particular for finding the validation leak! We are sorry for our late reply, rerunning experiments took us a lot longer than expected. We will post a general comment with a more complete answer. We will start by answering your clarification questions directly.

Q1 : Yes, as explained in our first general comment, the reported checkpoints were indeed trained by the code that included the validation leak, and your description of the trainer is correct. We have shown how checkpoint validation results changed by using a fresh user-disjoint validation set, and as you asked we have retrained all checkpoints reported.

Q2 We are sorry for the confusion, indeed the reported checkpoints used the objective as implemented in the supplementary material, not Equation 10 as written. The paper’s description is inaccurate and we will correct it in the revision. The differences are:

  • Categorical term : We regularize the KL between the aggregate posterior assignment over a fixed-size population draw and the prior weights, and not the expected per-example KL. This is an implementation trick that we failed to remember when writing the paper and we apologize (the paper was written after concluding most of the experiments).

  • Normalization : The reconstruction loss is the per-token mean cross-entropy. The z-term (the continuous term in VaDE) is the π~\tilde{\pi}-weighted KL to mixture components, in nats per latent dimension. The free bits (0.15 nats/dim) are on this term only.

  • Annealing : βKL\beta_{\mathrm{KL}} is annealed with cyclical annealing, and is constant within an epoch. So it goes 0 -> 0.05 -> 0.10 -> 0.15, then 0.2 for the last six epochs of the 10 epoch cycle. We have made it more explicit in the paper.

  • Pretraining: Section 3 described it correctly (z sampled from the posterior, no KL term is used), but Appendix A contradicted it. The appendix now matches the code and Section 3: z is indeed sampled (with the reparametrization trick) with no KL term and no prior, encoder and adapters are trained end to end through the frozen decoder, and the reconstruction is the same per-token mean. The mixture is then fitted to the posterior means.

The objective is therefore VaDE-style, but we realize it is not an ELBO, so we won’t describe it as one anymore. This trick is borrowed from IMSAT : since the per-example categorical KL decomposes into the aggregate posterior’s KL plus the mutual information between examples and cluster assignments, replacing it by the aggregate posterior actually rewards the mutual information instead of rewarding it.

Now we would like to share the results of some of the experiments that you have requested.

LoRA SFT on the identical 4,000 conversations (W2, Q3) is reported in our general comment (Table C): Qwen3.5-4B, 54.8 M parameters, same data and system prompt, three validation-selected epochs. MultiUS is ahead on every WildChat column, e.g. first turns +0.143 [+0.015, +0.253] against the best epoch (e18) and +0.184 [+0.082, +0.282] against e4, the last epoch that passes our CE constraint. On PRISM self-play the two are within each other's CIs.

Ablations Each ablation changes one thing in the leak-free full-dimensional configuration τ = 0.6, l₀ = 15 (54.5 M parameters): same 4,000 conversations, 30 epochs and seeds, the frozen selection rule on the fresh validation set, and the test protocol of Table C (1,500 self-play conversations, plus 3,000 first messages from prior samples). For K = 1: a single Gaussian prior. For Fixed z: the same latent (z = 0) for every conversation (because we use a bias in the MLP that produces the steering parameters, it’s equivalent as if we had a fixed steering adapter), in training and generation. For the Norm-preserving variant: the output norm is fixed to the input's and the angle is free, with no τ; same parameters. MAUVE against WildChat users, 95% CIs by cluster subsampling; differences are paired.

We have implemented the norm-preserving variant by making minimal changes to the original implementation:

     g       = gate_scale(LN(W_x x))                  # no tanh, no tau: magnitude unbounded
    delta   = raw * g * rms(x) / rms(raw)
    y       = ||x|| * (x + delta) / ||x + delta||    # output norm exactly ||x||; angle unbounded

Table E.

K = 8 (base)K = 1fixed znorm-preserving
CE margin (validation)+0.087 [−0.047, +0.227] tie+0.087 [−0.057, +0.240] tie+0.187 [+0.074, +0.305] pass−0.404 [−0.559, −0.241] fail
MAUVE, self-play first turns0.6010.7570.4510.648
MAUVE, 3,000 first messages0.6020.6570.3880.577
MAUVE, all self-play turns0.5160.5970.3930.579
training copies2.2%0.1%3.3%4.7%

On PRISM all four are within each other's CIs.

Sampling a latent user per conversation is essential. With one fixed z, MAUVE falls on every WildChat measurement: −0.150 [−0.276, −0.027] on self-play first turns, −0.214 [−0.301, −0.117] on first messages, −0.123 [−0.230, −0.030] over all turns. This holds although the fixed-z model has the best validation register criterion and passes the CE constraint.

The mixture does not make MultiUS more realistic With K = 1, MAUVE is higher on self-play first turns (+0.155 [+0.038, +0.261]) and within the CIs on first messages (+0.055 [−0.056, +0.160]), over all turns (+0.080 [−0.009, +0.194]) and on PRISM. Additionally, the K = 1 model also copies training prompts 20 times less. The mixture serves the purpose in our introduction (it’s more of a project requirement than an implementation trick to increase faithfulness): a single trained model that segments users into K behaviors, each of which can be sampled on its own. The components are distinct: first messages drawn from the same component share style (intraclass correlation 0.164 [0.078, 0.224] for the reported model, 0 with shuffled labels) and topic. Post hoc, most of the gap traces to one component, which captured the templated prompts of a single heavy training user: without its conversations, K = 8 scores 0.738 at matched n, level with K = 1 (0.753), where dropping any other component does not raise it. Paired CIs include 0, so this is suggestive only.

Norm-preserving steering at matched parameter count. Removing the bound improves the validation register criterion (1.266 against 1.453) but fails the CE constraint by a wide margin (paired difference with the bounded model −0.491 [−0.563, −0.419]) and doubles verbatim copying (4.7% against 2.2%). On the test set its MAUVE is within the CIs of the bounded model on every reference (all turns: +0.063 [−0.053, +0.140]). Its register is closer to human (first messages, Qwen scorer: −1.24 against −1.59). The match is in parameters, not in perturbation size: without the bound the realized ‖δ‖/‖x‖ averages 0.74 (maximum 1.19), where the bounded model cannot exceed 0.6. This is the trade we see when raising τ to 1.0 (Table 3): the bound keeps the model within the held-out constraint and limits copying but it does not buy better samples.

The selection criterion and MAUVE disagree on these ablations Fixed z has the best criterion and the lowest MAUVE; K = 1 the reverse. The criterion measures register only and cannot see diversity or coverage. This can be fixed by selecting on a distributional signal on validation.

USP. We did not run a sampled-profile USP comparator during the discussion period. This baseline is nevertheless important, and should be implemented in future work.

I thank authors for they responses and added experiments. Overall, unfortunately, I do not yet see enough reason to raise my score.

This is mainly due to:

  • the stated selection rule now appears to favor τ=1.0,l0=15\tau=1.0, l_0=15, while τ=0.6,l0=7\tau=0.6, l_0=7 is retained for consistency. This choice needs to be reconciled with the pre-specified procedure.
  • the requested K=1K=1, fixed-latent, and alternative-bound ablations remain pending, so the contributions of the mixture prior and norm-bounded steering are still unclear.
  • the leak-free results weaken the original register and stopping-behavior claims, which should be revised accordingly.

Comments on the paper

4 comments

We thank all reviewers for their careful reading, and for all of the constructive feedback that we have received. We are working hard to learn from them and integrate your suggestions into our work as fast as possible, and will keep you updated as we progress.

TL;DR: The leak is confirmed and fixed. Re-running selection on 2,000 fresh, user-disjoint conversations leaves the ranking and the selected configuration unchanged. We are retraining every configuration so that all numbers in the revision come from leak-free models, and new experiments (LoRA-SFT, ablations, backbone) are under way.

In this first comment, we address the leak discovered by reviewer P1Ag (W1/Q1). In the trainer that produced the reported checkpoints, the 2000 validation conversations (used for checkpoint selection) were excluded from the next token prediction loss, but not from four computations outside trainer batches : the fitting of the GM priors, the category-head initialization, the population draws used by the VICReg regularization (during pretraining) and the cluster terms (KL loss during training). The trainer never saw any samples from the evaluation set, however, this leak invalidates checkpoint selection and the CE constraint.

We have fixed and tested the new trainer, and now every checkpoint records the rows it was fitted on. A test replaces the validation features with noise and requires every fitted quantity to be bitwise identical (as a sanity check, it fails on the old code).

In order to measure the damage done, we have used the same procedure to build 2000 new conversations (from 1202 users) from fresh WildChat samples, user-disjoint from every existing split (Table A, next comment): The ranking of the configurations is unchanged CE-constraint verdicts are unchanged in 7 of 8 configurations. The one change is τ = 0.6, l₀ = 15, from tie to pass. The selected configuration, full-dimensional τ = 0.6, l₀ = 7, is unchanged. Its margin contains 0 again (−0.029 [−0.171, +0.113], against +0.008 [−0.125, +0.152] on the original set), which does not disqualify it under the checkpoint selection rule presented in the paper. Its selected epoch moves from 22 to 27.

In addition, we have started retraining the presented checkpoints with the fixed trainer. We already have the results for the selected configuration and two others: the low-rank τ = 0.3, l₀ = 7 model reported in the paper, and full-dimensional τ = 0.6, l₀ = 15 (Table B). Each is compared with its original run on the fresh set by a test fixed in advance: selection criterion within our resolution threshold of 0.12 nats, and a CE-margin difference whose CI includes 0.

  • Selected configuration (τ = 0.6, l₀ = 7): both conditions hold. Criterion +0.10 nats and CE margin −0.05 [−0.13, +0.02]. Its margin contains 0 (−0.08 [−0.23, +0.07]), so it stays selected.
  • The other two fail the second condition. Criterion −0.02 and +0.09 nats, but CE margins lower by 0.15 and 0.09 nats, CIs excluding 0.
  • In all three pairs the retrained run selects a later epoch (e29 vs e27, e25 vs e18, e19 vs e14), and the margin decreases with training in all six runs. At matched epochs, retrained and original models differ by at most 0.05 nats. Within-run epoch selection is less stable than the constraint's resolution. This suggests the failed comparisons reflect epoch-selection noise rather than an effect of the leak.
  • Because two of three comparisons fail our own test, we are retraining the remaining configurations, so that every number in the revised paper comes from a leak-free model.

We are regenerating the test metrics (MAUVE,behavior, etc.) for the leak-free selected models and will post the results in the coming days, with a revised version of the paper and supplementary material.

In parallel, we are running additional experiments in response to the reviewers' suggestions, prioritizing :

  • a Qwen3.5-4B LoRA-SFT baseline on the identical 4,000 conversations (P1Ag, cwbW Q3, h2bC)
  • The ablations (K = 1, fixed-latent, and norm-bound variant) requested by P1Ag
  • UserLM-8B generations with intent, to better match the setup presented in their original paper (h2bC, cwbW Q3)
  • MultiUS on UserLM-8b's Llama-3 backbone, to investigate the effect of backbone choice (h2bC, cwbW Q3)
  • logppl by the simulators themselves (h2bc W1)

We will report each as it completes.

Table A. Selection replayed on the fresh validation set. Same checkpoints, same scripts and same GPU type: only the validation data change. Criterion: mean |Δlog ppl| over epochs 12–19 (lower is better, Appendix H). Epoch: the selected epoch within the run. CE margin: unsteered minus steered held-out CE, 60 conversations. Unsteered CE is 3.161 (original) and 3.401 (fresh), as the conversations differ. Verdict (Section 4): fail if the CI lies below 0, tie if it contains 0.

adaptersτl₀criterion orig → freshepoch orig → freshCE margin, originalCE margin, freshverdict orig → fresh
full-dim.0.3152.280 → 2.167e22 → e20+0.356 [+0.236, +0.485]+0.465 [+0.337, +0.596]pass → pass
full-dim.0.6151.701 → 1.478e25 → e18+0.030 [−0.089, +0.160]+0.233 [+0.094, +0.376]tie → pass
full-dim.1.0151.580 → 1.309e26 → e23−0.257 [−0.390, −0.117]−0.154 [−0.289, −0.015]fail → fail
full-dim.0.3†71.933 → 1.832e18 → e13+0.377 [+0.264, +0.497]+0.497 [+0.375, +0.620]pass → pass
full-dim.0.671.475 → 1.295e22 → e27+0.008 [−0.125, +0.152]−0.029 [−0.171, +0.113]tie → tie
full-dim.1.071.349 → 1.265e20 → e22−0.169 [−0.306, −0.021]−0.147 [−0.282, −0.009]fail → fail
low-rank0.3†71.844 → 1.654e16 → e14+0.283 [+0.167, +0.406]+0.327 [+0.206, +0.453]pass → pass
low-rank0.671.481 → 1.305e25 → e26−0.222 [−0.355, −0.075]−0.172 [−0.324, −0.014]fail → fail

† 20 epochs instead of 30. Bold: the selected configuration, unchanged (lowest criterion among full-dimensional configurations that do not fail. τ = 1.0, l₀ = 7 has a lower criterion. τ = 1.0 fails at both depths on both sets).

Table B. Leak-free retrains against the original runs, both scored on the fresh validation set. CE margin as in Table A, with leak-free minus original paired over the same 60 conversations.

configurationcompared atcriterion or CE margin, originalleak-freedifference
full-dim. τ = 0.6, l₀ = 15criterion, e12–191.4781.453−0.025
each at its selected epoch (e18 / e25)+0.233 [+0.097, +0.382]+0.087 [−0.047, +0.227]−0.146 [−0.199, −0.093]
matched, e18+0.233 [+0.097, +0.382]+0.212 [+0.084, +0.351]−0.021 [−0.071, +0.032]
matched, e25+0.061 [−0.078, +0.209]+0.087 [−0.047, +0.227]+0.027 [−0.040, +0.091]
full-dim. τ = 0.6, l₀ = 7 (selected)criterion, e12–191.2951.393+0.098
each at its selected epoch (e27 / e29)−0.029 [−0.167, +0.117]−0.083 [−0.230, +0.072]−0.054 [−0.130, +0.020]
matched, e27−0.029 [−0.167, +0.117]−0.064 [−0.209, +0.091]−0.034 [−0.100, +0.031]
matched, e29−0.087 [−0.216, +0.054]−0.083 [−0.230, +0.072]+0.003 [−0.064, +0.071]
low-rank τ = 0.3, l₀ = 7†criterion, e12–191.6541.748+0.094
each at its selected epoch (e14 / e19)+0.327 [+0.211, +0.453]+0.240 [+0.099, +0.392]−0.087 [−0.147, −0.025]
matched, e14+0.327 [+0.211, +0.453]+0.375 [+0.249, +0.512]+0.048 [+0.006, +0.090]
matched, e19+0.213 [+0.082, +0.350]+0.240 [+0.099, +0.392]+0.027 [−0.043, +0.104]

The "selected epoch" rows are the comparison we specified in advance. The matched-epoch rows were added after seeing it. All three retrained runs fit the prior on exactly their 4,000 training conversations. The CIs of Table B come from a separate bootstrap draw and differ from Table A in the third decimal.

We apologize for our delayed response, running the experiments took a lot longer than we had expected. As promised in our first general comment, here are the test-set results generated from the models retrained without the leak, and some of the baselines asked by the reviewers. We are grateful to your suggestions, as they helped us improve our methodology. Our “headline” numbers are lower than initially submitted, and given the result we change one of the paper’s claims.

Here we present the results for :

  • A LoRA SFT baseline : In order to match (approximatively) the parameter count of our reported model, we train it with rank 27 (54.8 M parameters) on the same 4,000 conversations and system prompt. We have chosen to report 3 checkpoints : epoch 0 (lowest validation CE), epoch 4 (the last epoch to not fail the CE constraint) and epoch 18 (same selection rule as MultiUS : best register in the validation set)
  • UserLM-8B with an intent : in our first submission, we had decided to run UserLM-8B without an intent to match the same “scenario” as MultiUS, which does not see any information about the user, and because this was a reported configuration in their paper. We now report UserLM ran with an intent extracted by GPT-5.6-luna (GPT-4o is not available anymore) on another real conversation, using the author’s intent generation prompt. We run it with and without its decoding guardrails (bad on certain first words, redraw of verbatim repeats up to 5 times).

Table C. MAUVE against real users (higher is better). SP = self-play.

systemSP first turn, WildChatSP all turns, WildChatSP first turn, PRISMSP all turns, PRISMfirst messages, WildChatfirst messages, PRISM
other real users (ceiling)0.9640.8990.9560.9490.9530.948
MultiUS, as submitted0.676 [0.609, 0.741]0.569 [0.452, 0.669]0.105 [0.073, 0.147]0.095 [0.078, 0.124]0.595 [0.519, 0.646]0.073 [0.054, 0.097]
MultiUS, leak-free0.626 [0.556, 0.707]0.518 [0.425, 0.590]0.083 [0.050, 0.113]0.075 [0.057, 0.096]0.610 [0.562, 0.671]0.074 [0.056, 0.085]
UserLM-8b + intent + guardrails0.764 [0.704, 0.822]0.548 [0.459, 0.617]0.087 [0.056, 0.124]0.066 [0.054, 0.082]n/an/a
UserLM-8b + intent0.736 [0.636, 0.822]0.502 [0.398, 0.593]0.088 [0.059, 0.123]0.067 [0.056, 0.078]n/an/a
UserLM-8b, no intent (as in the paper)0.0650.0470.0120.012n/an/a
LoRA SFT, e00.2910.2500.0560.0590.2770.049
LoRA SFT, e40.4420.3820.0660.0570.3850.056
LoRA SFT, e180.482 [0.414, 0.572]0.357 [0.300, 0.419]0.076 [0.040, 0.114]0.064 [0.045, 0.085]0.342 [0.302, 0.392]0.053 [0.040, 0.069]
prompted Qwen3.5-4B0.0080.0080.0130.013n/an/a

The all-turn comments use the smallest system’s n (4449), so the submitted model’s values differ from the paper’s very slightly. Real users from WildChat score 0.112 for the first turn and 0.099 for all turns against PRISM : so this bounds the PRISM columns. The intent rows allow 1,024 tokens per user turn, but without an intent that limit does not change anything (0.066). CIs subsample 80% of conversations without replacement

Here are our interpretations :

  • Leak-free vs leaky : Self-play / multi-turn MAUVE is lower by about 0,05 on WildChat (0.626 against 0.676 on first turns, 0.518 against 0.569 on all turns) and by 0.02 on PRISM. None of these differences are significant (paired CIs include 0). And on first messages it does not change (0.610 against 0.595; 0.074 against 0.073). We updated the paper to show only the leak-free numbers
  • Against prompting, UserLM-8B without intent and LoRA SFT : the leak-free model is ahead in every column, and only the CIs o f the two PRISM self-play columns include 0
  • Against UserLM-8B with an intent: UserLM-8B is clearly better on MAUVE for WildChat first turns (0.764 against 0.626; paired difference 0.139 [0.045, 0.226]), while the CIs overlap on PRISM columns.

WildChat is UserLM-8B’s training corpus (but that’s no excuse). We therefore revise our paper so it no longer claims that MultiUS performs UserLM-8B. We have therefore a weaker claim : without any intent or guardrail, a frozen 4B model with 1.8% trained parameters is far ahead of prompting and of UserLM-8b without an intent, and ahead of LoRA SFT on the same data. Given a per-conversation intent written by a frontier model, UserLM-8b matches or beats it. We would like to stress also that thanks to activation caching (we can cache activations until the earliest layer we steer), MultiUS is significantly faster to train (on a L40S GPU, ≃ 8.2 hours when earliest layer steered is 15, while the LoRA on Qwen3.5-4B took ≃23.7 hours) . Register and behavior follow in the next comment.

Table D. Register and behavior in self-play. Register: median log-perplexity of first turns minus that of real WildChat users, under two scorers (0 = human, negative = more fluent than humans). As in the paper, Mistral-7B, which generated none of the texts, is the primary scorer. Copies: share of user messages of 20+ characters that are exactly a user message of our 4,000 training conversations.

systemregister, Qwen3.5-4Bregister, Mistral-7Buser turnsP(8 turns)median length (chars)self-repeatcopies
real users+0.16+0.123.660.103640.0260.08%
MultiUS, as submitted−1.38−0.773.900.131700.0791.8%
MultiUS, leak-free−1.62−0.873.120.052790.0763.1%
UserLM-8b + intent + guardrails−1.27−0.873.860.2781260.104†0.05%
UserLM-8b + intent−1.26−0.954.240.3531200.3890.05%
UserLM-8b, no intent−4.39−3.333.170.2799020.5890.00%
LoRA SFT, e0−2.45−1.413.400.1071130.2450.00%
LoRA SFT, e4−1.52−0.882.970.057970.10912.0%
LoRA SFT, e18−0.58−0.393.600.154880.15837.6%
prompted Qwen3.5-4B−1.84−1.717.380.7191930.0010.03%

The register row for real users is a second, disjoint sample of real users and its behavior columns are the 1,500 conversations that seed the self-play.

As you can see :

  1. Every simulator is still more fluent than real users The leak-free model, UserLM-8b with an intent and LoRA e4 are close to each other under both scorers, and the leak-free model is 0.1 to 0.25 further from human than the submitted one. The two scorers agree on the ordering of the large gaps.
  2. LoRA e18 is closest to human register because it recites 37.6% of its self-play messages, and 81% of its first messages, are exact copies of training turns. At epoch 0 (where it does not copy) its register is worse than prompting. This is the problem of fine-tuning on only 4,000 conversations. The three epochs show it: epoch 0 repeats itself, epoch 4 ends conversations early, epoch 18 recites the training data.
  3. MultiUS also copies more than real users: 3.1% of self-play messages (7.2% of first messages) against 0.08% (0.13%), although an order of magnitude below LoRA’ epochs 4 and 18. UserLM-8b copies almost nothing from our training split. The revised paper reports this rate next to the human one.
  4. Conversation shape MultiUS messages have a length close to real users' (79 characters against 64, while with UserLM-8b with an intent it’s 126), and 0.7% of its conversations end after one user turn against 29 to 31% for UserLM-8b with an intent. But MultiUS ends conversations earlier (3.12 user turns against 3.66 and 5% reach 8 turns against 10%), whereas UserLM-8b ends conversations later (28% reach 8 turns). Self-repetition is lower for MultiUS than for unguarded UserLM-8b (0.076 against 0.389).

Another thing on the leak-free retrains (Table B of our earlier comment). All eight configurations are now retrained and scored on the fresh validation set. The paper now contains the figures computed on the leak-free models. Under the test we fixed in advance, six of the eight pairs differ from their original run (the two that agree include the reported configuration), which is why every number above comes from a retrained model. One consequence concerns selection. The retrained τ = 1.0, l₀ = 15 model no longer fails the CE constraint: its margin is −0.155 [−0.302, +0.004], so a tie by 0.004, and its selection criterion is only 0.021 nats lower than of τ = 0.6, l₀ = 7 (1.372 against 1.393). Additionally, every paired MAUVE difference includes 0 (all turns +0.047 [−0.056, +0.144]), and it is closer to humans on register (−1.26 against −1.62, Qwen scorer) and conversation length (3.57 against 3.12 user turns; humans 3.66). We keep the previously selected model for consistency.

Given the end of the discussion period is approaching, we will post individual replies while we have time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.