- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a new synthetic data generation method to improve language model's personalization capability. The key idea involves using an oracle. It generates a new synthetic dataset M2D-CORPUS, that better simulates multi-turn conversations requiring better understanding of users' minds. It then trains M2D-Chat with Qwen-7B and shows that it is outperforming both the base model together with a wide range of other approaches on personalization bench
Critical strengths
The synthetic data generation framework seems to be new, although it has to be better presented.
The experiments themselves appear sound and seem to cover quite some analysis.
Critical weaknesses
Experiments are only conducted on small models on the scale of 7B. It’s unclear how well this generalizes to larger models, especially considering when GPT-5-mini seems to be greatly outperforming the results. Minor: the use of bolded numbers in Table 1 is fine, but perhaps you’d want to group M2D-Chat with the non-proprietary methods, since otherwise it’s confusing why it is bolded while GP-5-mini is clearly better
Questions for authors
Some examples for in “Evolving user-state simulation” would help readers understand. I am not sure what exactly is trust history, for instance. It might also make sense to define earlier in Sec 3 since 3.1 seems to also require that.
I don’t really understand Sec 3.3. Some example here could help. What does privileged mean here?
Sec 3.4 is also very hard to understand. Section 3 seems to read well before 3.3. From 3.3 to 3.4, it packs a lot of information. Another big question I have from reading Section 3 is what exactly is the and the other here. Is this just calling an LLM with the available information to synthesize examples?
I can’t understand the role of Section 4.2.
9 comments
Response to Reviewer 4imh, Part 2
(3) The gap to GPT-5-mini. All M2D-Corpus data is generated by GPT-4o-mini, which serves as both the user simulator and the Oracle teacher model.
- The trained 7B student exceeds GPT-4o-mini, its own data generator, on three of the four Table 1 metrics: PersonaMem-v1 56.4 vs. 48.6, PersonaMem-v2 42.0 vs. 37.3, and PrefEval-Gen 56.8 vs. 21.1.
- LEAP(Choudhury & Sodhi, ICLR 2025) reports the same outcome: 8B students trained by a teacher with privileged information surpass GPT-4o acting without it.
Our response to Reviewer TaX2 measures the contribution of the privileged access of teacher model to student states directly: with data, recipe, and student model fixed, training on privileged-Oracle targets improves PrefEval-Gen over training on state-blind-Oracle targets by +4.85 points on 900 PrefEval question.
GPT-5-mini is a newer and far more capable proprietary model of undisclosed size and training data compared to all 3 7-8B instruction model we selected. It plays no role in our data generation, and we list it as a reference point. The 14B experiment shows that the absolute score rises with the base model (PrefEval-Gen 62.8 at 14B vs. 52.3 at 7B; MultiChallenge 20.2 vs. 18.0), so the remaining gap between GPT-5-mini and opensource model trained on our M2D-Chat mainly reflects the base model's general capability. Our main claim concerns the gain that scalable simulated supervision gives a fixed open backbone, and this gain holds across model family and size, indicating method’s effectiveness and scalability
Response to Reviewer 4imh, Part 3
Weakness 2. Table 1 grouping and boldface
"Minor: the use of bolded numbers in Table 1 is fine, but perhaps you'd want to group M2D-Chat with the non-proprietary methods, since otherwise it's confusing why it is bolded while GP-5-mini is clearly better"
Thank you for pointing this out. In the revision, Table 1 groups the rows into four blocks:
- Proprietary reference models (GPT-4o-mini, GPT-5-mini).
- The base model, Qwen2.5-7B-Instruct.
- Task-specific methods built on Qwen2.5-7B-Instruct.
- M2D-Chat (ours), built on Qwen2.5-7B-Instruct.
The caption will state that bold marks the best result among models built on Qwen2.5-7B-Instruct (blocks 2 to 4), and that proprietary models are references outside this comparison.
Question 1. Examples of the user state and trust history
"Some examples for [c_t] in "Evolving user-state simulation" would help readers understand. I am not sure what exactly is trust history, for instance. It might also make sense to define [s_t] earlier in Sec 3 since 3.1 seems to also require that."
(1) Definition. The user state is a structured record updated at every turn:
| Component | Content |
|---|---|
| turn index, unresolved goals, trust history, and a summary of the interaction | |
| slowly changing information: values, background constraints, the user's position toward the assistant | |
| short-lived information: mood, current concerns, judgments made during the turn |
(2) Trust history. The trust history records how much the user trusts the assistant. After each assistant reply, the state update revises a trust value and records a one-sentence reason for the change. Trust changes in 455 of 11,314 state updates (4.0%).
(3) One turn of a training instance.
[persona] ---------------------> seen by: simulator, Oracle, student
A public relations professional focused on crisis communication, who prefers
clear step-by-step guidance without being micromanaged.
[private context] -------------> seen by: simulator, Oracle
Limited budget for professional development; a family health issue that takes
emotional bandwidth; online communication courses felt impractical.
Deal-breakers: a full-day workshop, a personal coach, a speaking club.
[user state s_{t-1}] ----------> seen by: simulator, Oracle
trust 5.5/10; patience 4/4; emotion: mild curiosity;
tension between practical application and limited time and resources.
[state update -> s_t] ---------> seen by: simulator, Oracle
trust 6.5/10, "because the examples provided were helpful and applicable to my work context";
patience 3/4; emotion: mild satisfaction and curiosity about consistency.
[user message m_t] ------------> seen by: simulator, Oracle, student
I appreciate the practical tips you've shared so far. Are there any resources or
specific tools that you recommend for reinforcing active listening and clear
communication in my daily work? I'd like to explore some options that fit into
my current routine without requiring too much extra time.
[Oracle reply, with s_t] ------> training target (loss on these tokens)
A few low-friction tools can make these habits easier without adding much to your day ...
That's enough to spot patterns without turning practice into another project.
If you want a resource to read, The Crisis Communications Handbook ... you could dip
into relevant sections.
[Oracle reply, without s_t] ---> not used for training; shown for comparison
A few low-effort options can fit into routines you already have ... For structured
practice, Crucial Conversations offers tools for difficult discussions ... You could
start with one checklist and one reflection habit for two weeks.
The student is trained on the tokens of the Oracle reply written with . It never sees the private context or the user state, in training or at deployment.
Response to Reviewer 4imh, Part 1
Dear Reviewer 4imh,
Thank you for the careful review and the concrete suggestions on presentation. We are glad that you found the synthetic data generation framework new and the experiments sound. Below we quote each concern and question and answer it directly.
Weakness 1. Larger models and the gap to GPT-5-mini
We answer in three parts: (1) a new 14B scale controlled experiment, (2) the published setting our 7B experiments follow, and (3) the gap to GPT-5-mini.
(1) New experiment at 14B scale model.
We trained Qwen2.5-7B-Instruct and Qwen2.5-14B-Instruct on the same M2D-Corpus subsample (4,422 training instances, 1 epoch, same LoRA recipe), and evaluated each model against its own base with one evaluation harness. Because this controlled experiment uses a smaller training set than Table 1 (8,244 examples, 2 epochs), its 7B model scores differ from Table 1 in the paper.
LLM-as-Judged. PrefEval-Gen, ALOE, and MultiChallenge use each benchmark's official judge prompt and rubric, with GPT-6-sol as the judge model
Findings.
(2) Our 7B setting follows well-established published work. Training at the 7B to 8B scale and reporting a stronger proprietary model as a reference is the standard setting for this line of work: