- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
Companion chatbots are usually tested against a simulated user who keeps talking no matter how the chatbot behaves. This paper builds a simulated user who can stop. An LLM judge reads each chatbot reply and decides whether it gives that user what they need before opening up (warmth, trust, or anonymity). Two bad replies in a session end the session. A low satisfaction score after a session means the user never comes back. Six LLMs, each under four prompting setups, talk to these users for up to ten sessions. The paper measures how often each model asks for personal information, how much of the user’s personal history it draws out, and how long the user stays.
I can summarize into 3 main findings. First, across the six models, the ones that ask more questions keep users for less time. Second, when the same models run for a fixed 200 turns instead, they draw out about twice as many personal memories and their ranking changes: GPT-5.6 Luna moves from 5th to 2nd, and Gemini 3.1 Pro loses ~1% of its score. Third, training a 4B model with a penalty for every reply the judge dislikes raises its satisfaction score from 0.57 to 0.92 and keeps 94.4% of users to the last session. A small human study compares two models and prefers the one the new setup ranks higher.
Critical strengths
- It's an interesting result overall. The paper asks whether model rankings change when the simulated user is allowed to leave and shows under their setup that they do. With users who cannot leave, the six models score between 14.2 and 18.4 memories drawn out. With users who can leave, they score between 5.0 and 9.3, and the order changes. Provides a new way of evaluation particulalrly for companion chat bots.
- The setup is easy for others to adopy. Letting the chatbot’s behavior decide how long the conversation lasts is a small change to any existing simulator.
Critical weaknesses
**1. A single LLM judge sits in the middle of everything, so three of the four findings may describe the judge rather than the chatbots. **
The judge labels each reply good or bad. A bad label drives pretty much everything: the user stops sharing personal information, the user moves one step closer to leaving the session, the satisfaction score goes down, and the chatbot is penalised during training. Basically every number in the paper comes from these labels. The judge labels each reply good or bad, and a bad label drives pretty much everything: the user stops sharing, moves closer to leaving, the satisfaction score drops, and during training the chatbot is penalised. Nothing in the paper checks the judge against anything other than itself, so a model that pleases the judge wins every metric whether or not it pleases people.
First, the RL result is the clearest case. The model is trained to avoid the judge’s bad labels and then scored by the same judge. It beats the Oracle, which was handed the judge’s rubric directly, which is what you would expect from a model that learned to satisfy the judge rather than the user. Wang et al. (ACL 2024) and Williams et al. (ICLR 2025) found exactly this when training against an LLM evaluator.
Second, the finding that asking more questions drives users away follows from the judge’s rules rather than from user behavior. If the judge calls a question a bad reply, the user shares less and is closer to leaving in the same step, so the two outcomes move together because one label caused both. The paper’s own Oracle shows this: it asks the most questions of any model and keeps users the longest. The correlations are also just six points each, one per model, with no uncertainty.
All of this is fixable, but it needs the judge to be checked against something other than itself.
A smaller presentation point: the paper is math-heavy where it does not need to be. Section 2.3 uses seven numbered equations, including a definition of the minimum over an empty set, to say that the user leaves after two bad replies and does not return when satisfaction falls below a threshold.
2. The main results have no uncertainty
From what I can tell the paper never says how many simulated users it runs, but every percentage in the RL table is a multiple of 1/18, so the main table is an average over 18 users from what looks like one run, with no error bars? The headline claim that GPT-5.6 Luna moves from 5th to 2nd rests on 9.2 memories versus DeepSeek’s 9.3. A 0.1 gap over 18 users could flip with a different seed, and the one plot that does show uncertainty has spreads far wider than that.
Questions for authors
- Which LLM runs the judge, the simulated user, and the satisfaction scorer? If it is the same family as one of the evaluated models, that model’s retention numbers need a second judge.
- How many simulated users and how many runs are behind each cell of the main table? Please add intervals.
- Has the RL model been scored by a judge it was not trained against, or by people?
- Which measure does the method-effect table use, and how do you reconcile it with the main table for Gemini 3.1 Flash-Lite? PerLTQA is a Chinese dataset. How were the English personas produced?
4 comments
To Reviewer Wrt5 (2/2)
[R5] Response to “result uncertainty” (W2 & Q2):
Our RL experiments used 18 personas (line 993), while other non-training experiments used 36 diverse user personas; we will clarify this in the manuscript.
The original table is one-run results. To better quantify result uncertainty, we re-ran the main experiment 5 times and report 95% CI. Below are the Vanilla results (mean [95% CI], t-interval with df = 4, ret=0.5):
| Companion (Vanilla) | Erate | Dmem | Rsession | Rturn | Rsatisfaction |
|---|---|---|---|---|---|
| Qwen 3.8-27B | 0.80 [0.79, 0.80] | 0.18 [0.17, 0.19] | 5.73 [5.30, 6.16] | 82.7 [76.8, 88.6] | 0.64 [0.63, 0.65] |
| DeepSeek V4 Pro | 0.23 [0.22, 0.23] | 0.25 [0.25, 0.25] | 7.81 [7.75, 7.87] | 129.6 [129.0, 130.3] | 0.78 [0.78, 0.79] |
| Gemini 3.1 Pro | 0.83 [0.83, 0.83] | 0.15 [0.15, 0.15] | 4.28 [4.28, 4.28] | 68.6 [68.6, 68.6] | 0.57 [0.57, 0.58] |
| Gemini 3.1 Flash-Lite | 0.77 [0.75, 0.78] | 0.22 [0.21, 0.23] | 5.24 [5.00, 5.49] | 82.9 [78.6, 87.1] | 0.67 [0.66, 0.67] |
| GPT-5.6 Sol | 0.27 [0.25, 0.28] | 0.18 [0.17, 0.18] | 5.83 [5.52, 6.14] | 82.8 [78.3, 87.3] | 0.69 [0.68, 0.69] |
| GPT-5.6 Luna | 0.28 [0.25, 0.30] | 0.25 [0.25, 0.25] | 7.12 [7.09, 7.15] | 117.0 [117.0, 117.1] | 0.74 [0.73, 0.75] |
The 95% CIs are narrow and the trends in Section 4.1 are preserved. Full results for all methods will be included in the revised Table 1.
[R6] Response to “user-side model selection” and “need a second judge” (Q1):
The user-side models are based on gemini-3.1-flash-lite (Table 9 in Appendix). Regarding “need a second judge”, we believe the new results in [R1] with 4 different judges can address your concern.
[R7] Response to “the scoring of RL models” (Q3):
The RL model is trained against DeepSeek-V4-Flash (Appendix D) and evaluated by a different judge, the one in the original Retentopia (gemini-3.1-flash-lite).
[R8] Response to “method effect table” (Q4):
We’d like to kindly inform you that the metric used in Table 3 is the count of memories revealed by the user (|M_revealed|).
Regarding gemini-3.1-flash-lite, the two tables report different measures of information disclosure. Table 1 uses D_mem, a normalized, hierarchy-weighted disclosure score, to show users’ disclosure tendency in Retentopia. On the other hand, Table 3 and Figure 4 are based on the unweighted, exact number of revealed memories (|M_revealed|) to facilitate clearer model comparison across evaluation settings. Higher D_mem doesn’t necessarily indicate a large |M_revealed|.
[R9] Response to “PerLTQA English version” (Q4):
Thank you for asking this question. In Appendix C1, we mention using GPT-4o-mini to process the dataset, but we didn’t mention that this processing also includes the Chinese-to-English translation. We will include this in the revised manuscript. Also, the translation was further verified by one of the authors, who is native in Chinese.
Dear Reviewer, we hope you are doing well.
As the discussion period nears its end, we'd appreciate it if you could consider our responses and share any remaining concerns.
Thank you again for your time and consideration.
Thank you for the detailed follow up. Please include these changes (multiple judges and seeds) in the revised PDF.
I am satisfied with the 2 weaknesses I mentioned are addressed and will update my score to a 3 to reflect this.
To Reviewer Wrt5 (1/2)
Thank you for your time and effort; we are glad you found our work interesting. We will improve our work based on your comments.
[R1] Response to “robustness and generalization of the LLM judge” (W1):
To address cross-judge robustness and generalization, our RL experiments (Table 2) adopted different LLM judges for training (DeepSeek-V4-Flash; Appendix D). We will move these important implementation details from Appendix to the main text.
To further address your concern, we re-conducted all computational experiments using 4 new judges. We present the same experiments in Figure 4(b) regarding our central claim (retention constraint changes evaluation results) and will include other results in the revised manuscript.
Findings from Figure 4(b) remain consistent across all judge LLMs (cell-level Spearman ρ = 0.80–0.96). Retention drops by about half from the guaranteed setting, and gemini-3.1-pro falls from above gpt-5.6-luna to the lowest rank. Although absolute values vary with judge strictness and the top position alternates between two nearly tied models, the other experiments remain consistent with the original judge, and the judges agree well on return decisions (Fleiss' κ = 0.69). We will include this analysis in the revised manuscript.
[R2] Response to “judge-human alignment” (W1):
We’d like to mention that, in Table 6 & 7 in Appendix B, we’ve conducted human verification of our judge-based within- and post-session retention (73% and 87% agreement rates, respectively). While these results do not imply perfect alignment with human (discussed in Limitations), they provide independent evidence that our retention judgments are not evaluated solely against themselves.
[R3] Response to “too math-heavy” (W1):
As a formal study to distinguish retention-constrained evaluation of personalized companions from existing evaluation, we intentionally provide a method-agnostic, stricter formulation. As discussed in line 147, these formulations outline the evaluation and leave the specific implementation as a design choice. The simple “threshold” design is how we instantiate these concepts.
However, to facilitate easy understanding, we will revise Section 2.3 by transforming complex equations into in-line textual explanations, whilst moving the math equations to Appendix.
[R4] Clarification to “more questions driving users away is the judge’s rule” (W1):
We would like to clarify that Retentopia does not impose a universal rule that asking more questions leads to user dissatisfaction or dropout. As described in Section 3.2.1 and Table 17 (Appendix C.1), our framework incorporates three distinct disclosure preconditions of users (i.e., warmth-, trust-, and anonymity-first), each with different preferences towards AI responses. Our judge evaluates whether a companion's behavior satisfies the user's specific preferences, rather than penalizing elicitation frequency.
The oracle example you mentioned (“The paper’s own Oracle shows …”) supports the fact that we did not apply an “asking more questions drives users away” rule, as it asked the most (E_rate = 0.94) yet kept the user long. Also, human evaluation in Figure 5 shows that “ungrounded personalization” is the main cause of dropout, rather than probing.