acceptodds
Under review as a conference paper at ICLR 2027

Learning from What Users Do Not Say: Self-Evolving Assistants by Internalizing Latent Thoughts

Abstract

Large language models (LLMs) are increasingly used as interactive assistants, yet their ability to understand users remains limited by partial observability: user utterances often reveal only incomplete information about the underlying intents, preferences, and concerns. In this paper, we study whether such latent user information can serve as privileged supervision during training and be internalized into the assistant for real-world interactions. We propose OASIS, a three-stage framework that learns from latent user thoughts without relying on a stronger teacher model. OASIS first uses user simulators to expose latent user states for supervised cold-start training, and then performs on-policy asymmetric self-distillation, where the same assistant learns from the information gap between a privileged view with latent thoughts and an observable view with dialogue only. Experiments on open-ended general assistance and goal-oriented commercial customer service show that OASIS consistently improves assistant performance across model scales and evaluation settings, with on-policy self-distillation providing up to 52.4% and 22% further improvements over the SFT initialization on WildBench and DianJin-CSC, respectively. These results demonstrate that latent user information can be effectively internalized into the assistant through asymmetric self-distillation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
self-evolve, multi-turn dialogue

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_Xf1UOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes OASIS, a framework that uses latent user thoughts as training-time privileged information to improve assistants that observe only dialogue at deployment. User simulators generate multi-turn interactions with explicit latent thoughts, which inform assistant-generated responses for supervised fine-tuning. A subsequent on-policy stage trains the assistant using token-level log-probability differences between a frozen reference model’s privileged and observable views, without requiring a stronger response-generating teacher. The central contribution is using dynamically generated user states as supervision for assistant self-improvement. Experiments with Qwen3-4B and Qwen3-8B on WildBench and DianJin-CSC report improvements over the base models and SFT initialization, across two WildBench judges and both reference-context and generated-context customer-service evaluations. The paper also includes ablations and human assessments of simulated thought quality.

Critical strengths
  1. A well-motivated use of training-time user information. OASIS uses simulator-generated latent user thoughts to supervise an assistant that operates solely on observable dialogue at deployment. This addresses a meaningful challenge: benefiting from richer user information without requiring users to provide it during actual interactions.

  2. Consistent reported improvements across complementary settings. The main results show gains across both general assistance and commercial customer service, two model sizes, and two independent WildBench judges. The full method also consistently improves upon its SFT initialization. This breadth provides encouraging evidence of the approach’s empirical utility, even though the precise contribution of latent-thought supervision requires stronger controlled comparisons.

Critical weaknesses
  1. OASIS uses assistant-generated training trajectories filtered for satisfactory final outcomes, whereas the Standard-SFT baseline uses ordinary WildChat dialogues. This simultaneously changes the data source, response style, trajectory quality, and filtering procedure. Consequently, the substantial advantage of OASIS-SFT may partly stem from higher-quality synthetic data and successful-trajectory filtering; the current comparison cannot isolate the contribution of latent thoughts. The authors should consider a controlled comparison that keeps the initial tasks, user simulator, generation model, filtering criteria, and training budget fixed, varying only whether latent thoughts are provided when generating the training responses.

  2. A major concern is whether the synthetic user data are sufficiently reliable and faithfully reflect real-world user intentions. Since the paper emphasizes understanding implicit needs, the evaluation metrics should assess whether the model correctly identifies the constraints that genuinely matter to the user and whether its final responses better satisfy that user’s actual needs. The paper should also address how to handle unsupported inferences about user preferences and hallucinated assumptions.

Questions for authors
  1. [1] also concerns latent user information and train model for better interactive assistants with synthetic users, can author identify their main difference from this work?

[1] Bo Peng, Zhiheng Wang, Heyang Gong, and Chaochao Lu. 2025. IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 17007–17040, Suzhou, China. Association for Computational Linguistics.

2 comments
AuthorsOpenReview ↗

(1/2) Official Comment by Authors

Thank you for acknowledging the value of well-motivated use of training-time user information and consistent performance improvements. We have carefully prepared responses to address your concerns.

W1: Unfair comparison between OASIS-SFT and Standard-SFT in ablation study.

Thank you for raising this important concern. We agree that a controlled comparison would provide a fairer evaluation and better demonstrate the importance of latent thoughts.

  • We conducted an experiment with the key difference from OASIS-SFT being whether latent thoughts are accessible. Specifically, SFT w/o Privileged uses the same initial tasks as OASIS-SFT, with Qwen3-8B and Qwen3-4B serving as assistant models that engage in multi-turn interactions with the user simulator without access to its latent thoughts. The final SFT dataset contains about 9,467 dialogues and 28,401 turns, matching the scale of OASIS-SFT.

  • The experimental results below show that OASIS-SFT achieves WB Scores of 24.71 and 15.55 on Qwen3-8B and Qwen3-4B, outperforming SFT w/o Privileged (−17.50 and −20.41) by 42 and 36 points. These results show the benefits of users' latent thoughts as privileged supervision.

ModelWB Score
Qwen3-8B-19.51
    + SFT w/o Privileged-17.50 (↑2.0)
    + SFT (Ours)24.71 (↑44.2)
    + OPD w/o Privileged24.98 (↑44.5)
    + OPD (Ours)37.66 (↑57.2)
Qwen3-4B-23.48
    + SFT w/o Privileged-20.41(↑3.1)
    + SFT (Ours)15.55 (↑39.0)
    + OPD w/o Privileged15.71 (↑39.2)
    + OPD (Ours)18.28 (↑41.8)

W2: User simulator reliability and assistant ability.

Thank you for raising this important concern. We agree that the faithfulness of generated user thoughts, as well as the model's ability to accurately understand and satisfy real users' needs, is essential for validating the effectiveness of OASIS. We highlight our human evaluation of Faithfulness of Simulated Thoughts and add it with evaluations of Real-User Need Understanding. Specifically,

  • Faithfulness of Simulated Thoughts. OASIS's general-domain user simulator is trained on ThoughtTrace, which contains latent thoughts reported by real users rather than relying on synthetic user preferences. As reported in Table 2, we evaluate its faithfulness on 500 held-out real-user turns with three independent human annotators. The trained simulator improves Human-Thought Alignment from 3.41 to 3.96 (out of 5), and numbers of ≥4 increase from 69.2% to 86.8%. These results support the reliability of simulated thoughts as proxies for real user states, while acknowledging that they cannot perfectly reproduce real user intentions.

  • Real-Users Need Understanding. To directly evaluate whether OASIS better understands and satisfies real users' needs, we conduct an evaluation on 200 held-out real-user conversations from ThoughtTrace, using users' self-reported thoughts as evaluation references while providing the assistant with only observable dialogue. Three independent human annotators, each holding a master's degree in engineering, evaluate the assistant responses along four dimensions: (1) Constraint Alignment, whether responses correctly identify and address constraints that genuinely matter to users; (2) Need Satisfaction, whether responses effectively satisfy users' actual needs; (3) Assumption Grounding, whether responses avoid unsupported assumptions about user preferences; and (4) Clarification Appropriateness, whether assistants appropriately seek clarification when important constraints remain uncertain. These results suggest that OASIS better identifies and addresses real users' needs while reducing unsupported preference assumptions. Detailed evaluation criteria are provided in Appendix A.6 (L810-L838).

ModelConstraint AlignmentNeed SatisfactionAssumption GroundingClarificationOverall
Qwen3-8B57.258.875.952.661.1
+ OASIS-SFT70.172.083.768.473.6
+ OASIS76.878.187.377.379.9
AuthorsOpenReview ↗

(2/2) Official Comment by Authors

Q1: Differences Between OASIS and IP-Dialog

Thank you for raising this question about IP-Dialog. Both works explore how implicit user information can improve dialogue assistants, but they differ primarily in research objectives, the source and representation of implicit information, and how such information is utilized during training. Specifically:

  • Research objectives and implicit information. IP-Dialog focuses on evaluating and improving implicit personalization using synthetic data constructed from pre-defined user attributes. In contrast, OASIS aims to improve interactive assistants by leveraging turn-level latent thoughts generated by a trained user simulator, capture users' evolving intentions and constraints during interactions, rather than relying on pre-defined user attributes.

  • Training method. IP-Dialog uses SFT to train the assistant to infer relevant attributes and generate analyses and responses. OASIS uses latent thoughts as training-time privileged information to guide response generation during SFT. Subsequently, OASIS provides dense supervision at interaction states visited by the student through OPSD, using two views of the same model with and without latent thoughts. This enables the assistant to internalize the benefits of latent thoughts while relying only on observable dialogue at deployment.

We have added a citation to IP-Dialog in Section 5 (Related Work, L474–L476) and clarified the research objectives and key differences.


Thank you again for the thoughtful questions and positive evaluation of our work, which serve as great encouragement for us. We hope these clarifications resolve your concerns, and we sincerely appreciate your willingness to further evaluate our work.

Reviewer_8eN1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

OASIS proposes a three-stage framework to improve LLM assistants by leveraging latent user thoughts (unexpressed intents, preferences, and constraints) as privileged supervision during training, then internalizing this knowledge via on-policy asymmetric self-distillation so no latent information is needed at deployment. Evaluated on WildBench (general assistance) and DianJin-CSC (commercial customer service) using Qwen3-4B and Qwen3-8B, the method reports up to 52.4% improvement over SFT on WildBench and 22% on DianJin-CSC, with ablations isolating the contribution of latent thoughts from additional training signal.

Critical strengths

Compelling problem formulation. The partial observability framing is well-motivated and immediately accessible, the observation that user utterances systematically under-represent latent states is grounded in ThoughtTrace evidence and the skincare example in Figure 1 makes the intuition clear to a broad audience.

Clean, reproducible framework design. The three-stage separation (simulator, cold-start SFT, on-policy self-distillation) is principled, each stage has a formal objective and a corresponding ablation isolating its contribution, and the no-external-teacher constraint is a meaningful practical contribution for resource-constrained settings.

Critical weaknesses

Core

Novelty. latent-state simulation, supervised training, distillation for deployment pipeline is architecturally similar to recent pubblications like Mind2Dialogue and PRIDE papers, making the framework more like a recombination of established privileged-distillation paradigms rather than a novel contribution. The authors could strengthen the positioning by discussing concurrent efforts clarifying how OASIS differs from them. Acknowledging this emerging research direction would help readers situate the contribution.

No user personas. OASIS models latent thoughts as turn-level states but has no mechanism to distinguish who is thinking: two users saying the same thing (e.g., "help me pick a gift") may need opposite recommendations (cheap vs. premium), yet the framework lacks persona conditioning, stratified evaluation, or any explicit diversity in the user simulator. This risks the distilled assistant collapsing toward the majority preference profile in the training data, systematically underserving minority user types without any way to detect it in the reported aggregate metrics.

No human evaluation of the assistant. Human evaluation is limited to a single 500-turn faithfulness check of the simulator with 3 annotators, no inter-annotator agreement, no significance tests, no confidence intervals, and no scoring rubric definition, and even this check is not fully interpretable, as the ≥4 (%) column in Table 2 never states what scale is used, whether the threshold applies jointly or per dimension, or what a score of 4 semantically represents. The actual assistant quality (the paper's core contribution) is judged entirely by LLMs (GPT-OSS-120B and Qwen3-235B), meaning internalized latent thoughts improving assistance is never validated by humans.

Only Qwen models tested. The entire pipeline including simulator (Qwen3-8B, Qwen2.5-7B), assistants (Qwen3-4B, Qwen3-8B), and one of two judges (Qwen3-235B) operates exclusively within the Qwen family, meaning gains may reflect family-specific pretraining or alignment artifacts rather than a general training principle. Without any cross-family validation the results cannot distinguish whether OASIS works because of its method or because teacher and student share architectural inductive biases.

Benchmark mismatch. WildBench includes math and coding/debugging categories where correctness is objective and independent of hidden user preferences, yet OASIS is applied uniformly with no per-category breakdown or justification. Gains on these tasks likely reflect extra training signal rather than latent thought internalization, a confound never disentangled in the paper.

No statistical significance analysis. All results are single-run point estimates with a fixed seed (42), with no significance tests, confidence intervals, standard deviations, or p-values anywhere in the paper. Several ablation differences are small enough to plausibly fall within random variation, yet are presented as meaningful conclusions: without multi-seed runs or paired tests, the reported gains cannot be reliably distinguished from noise.

Sim-to-real transfer gap dismissed as future work, but it's the core assumption. The authors acknowledge that simulator behavior may differ from real users but frame it as a future exploration, yet the entire OASIS pipeline depends on the simulator faithfully representing real latent thoughts, making this the foundational validity assumption, not an optional extension.


Minor

Missing citation. ThoughtTrace first appears at line 46 without a reference, potentially confusing readers about whether it is a novel contribution or prior work. The citation appears only later in the paper.

Same model teaches itself. Using the identical assistant backbone as both teacher (privileged view) and student (deployment view) creates an untested representational ceiling, unlike classical distillation where a stronger teacher provides richer supervision.

OPD acronym never defined. OPD (presumably On-Policy Distillation, the ablation variant without privileged access) appears in Table 1 and ablation tables without ever being expanded, forcing readers to reverse-engineer its meaning from context. This is a basic presentation oversight that hinders reproducibility.

WB Score undefined. WB Score in Table 1 is never explained: its scale, computation, and reference baseline are left entirely to the reader to infer from external sources, with values ranging from -33 to +58 giving no intuition without consulting the original WildBench paper. A self-contained paper should define its primary evaluation metric.

Questions for authors

Have you tested OASIS on scenarios where two users with identical utterances have opposing preferences (e.g., budget vs. premium)? Can you provide per-persona-type evaluation breakdowns?

What is the specific architectural or methodological contribution that differentiates OASIS from Mind2Dialogue's privileged distillation pipeline?

Can you provide even preliminary results on one non-Qwen backbone (e.g., LLaMA-3-8B) to rule out family-specific artifacts?

Can you provide per-category WildBench breakdowns showing that gains on math/coding are not simply from additional training signal?

Can you report standard deviations or confidence intervals for the key results?

8 comments
AuthorsOpenReview ↗

(1/5) Official Comment by Authors

Thank you for acknowledging the value of our framework design and motivation formulation. We have carefully prepared responses to address your concerns.

W1&Q2: Differences Between OASIS and Mind2Dialogue.

Thank you for highlighting Mind2Dialogue, a concurrent work first posted on arXiv on September 14, 2026, four days before the submission deadline. We also thank you for bringing PRIDE to our attention. We agree that both methods leverage privileged information during training. OASIS's key contribution is to transform evolving latent user thoughts into on-policy self-distillation signals, enabling the assistant to learn from its own interactions rather than imitating a stronger teacher, a distinction from prior works. Specifically:

  • Mind2Dialogue employs a GPT-4o-mini Oracle to generate state-informed assistant responses, which are used for supervised fine-tuning. Since the training dialogues are generated by the Oracle, the student may produce different responses and encounter different subsequent user behaviors. OASIS addresses this distribution mismatch through OPSD. After SFT initialization, the assistant generates its own responses based on observable dialogue history, and these responses influence the simulated user's subsequent behavior. This allows training to cover interactions encountered by the assistant's current policy. For each generated token, we compute log probabilities using the same frozen SFT model, once with privileged user thoughts and once without them. Their difference provides token-level signals for reinforcing or penalizing the assistant's own decisions.

  • PRIDE leverages privileged annotations to distill empathetic reasoning from a larger teacher model into a smaller student. In contrast, OASIS uses evolving simulated user thoughts to guide the assistant's own response optimization through OPSD, without requiring a more capable teacher model.

Thank you for your valuable suggestion. We have added these two concurrent works to the Related Work section (L489–L492) and clarified the key differences among the three methods.

W4&Q3: Only Qwen models tested. Non-Qwen backbone (e.g., LLaMA-3-8B) is needed.

Thank you for your valuable suggestion. We conducted experiments with Llama3-8B-Instruct in both general and commercial scenarios. On WildBench, OASIS improves the WB Score over the original model by 39.0 and 20.35 points under the GPT-OSS-120B and Qwen3-235B judges, respectively. And in commercial setting, OASIS improves BR by 15.40 and 33.51 points and ACC by 15 and 15.43 pp over the original model under reference and generated contexts. We have added these results to Appendix A.4 (L762–L790).

Table 1. WildBench Results

ModelCreativePlanningMath / DataInfo / AdviceCoding / DebugRaw ScoreWB Score
Llama3-8B-Instruct-38.50 / 1.60-43.41 / -2.33-46.59 / -13.49-40.64 / 5.99-54.06 / -18.962.82 / 4.85-43.57 / -3.01
     + OASIS-SFT-20.72 / 28.63-24.54 / 14.62-36.19 / -1.27-15.50 / 21.49-39.62 / -3.213.76 / 5.77-24.88 / 15.41
     + OASIS8.06 / 26.05-5.62 / 18.68-22.54 / 10.953.61 / 20.20-20.19 / 1.604.77 / 5.87-4.57 / 17.34

Table 2. Commercial Scenario Results

Evaluation with reference context

ModelB-2B-4R-LBSBRACC
Llama3-8B-Instruct6.151.0613.8061.8235.7411.69
     + OASIS-SFT7.111.2816.5264.0039.8216.69
     + OASIS8.752.8020.3167.9051.1426.69

Evaluation with generated context

ModelB-2B-4R-LBSBRACC
Llama3-8B-Instruct0.480.061.4350.9718.5213.69
     + OASIS-SFT4.930.6714.3662.7137.0916.99
     + OASIS7.232.2519.6267.5152.0329.12
AuthorsOpenReview ↗

(2/5) Official Comment by Authors

W2&Q1: User persona diversity and per-persona evaluation under conflicting preferences.

Thank you for raising this queston. The user simulator is conditioned on user personas, with both persona and task information incorporated into turn-by-turn response generation. For the same context request, the assistant adjusts its response based on preferences expressed in the conversation history. When preferences have not yet been revealed, the assistant first asks about key constraints. Below, we provide additional analyses organized by task-relevant user types and present several cases for more intuitive illustration:

  • General tasks. The training dataset (ThoughtTrace) for our general-domain simulator contains user background information, as well as constraints and expectations recorded by users at each turn. Since the original WildBench dataset does not provide persona-type labels, our results do not include persona scores. We conducted an in-depth case study and found that benchmark samples do contain explicit user characteristics that influence response strategies. The following examples provide intuitive illustrations of budget constraints and preferences for step-by-step explanations, respectively.
User traitOriginal inputResponse
Budget-consciousCreate online marketing plan for magento, shopify services with budget of 200-300 SGD per monthStart with a monthly cap of SGD 250: SGD 150 for a small ad test, SGD 50 for case-study materials, and SGD 50 in reserve. Focus on one service and one channel first. Review the cost per qualified inquiry each week before spending more.
Prefers step-by-step explanationsGive numerical example and walk through it step by step to explain bellman equation. walk slowlyLet’s start with one action that always leads to the same next state. The reward now is 2, the next state’s value is 4, and the discount factor is 0.5. The current value is 2 + 0.5 × 4 = 4. The first part is what you get now, and the second is the discounted future value. We can look at multiple actions after this step.
  • Business tasks. We identify three persona dimensions based on users' explicit statements: Price-sensitive users exhibit evidence of cost trade-offs or constraints on additional spending; Efficiency-oriented users express preferences for direct, concise responses or reduced repetitive communication; and Users requiring detailed guidance show evidence of unfamiliarity with operations or explicitly request step-by-step, detailed instructions. Through rule-based screening and instance-by-instance semantic review of 1855 evaluation conversations, we identified these three user types and present one case for each.
User traitDialoguesOriginal inputResponse
Cost-sensitive109 (5.88%)That puts my mind at ease. Also, about repayment, I can pay at most 500 a month… Are there extra fees for paying in installments? I really cannot afford any more expenses.Your monthly limit is 500, including any fees. Before choosing an installment plan, we need to check the total fees and the actual monthly payment to see whether it fits that limit.
Efficiency-focused217 (11.70%)I have already explained the situation in detail. I just need a specialist to contact me now. I do not need to explain it again.You do not need to repeat the details. You want the charges on all settled bills checked and a specialist to call you back. What time would work for you?
Needs step-by-step guidance29 (1.56%)Hello, how do I apply for this loan? I really struggle with using my phone. I have logged in, but the busy screen is confusing. That would be great. Please go slowly and my hands are a little shaky.We can go slowly, one step at a time. You are already logged in. First, tell me which buttons you can see on the screen, and I will guide you from there.

These three user types correspond to different response strategies. Price-sensitive users require verification of total expenses before proceeding, efficiency-oriented users benefit from reduced repetitive communication, and users requiring detailed guidance need a slower pace and step-by-step instructions. Existing results also support improvements in strategy selection: Qwen3-8B's strategy accuracy on DianJin-CSC increases from 33.71 to 37.92, and from 33.72 to 39.45 under the generated-context setting.

AuthorsOpenReview ↗

(3/5) Official Comment by Authors

W3: Human evaluation of assistant and statistical reliability of user simulator.

Thank you for your valuable suggestion. We also agree that adding a human evaluation of the assistant is crucial for validating the effectiveness of OASIS. Below, we provide human evaluation results on assistant responses and further clarify the evaluation criteria and statistical analysis of the user simulator. Specifically:

  • Simulator Evaluation Criteria and Clarification. We would like to clarify the definitions of the three evaluation dimensions in Table 2: Plausibility measures whether the generated thoughts reflect reasonable mental activities that real users might have; Context Consistency measures their consistency with the preceding multi-turn conversation history; and Human-Thought Alignment measures their alignment with thoughts self-reported by real users. All dimensions are rated on a 1–5 scale, where a score of 4 indicates that the evaluation requirements are generally met. The ≥4 (%) metric represents the proportion of samples whose average score across the three dimensions is at least 4. After training on real-user data, the user simulator improves this proportion from 69.2% to 86.8%. We add details in Section 3.2 (L357–L364).

  • Human Evaluation of Assistant. To directly evaluate whether OASIS better understands and satisfies real users' needs, we conduct an evaluation on 200 held-out real-user conversations from ThoughtTrace, using users' self-reported thoughts as evaluation references while providing the assistant with only observable dialogue. Three independent human annotators, each holding a master's degree in engineering, evaluate the assistant responses along four dimensions: (1) Constraint Alignment, whether responses correctly identify and address constraints that genuinely matter to users; (2) Need Satisfaction, whether responses effectively satisfy users' actual needs; (3) Assumption Grounding, whether responses avoid unsupported assumptions about user preferences; and (4) Clarification Appropriateness, whether assistants appropriately seek clarification when important constraints remain uncertain. These results suggest that OASIS better identifies and addresses real users' needs while reducing unsupported preference assumptions.

    ModelConstraint AlignmentNeed SatisfactionAssumption GroundingClarificationOverall
    Qwen3-8B57.258.875.952.661.1
         + OASIS-SFT70.172.083.768.473.6
         + OASIS76.878.187.377.379.9
  • Inter-Annotator Agreement and Statistical Reliability. We compute ordinal Krippendorff's α for each evaluation dimension. For simulator faithfulness, the agreement scores are 0.74, 0.77, and 0.72 for Plausibility, Context Consistency, and Human-Thought Alignment, respectively. For assistant quality, the agreement scores are 0.72, 0.76, 0.79, and 0.70 across the four dimensions, indicating inter-annotator consistency. We further conduct paired significance tests and estimate 95% confidence intervals using 10,000 bootstrap resamples. For simulator evaluation, the ≥4 (%) metric increases from 69.2% to 86.8%, with 95% Wilson confidence intervals of [65.0%, 73.1%] and [83.6%, 89.5%], respectively. The improvement in Human-Thought Alignment is statistically significant (p < 0.001). For assistant evaluation, OASIS improves the overall score by 6.3 points over OASIS-SFT (95% CI: [4.1, 8.5], p < 0.001). Additionally, we provide detailed five-point scoring rubrics and clarify the calculation of the ≥4 (%) metric in Appendix A.5 to improve evaluation transparency.

We have supplemented our results with human evaluation to further validate reliability (Section 3.3, L394-L410). Detailed evaluation criteria are provided in Appendix A.5 & A.6 (L792–L838).

Reviewer_x376OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes OASIS, a framework for learning from latent user thoughts as training-time privileged information while requiring only observable dialogue at deployment. It combines supervised cold-start training with on-policy asymmetric self-distillation, without relying on a stronger assistant teacher. Experiments with Qwen3-4B and Qwen3-8B on WildBench and DianJin-CSC report improvements in general assistance and commercial customer service, with further gains from self-distillation over the SFT initialization.

Critical strengths
  1. A well-motivated use of user-side privileged information with a clear training–deployment distinction. The framework addresses partial observability in assistant–user interaction by using latent user thoughts as training-time supervision while retaining dialogue-only inputs at deployment. Using the same assistant backbone in the two information views makes the source of supervision explicit: the advantage comes from access to additional user information rather than a higher-capacity assistant teacher. Applying this signal to student-generated interaction states is also a reasonable way to address the distribution shift associated with purely offline demonstrations. This is a meaningful integration of user simulation, privileged supervision, and on-policy self-distillation; I do not interpret the underlying privileged-learning or self-distillation paradigms themselves as entirely new.

  2. Consistent reported improvements across model scales, domains, and evaluation settings. Full OASIS improves over OASIS-SFT under both WildBench judges and across the reported task categories. This breadth of evidence supports the practical value of the training approach beyond a single model or evaluator, although the paper does not establish multi-seed statistical significance.

Critical weaknesses

The WildBench evaluation lacks sufficiently detailed implementation to fully verify the reported scores.

Section 3.1 notes that the evaluation adheres to the benchmark protocol and designates GPT-OSS-120B and Qwen3-235B as judges. However, the WB_Score may be computed differently from the official WildBench formulation — whether via rescaling, or category-to-overall aggregation. This concern arises because WB_Score values in Table 1 and Tables 4–6 of Qwen3 base models are negative, whereas under the canonical WildBench calculation, only WB_Reward is expected to admit negative values. This point requires clarification.

Questions for authors

Need to specify the exact WildBench scoring configuration used for Tables 1 and 4–6

Please confirm whether the WB_Score reported in your results is computed identically to the official WB_Score from WildBench.

As an absolute scoring metric, WB_Score is supposed to be non-negative. By design, WB_Reward is measured against a baseline model and can therefore take both positive and negative values. WildBench does mention the possibility of score remapping or other modifications to the calculation pipeline, and any such adjustments should be explicitly stated.

2 comments

Thank you for acknowledging the value of well-motivated use of user-side privileged information, effective training methodology, and consistent performance improvements. We have carefully prepared responses to address your concerns.

W1&Q1: The WildBench evaluation lacks details to verify the reported scores.

Thank you for raising this important question. According to WildBench's official scoring protocol, WB Score is not the original absolute score but a rescaled version of the Raw Score, and therefore can take negative values. We follow the same evaluation procedure as WildBench, but replace the original GPT-4-Turbo judge with GPT-OSS-120B and Qwen3-235B. Specifically:

  • The Raw Score is calculated by averaging the judge's ratings across all evaluation samples, with each response rated on a scale of 1 to 10. According to the official repository and leaderboard, the Raw Score is first rescaled using (Raw Score − 5) × 2 and then multiplied by 10 for leaderboard aggregation. Therefore, our reported WB Score = (Raw Score − 5) × 20, with a range of [−80, 100]. A negative WB Score indicates that the average Raw Score is below 5.

  • WildBench considers a Raw Score of 5 to be borderline acceptable. Under GPT-OSS-120B evaluation, Qwen3-8B and Qwen3-8B-OPD achieve Raw Scores of 4.02 and 4.10, respectively. In contrast, under Qwen3-235B evaluation, all models achieve Raw Scores above 5. We would also like to clarify that we use WB Score, which is based on absolute scoring of individual responses, rather than WB Reward, which is based on pairwise comparisons between models. Despite differences in absolute scores, both judges produce consistent overall model rankings.

We have added a detailed explanation of WildBench and the rescaling procedure in Section 3.1 (L295–L299), and included an additional Raw Score column in the WildBench-related tables.

Multi-seed experiment and statistical significance analysis mentioned in Strengths.

Thank you for raising this important point. We supplement key results with multi-seed training experiments, confidence intervals, and statistical significance tests. Specifically:

  • Multi-Seed Evaluation. We repeat key experiments with three independent training seeds (42, 123, and 456) and report the mean and standard deviation across runs. WildBench is evaluated using GPT-OSS-120B as judge, while DianJin-CSC reports response-strategy accuracy under generated-context evaluation. All results are reported as mean ± standard deviation. These results indicate that the main performance improvements of OASIS remain consistent across different training seeds.

    ModelWildBench (WB Score)DianJin-CSC (ACC)
    Qwen3-8B
         + OASIS-SFT24.7 ± 0.733.7 ± 0.6
         + OASIS37.7 ± 0.839.4 ± 0.8
    Qwen3-4B
         + OASIS-SFT15.6 ± 0.529.3 ± 0.8
         + OASIS18.3 ± 0.635.8 ± 0.9
  • Statistical Significance Analysis. We conduct paired significance tests and estimate 95% confidence intervals using 10,000 bootstrap resamples. On WildBench, compared with OASIS-SFT, OASIS achieves statistically significant improvements of 12.95 points for Qwen3-8B, 95% CI: [10.2, 15.6], p < 0.001 and 2.73 points for Qwen3-4B, 95% CI: [0.4, 5.0], p < 0.05.

We add the standard deviations, confidence intervals, and significance tests in Appendix A.7 (L840-L859).


Thank you again for the thoughtful questions and positive evaluation of our work, which serve as great encouragement for us. We hope these clarifications resolve your concerns, and we sincerely appreciate your willingness to further evaluate our work.

AuthorsOpenReview ↗

Looking forward to your reply!

Dear Reviewer x376,

We hope this message finds you well.

We have carefully addressed the your comments and made revisions to our manuscript, as detailed in our responses and the revised PDF.

As the discussion period is approaching its end, we would greatly appreciate it if you could take another look at our responses and share your feedback. Your insights are invaluable to improving the quality of our work, and we sincerely appreciate the opportunity for further discussion.

Thank you very much for your time and consideration. We would be happy to provide any additional clarifications or details if needed.

Best regards,

The Authors of Submission #7428

Comments on the paper

1 comment
AuthorsOpenReview ↗

Note to AC: Rebuttal Summary

Dear AC,

We would like to sincerely thank you and all reviewers for the careful reading of our manuscript and for the constructive suggestions, which have substantially improved the clarity, rigor, and scope of the work.

To assist your evaluation, we provide a brief summary of our work.

Paper Overview

LLM assistants face partial observability: user utterances reveal only part of the underlying intents, preferences, and concerns. We study whether such latent user information can serve as training-time privileged supervision and be internalized into an assistant that observes only dialogue at deployment.

We find that latent user thoughts provide effective privileged supervision, improving response quality even for the same assistant. Building on this finding, we propose OASIS, which leverages a user simulator trained on real users' self-reported thoughts for cold-start training, then employs on-policy asymmetric self-distillation to bridge the gap between privileged and dialogue-only views of the same assistant. This enables OASIS to internalize unspoken user needs without requiring latent thoughts at deployment, improving over cold start initialization by up to 52.4% on WildBench and 22% on DianJin-CSC.

Discussions with Reviewers

  • Reviewer Xf1U (raised to 4): "Thank you for the response and the extensive experiments. I have decided to raise my score to Accept."
  • Reviewer 8eN1 (raised to 3): "I thank the authors for the extensive effort they put into the rebuttal phase. I raised my rating from weak reject to weak accept, with Soundness going from 1 to 2."
  • Reviewer x376 (current score 3): The main concern was the computation of the WB Score, which we have addressed by providing a detailed explanation of the score rescaling procedure.

Recognized Strengths by Reviewers

  • Well-motivated problem formulation (x376, 8eN1, Xf1U): using user-side latent information as privileged supervision, with a clear training–deployment distinction, addresses a meaningful challenge in real-world assistance.
  • Principled framework design (x376, 8eN1, Xf1U): a clean three-stage pipeline in which each stage has a formal objective and a corresponding ablation, and the no-stronger-teacher design is a meaningful practical contribution.
  • Consistent empirical improvements (x376, Xf1U): gains hold across model sizes, domains across general and commercial scenarios.

At the end of rebuttal phrase, all three reviewers hold positive scores of OASIS.

Main Concerns and Questions Resolved

IssueReviewerSolution
WB Score computationx376, 8eN1We added the explanation in Section 3.1 (L294–L299) and a Raw Score column to all WildBench tables.
Contribution of latent thoughtsXf1U, 8eN1Controlled ablations (SFT w/o Privileged) in Section 4.1 (L414–L431) and a per-category WildBench case analysis show that the gains stem from latent thoughts.
Real-user validity8eN1, Xf1UA human evaluation of the assistant on real-user conversations in Section 3.3 (L394–L410) verifies that OASIS truly better addresses real users' needs. The detailed rubrics and inter-annotator agreement are provided in Appendix A.5 & A.6 (L792–L838).
More Experimentsx376, 8eN1(1) Multi-seed results with significance tests in Appendix A.7 (L840–L859). (2) Results on Llama3-8B-Instruct in Appendix A.4 (L762–L790) show generalization beyond the Qwen family.

We sincerely thank all reviewers for their constructive feedback and positive scores, which serve as great encouragement for us. Their valuable suggestions have helped us further improve the quality of our manuscript, in which all revisions are highlighted in blue.


We believe that our revisions, supplementary experiments, and detailed responses have carefully addressed all concerns. We hope that these efforts will be taken into consideration in the final evaluation, and we are truly grateful for your time and expertise throughout this review process.

Best regards,

Authors of #7428

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.