acceptodds
Under review as a conference paper at ICLR 2027

Scaling Laws for Simulation Fidelity: Scale Alone Is Not Enough

Abstract

Both humans and AI can learn new skills by practicing with simulated human partners. Current LLM simulations can already provide meaningful learning signals, but they remain significantly less effective than human partners. Although LLM simulations look plausible on the surface, they often fail to reproduce human behavior patterns that are necessary for realistic and effective practice. We call this a gap in simulation fidelity. First, we introduce and validate distributional fidelity metrics, which we use to quantify fidelity gaps in two representative simulation tasks. We show that domain adaptation fails to close these gaps. This motivates our investigation of scaling laws for simulation fidelity. Web pre-training data is cheaper and more scalable than in-domain data, so we systematically study how simulation fidelity scales with pre-training compute on webtext. We discover that fidelity scales log-linearly with pre-training at smaller scales, but fidelity quickly asymptotes. Relative to other skill domains like creative writing and open-ended question answering, the rate of simulation performance improvement is much slower. Furthermore, domain adaptation does not alter the rate at which pre-training compute benefits model fidelity. In summary, we conclude that, even at scale, webtext data is an insufficient grounds for building more faithful LLM simulators. We recommend greater investment in larger-scale in-domain datasets than what are currently available.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
compute scaling laws, domain adaptation, user simulation, patient simulation, skill training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Hw6DOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work focuses on the task of LLM-simulated human partners, where the authors aims to investigate simulations fidelity. Specifically, the authors propose new metrics to quantify the fidelity of LLM simulations and identify persistent performance gaps in frontier models that cannot be fully mitigated through in-domain adaptation.

Critical strengths
  1. The research topic, investigating simulation fidelity, is interesting and holds practical value.
  2. The proposed fidelity metric is interesting and meaningful, providing a quantitative method to measure the simulation fidelity.
  3. The experiments are relative extensive, examining simulation fidelity from various perspectives.
  4. Overall, this work is easy to follow.
Critical weaknesses
  1. Although investigating simulation fidelity is an interesting research direction, this work does not provide an effective solution or a promising framework for improving simulation fidelity. Instead, it primarily focuses on measuring fidelity and identifying existing performance gaps. The authors should clarify the practical implications of their findings and discuss how the proposed metrics could help improve LLM-simulated human partners.
  2. The selection and construction of the features described in Section 2.2 need further clarification. Since these features play a critical role in measuring simulation fidelity, the authors should provide more details about how they are selected, defined, and constructed. They should also justify why these features are appropriate for capturing the key aspects of simulation fidelity and discuss any potential limitations of the chosen feature set.
  3. The proposed simulation fidelity metric relies on ground-truth human feature distributions, which are generally difficult to obtain in many real-world scenarios. This requirement may limit the practical applicability of the proposed method. The authors should clarify the data requirements of the metric and discuss how it could be applied when ground-truth human feature distributions are unavailable or only partially accessible.
Questions for authors

Please read the above sections.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Hw6D

Thank you for your recognition of this paper’s strengths. We address your concerns below.

  • Weakness 1: Practical implications. Our experiments provide three practical insights and recommendations. We will expand on this discussion with the extra page of the camera-ready paper.

    • (1) Our core recommendation for improving simulation fidelity is to invest more heavily in domain-specific datasets. This new plot visualizes the benefits of in-domain data for patient simulation ( anonymous link: https://osf.io/em3np/files/qsa7w?view_only=b725212241894c6793de2123a0a81803 ) and these results confirm that the in-domain data scale needs to grow with model size.
    • (2) One key finding is that even the best models over-act and fail to provide enough resistance. There is a rapidly growing literature on mitigating model sycophancy that can provide concrete mitigation strategies. Our contribution is to systematically characterize the problem so that practitioners can apply not only new techniques, but also existing techniques proven effective on related problems.
    • (3) By demonstrating the relative limitations of pre-training compute scale, we draw attention to the importance of a problem that will not be solved as readily by the current trajectory in AI. And by offering concrete metrics, we allow model developers to isolate and iteratively test the faithfulness of simulations along concrete feature axes.
  • Weakness 2: Feature selection. Our features were selected from established conversational and psychotherapy frameworks to capture behaviors relevant to each simulation setting. For user simulation, the seven features cover dialogue act, feedback, politeness, topic shift, emotion, emotional disclosure, and information disclosure. For patient simulation, we draw on psychotherapy-process instruments and constructs such as the Experiencing Scale (Klein et al., 1969), Insight (Hill et al., 1992), and Innovative Moments (Gonçalves et al., 2011), informed by the Generic Model of Psychotherapy, RDoC, and MITI 4.2. Together, these capture how clients present problems, engage emotionally, express expectations, and show signs of change. Their relevance is supported by our frontier-model analyses, which show systematic differences in these dimensions (Appendices F and H). Definitions and codebooks are provided in Appendices C and G, and reliability-based selection is described in Appendix E. We will clarify the feature construction and limitations, including that reliability screening may exclude clinically important behaviors and that our features do not capture all aspects of fidelity, such as longitudinal consistency.

  • Weakness 3: Ground-truth data requirements. Our fidelity metric is intended for settings where representative human interaction data can be collected or already exists, such as user-interaction datasets and psychotherapy transcripts. Once annotated, these reference distributions can be reused to evaluate multiple simulators. Smaller reference sets can still be used, although uncertainty increases. Our split-half procedure provides one way to estimate finite-sample variation. Without any human reference data, the JSD metric cannot be computed directly. We will clarify these requirements and distinguish direct fidelity measurement from proxy-based comparisons using language-model loss.

Thanks authors for their response. Althought the authors present some extra clarification, the Practical implications still appear a little limited. Moreover, I'm not sure whether it is in accordance with regulations to add extra link in the rebattle stage. In current stage, I will retain my positive scores.

Reviewer_L7rUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how well LLMs simulate human conversational partners in two settings: user simulation (PRISM) and therapy-patient simulation. It proposes three contributions.

  • A fidelity metric. Fidelity is measured as the Jensen–Shannon divergence between human and model distributions over LLM-judged behavioral labels, such as 7 user features and 10 patient features.

  • Current gaps. Frontier chat models “over-act” and provide too little resistance. In patient simulation, they are worse than a 2.5B base model. Continued pre-training (CPT) on about 15M in-domain tokens does not close the gap.

  • Scaling analysis. Using 85 IsoFLOP checkpoints pretrained on DCLM, the authors find that fidelity improves log-linearly but shows signs of saturating. Using BPB as a proxy, simulation domains lose ground relative to code faster than other generation domains, and CPT shifts the level of the curve but not its slope.

The authors conclude that webtext scaling is insufficient and that larger in-domain datasets are needed.

Critical strengths
  1. The paper proposes an important problem. Simulator fidelity matters for both RL training of agents and human skill training. Framing it through scaling laws is novel relative to prior MCQ-style work.

  2. The paper contains theory-grounded features. The behavioral features come from established clinical and dialogue coding schemes (Klein Experiencing Scale, CEAS, HIll, IMCS, etc.) rather than generic "realism" ratings.

  3. The experiments show a controlled compute suite. The IsoFLOP suite has a known pre-training corpus, which permits clean scaling fits and level-vs-rate analysis of domain adaptation.

Critical weaknesses
  1. The metric is marginal, not conditional.
  • The charpter §2.3 sums one-hot labels over all utterances, so πf\pi_f is a corpus-level label histogram. A simulator could match these marginals while being wrong on every individual context.
  • For patient features, the judge sees the candidate turn with no context (App. G.2), so the labels are not conditioned on hh either.
  • The claim of "conditional distributions" is therefore not supported. A per-context or context-stratified measure is needed.
  1. Judge validation is inconsistent and incomplete.
  • Table 1 reports judge-vs-human kk against "ground truth datasets." Several of the cited sources are coding manuals, not labeled corpora.
  • Appendix E instead says patient features were admitted by agreement between two LIM judges (Qwen2.5-14B and Gemini 2.5 Flash, k ≥ 0.35).
  • Expert validation covers only three AnnoMI features, none of which are among the ten features used.
  • Three judges are mentioned across the paper (GPT-4.1-mini, Qwen, Gemini). The text does not say which one produced each number.
  1. Internal numerical inconsistencies.
  • User simulation: Figure 3 shows best checkpoints near 0.03 at 102010^{20} FLOPs, close to the frontier model. Yet Figure 1 reports Delphi-2.5B Base at 0.082.
  • Patient simulation: Figure 3 shows selected checkpoints near 0.003 from about 3×10193×10^{19} FLOPs onward. Table 2 reports base JSD of 0.0101 at 3×10193×10^{19} and 0.0061 at 9×10199×10^{19}.
  • Data details: the minimum target length is 60 characters in §2.1 but 120 in App. B. The sample count is 1,826 in §2 but 500 in App. B. The compute range is 1018−102010^{18} - 10^{20} in Chapter §4 but 3×1018−3×10203×10^{18} - 3×10^{20} in §5.3/App. D.
  1. Presentation.
  • There are many typos and grammatical errors (e.g., “pre-trainined,” “slower relative is,” “continued pre-training on can,” the duplicated “Asymptote in performance”).
  • Figure 2 is labelled “fine-tuning” though the method is CPT.
  • Key methodological details sit only in the appendix.
  1. The title and conclusions overreach the evidence.
  • “Scale alone is not enough” is drawn from models of at most 2.5B parameters and 102010^{20} FLOPs, about five orders of magnitude below frontier compute.
  • The frontier models differ from the base models in post-training, prompting format (instructions vs. few-shot completion), and decoding. So the frontier gap cannot be attributed to scale, and “post-training harms fidelity” is not causally tested. A matched base/instruct comparison would be needed.
  • The abstract says fidelity “quickly asymptotes,” but the user-simulation fit is linear with R2=0.975R^2 = 0.975.
  1. Feature selection may drive the saturation result.
  • Table 4 shows that the excluded patient features have large initial divergences and steep slopes.
  • The retained features already sit near the floor (composite JSD = 0.003-0.012).
  • The reported asymptote may therefore reflect which features were kept, or judge noise, rather than a real fidelity ceiling.
  • Figure 1 has no confidence intervals. Differences such as 0.012 vs 0.003 need significance tests.
Questions for authors
  1. How is the “human noise floor” computed (split-half resampling, bootstrap)? Can you report CIs and significance tests for Figure 1 and Table 5?

  2. How does the human noise floor account for inter-person variability given a single human continuation per context?

  3. Can you report a context-conditional fidelity metric, such as per-context expected JSD or a metric stratified by dialogue state, and rerun patient labeling with context visible to the judge?

  4. Please reconcile the discrepancies noted above: Figure 1 vs Figure 3 for user simulation, Figure 3 vs Table 2 for patient simulation, and the sample counts and length filters.

  5. Can you include a matched base vs. instruction-tuned pair (e.g., an open model family) to directly test the post-training claim?

  6. Does fidelity improve when the in-domain data scale or adaptation budget grows with model size? This seems necessary to support the recommendation to invest in in-domain data.

2 comments
AuthorsOpenReview ↗

Response to Reviewer L7rU

Thank you for your support for the paper’s strengths. We now address each of the weaknesses you raise in turn.

  • Weakness 1 and Question 3. We update our results with a more appropriate conditional fidelity metric (see General Response 1)

  • Weakness 2. Please see General Response 2 for a clarification about the judge validation.

  • Weakness 3 and Question 4. Thank you for your careful attention to the numerical details of the paper. Many of the apparent inconsistencies come from different model families and evaluation protocols, rather than numerical errors. We will make these distinctions explicit in the final paper.

    • Figure 1 vs. Figure 3. The scaling figure 3 draws on Marin’s full set of 85 IsoFLOP checkpoints and fits the curve on the best model for each IsoFLOP bucket, whereas the "Base" entries in Figure 1 report Marin’s Delphi-2.5B Base, which is the 2.5B model that Marin selected as the IsoFLOP-bucket winner for their held-out corpus. Delphi-2.5B Base is not the optimal checkpoint for our evaluation sets, but we selected it as a useful comparison to calibrate the effects of continued pre-training (CPT). Given compute limitations, we could not perform CPT on all 85 IsoFLOP checkpoints.
    • Sample count 1,826 vs. 500 and minimum target length 60 vs. 120. Our main fidelity experiments in Sections 3-4 use all 1,826 conversation turns, with client turns of at least 60 characters for larger statistical power. Our Section 5 experiments use loss rather than fidelity, so to be consistent with evaluations of 500 documents from longer-form text like code, math, and Wikipedia, we use 500 patient conversations with client turns of at least 120 characters.
    • Compute ranges. As in Figure 1 vs. Figure 3, the apparent discrepancy is between Marin’s full set of 85 IsoFLOP checkpoints used for scaling experiments, and Marin’s final IsoFLOP-bucket winners used for CPT experiments.
  • Weakness 4. Thank you for your edits. We will fix typos and presentation issues in the camera-ready.

  • Weakness 5 and Question 5. Please see General Response 3 for the new experiments you suggested. In the camera-ready paper, we will tone down claims that “scale alone is not enough.”

  • Weakness 6. Our feature selection was guided by an extensive review of the psychotherapy literature, rather than by the observed scaling trends. We draw on established psychotherapy-process instruments and constructs, including the Experiencing Scale, Insight, and Innovative Moments, which prior work has linked to therapeutic outcomes. Together, these features capture important aspects of how real clients communicate during therapy, including how they present their problems, engage emotionally, and express signs of psychological change. Since our goal is to evaluate how faithfully LLMs simulate human conversational partners, we believe the retained features provide a meaningful and sufficiently broad measure of the behavioral dimensions relevant to this application. Features were retained based on annotation reliability, not their initial JSD or scaling slopes. While this criterion excludes some clinically relevant behaviors that are more difficult to annotate consistently, we believe the retained features capture important aspects of human behavior in therapeutic conversations. Their relevance is also supported by our frontier-model evaluations, which reveal systematic differences between simulated and real clients, such as excessive emotional expression and insight (Appendices F and H).

  • Questions 1 and 2. Please see General Response 4 for details on the human noise floor.

  • Questions 6. Please see General Response 5 where our new plot shows fidelity improves when the in-domain data scales with model size.

  • Ethical Concerns. We appreciate your time to ensure the ethical conduct of this research.

    • Data licensing. The AS license permits the use of data for evaluating LLMs, and there is a precedent for this in the literature (see, for example, https://mental.jmir.org/2026/1/e95389). The GPT, Claude, and Gemini APIs we use for evaluations do not use data submitted through these APIs to train or improve models. We will remove verbatim quotes from the Appendix in the camera-ready paper.
    • Release scope. We will not distribute any transcript-derived data or models or model generations.
    • Concrete mitigations for impersonation. We will add the following concrete mitigations: controlled access (see release scope above) and misuse evaluations (red-team models for impersonation, manipulation, and deceptive interactions before release), issuing models to the public in a staged release manner as safety and utility are more rigorously demonstrated by experiments.

Thanks for the additional explanations. However, there are numerous spelling errors in the manuscript; I suggest that the author correct these during the rebuttal phase and highlight all such corrections. Besides, there are limited ablation studies in this paper. Therefore, I had to keep my original score.

Reviewer_sZ4YOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how faithfully language models simulate human users and psychotherapy patients. It compares distributions of behavioral labels, evaluates a controlled suite of pretrained models, and studies domain adaptation. And it finds improvements with compute but persistent behavioral discrepancies, motivating investment in domain-specific simulation data.

Critical strengths
  • The application is meaningful. Simulator quality matters for training conversational agents and for practicing professional interactions. Showing that a capable assistant can still be an unrealistic partner is useful for model selection and evaluation.
  • The extension to open-ended interaction is substantive. The paper moves beyond discrete responses and aggregate capability benchmarks to user and patient utterances with interpretable behavioral dimensions.
  • The controlled model suite supports a useful empirical study. Examining many compute-matched checkpoints is more informative than comparing a few commercial models with different training pipelines.
Critical weaknesses
  • The connection between the proposed metric and practical simulator quality is incomplete. Matching aggregate behavioral frequencies does not ensure that a simulator responds appropriately to a particular history. A system could preserve those frequencies while assigning responses to the wrong contexts. The paper does not show that lower JSD improves subsequent agent performance or practitioner learning.
  • The strongest resource-allocation conclusion is premature. A plateau over a limited compute range and limited adaptation budgets does not establish a general limit of web pretraining. Differences in irreducible uncertainty can also affect raw cross-domain loss ratios. These issues matter because the paper recommends redirecting investment toward larger domain-specific datasets.
  • The judge-validation account needs clarification. The main text describes human–judge agreement, while the appendix also uses agreement between two LLM judges for feature admission. The source and validation of each behavioral measure should be explicit before the metric is used to guide deployment decisions.
Questions for authors

refer to weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer sZ4Y

Thank you for your support for this work and for highlighting its particular strengths. Please see our “General Response” where we respond to the three weaknesses you raise.

  • Weakness 1: Measuring simulator quality. We agree that matching aggregate behavioral frequencies does not ensure that a simulator responds appropriately to a particular history. We update our results with a more appropriate conditional fidelity metric (see General Response 1)
  • Weakness 2: Scaling claims. Our paper measures the limitations of scaling pre-training data and recommends in-domain data. Please see Table 2 in Appendix D which shows that there are systematic performance gains from continued pre-training (CPT) on in-domain data, but the current scale of available in-domain data is not enough for statistically significant improvement on larger models. Please also see General Response 3 for new validations of our claim that scaling pre-training data is not enough. We will revise the abstract and Appendix D to make clear that the plateau applies only to the compute and data ranges studied, rather than establishing a general limit of web pre-training.
  • Weakness 3: Judge validation. Please see General Response 2 for a clarification.

I thank the authors for their detailed rebuttal. I think my current score already reflects my assessment, so I will keep it unchanged.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to the Reviewers

We group our responses to shared concerns below with additional experiments to clarify our findings.

  1. Conditional rather than marginal fidelity metric (Reviewer L7rU Weakness 1, Reviewer sZ4Y Weakness 1). We recompute our results with a conditional metric. Our paper’s main conclusions still hold (see the table):

    A. LLMs are unfaithful simulators, and significantly underperform Human Level

    B. Continued pre-training (CPT) fails to close fidelity gaps

    C. Frontier models are not better than base models at patient simulation

    D. Similarly, instruction-tuned models are not better than base models at patient simulation

Model / referencePatient (Conditional)User (Conditional)Patient (Marginal)User (Marginal)
Human referenceN/A (0.0)N/A (0.0)0.003 [0.002, 0.003]0.005 [0.004, 0.005]
Frontier LLM0.532 [0.518, 0.547]0.315 [0.299, 0.332]0.064 [0.056, 0.076]0.029 [0.024, 0.036]
Delphi Base0.531 [0.515, 0.547]0.432 [0.418, 0.445]0.023 [0.018, 0.031]0.082 [0.075, 0.093]
Delphi CPT0.524 [0.508, 0.539]0.366 [0.354, 0.379]0.014 [0.011, 0.019]0.033 [0.029, 0.040]

Our updated scaling laws have similarly steep slopes and fits

Patient (Conditional)User (Conditional)
Scaling Law Slope (ΔJSD/decade)-0.0035-0.0432
Scaling Law Fit (R2R^2)0.7840.967
  1. Judge validation (Reviewer L7rU Weakness 2, Reviewer sZ4Y Weakness 3). We will fix Table 1 and the body text to clarify the judge validation. For user simulation, the agreement κ was computed between the LLM and human Ground Truth Dataset. For patient simulation, the reference datasets did not include human labels for the core features, so instead, we computed LLM-LLM agreement (between Qwen2.5-14B-Instruct and Gemini 2.5 Flash) as a rougher proxy of judge validity. Ultimately we used Gemini 2.5 Flash for annotating patient simulation, and GPT-4.1-mini for annotating user simulation.

  2. The limitations of scaling pre-training data (Reviewer L7rU Weakness 5, Reviewer sZ4Y Weakness 2). We attributed the frontier performance gap to a limitation in scaling pre-training data, but we had not tested the effects of post-training on performance. The following new results compare the performance of a base against an instruct model as recommended by Reviewer L7rU. This supports our hypotheses that:

    A. For user simulation, post-training (instruction-tuning) improves performance, which supports the idea that the frontier gap is not a result of instruction-tuning

    B. For patient simulation, instruction-tuned Qwen2.5 is not significantly less faithful than base Qwen2.5 by either metric. By our new conditional metric, frontier models are not significantly worse than any other model, suggesting they are not performance-capped by instruction tuning and that there is a limitation to the benefits of pre-training scale.

Patient (Conditional)User (Conditional)Patient (Marginal)User (Marginal)
Qwen2.5-7B Base0.522 [0.508, 0.537]0.401 [0.387, 0.416]0.015 [0.012, 0.021]0.064 [0.056, 0.075]
Qwen2.5-7B Instruct0.518 [0.503, 0.533]0.368 [0.353, 0.383]0.024 [0.020, 0.029]0.035 [0.030, 0.043]
  1. Human noise floor (Reviewer sZ4Y Weakness 1, Reviewer L7rU Questions 1 and 2). Here we clarify our human noise floor, which measures finite-sample variation between marginal human distributions using split-half resampling. For user simulation, we randomly divide the 456 PRISM human continuations into two disjoint groups, compute the label distributions for each of our seven features, and average their JSD. For patient simulation, we split the 150 conversations containing 1,500 held-out human turns and repeat the calculation across ten features. Averaging over 1,000 random splits gives 0.00453 for user simulation and 0.00279 for patient simulation. Reviewer L7rU is correct to point out that this noise floor is computable only for marginal distributions rather than conditional distributions as there is only a single human continuation in the latter case.

  2. The benefits of in-domain data (Reviewer sZ4Y Weakness 3, Reviewer L7rU Question 6). Given the limitations of pre-training data, we endorse more investment in larger in-domain datasets. We created a new plot to visualize the benefits of in-domain data for patient simulation ( anonymous link: https://osf.io/em3np/files/qsa7w?view_only=b725212241894c6793de2123a0a81803 ) and these results confirm that the in-domain data scale needs to grow with model size. For smaller models, increasing the token budget up to 13M produces statistically significant gains, but for larger models, 13M tokens is insufficient.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.