acceptodds
Under review as a conference paper at ICLR 2027

BiasFate: Calibration Limits of Reweighted LLM Simulations

Abstract

Can reweighting LLM-generated survey responses improve on the human sample used to calibrate them? BiasFate links calibration information, proposal coverage, and estimation error for categorical responses. With reused empirical proposal counts and no clipping, self-normalized importance sampling (SNIS) equals the calibration mean restricted to observed categories; full empirical coverage recovers the direct mean exactly. Under covered support with known proposal probabilities and independent draws, unnormalized plug-in weighting adds nonnegative variance, while SNIS inherits calibration variance to leading order. Support truncation and clipping introduce distinct sources of bias. Across roughly 640K generations, four –B models, a 34B replication, four task families, and released GlobalOpinionQA marginals, the direct calibration mean beats deployed clipped SNIS in 270 of 288 matched configurations. Removing clipping reduces covered-cell SNIS RMSE from 0.1027 to 0.0245, versus 0.0148 for direct estimation. Plateau diagnostics transfer to held-out questions, while diagnostic-guided routing and allocation show no improvement over matched controls. These results explain when simulation reproduces calibration information, how it loses target mass, and why diagnostic accuracy and estimation gains require separate evaluation.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
LLM-based simulation, importance weighting, self-normalized importance sampling, survey calibration, support deficiency, estimation diagnostics

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PHGVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies whether reweighting LLM-simulated categorical responses improves over direct estimation with the same real human sample that is used for reweighting. Theoretical and empirical analyses demonstrate that reweighting does not have substantial advantages over direct estimation.

Critical strengths

None

Critical weaknesses

The writing in this paper is extremely difficult to follow.

  1. Many technical terminologies and notations are used without definitions or explanations on how they relate to the problem (e.g., g^\hat{g}, b^\hat{b}, "plateau", "recovery-screen" in Section 3.3). Moreover, a large number of terminologies that appear later in the text are referred to in much earlier sections, without any explanations on their purpose. For example, Section 3 refers to experiment details in Section 5 (e.g., "human-target studies" and "LLM-teacher studies" in lines 142-143, and "The Chinese and English threshold studies" in line 180) without giving any explanations.

  2. The theory in Section 4 is presented with barely any motivations. The theorem statements themselves are also hard to understand. In particular, the results in Section 4.3 are basically stated in words, with a number of terminologies unexplained. The paper offers almost no explanations and interpretations of the theorems. It is completely unclear to me how the theorems relate to each other, and contribute to the conclusions of the paper.

  3. The experiments in Section 5 have similar presentation issues. It reports numerous experiment results without clearly explaining the experiment goals and setups, interpretations of the findings, and how they support the paper's conclusions.

Questions for authors

None

Reviewer_rW9sOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors study whether giving different weights to AI-generated survey answers can improve estimates based on a small reference sample. They find that directly using the reference sample generally works better. However, much of this conclusion follows from how the problem is set up, and the proposed checks do not demonstrate much additional practical value.

Critical strengths
  • The main result is clearly stated (§4.1).
  • The authors test several variations, include questions not used to develop the method, and report unsuccessful experiments (§§5.2–5.4).
  • They also explain important limits on their conclusions.
Critical weaknesses
  • For each question, the authors already have a reference sample and know how to score its answers (§3.1). They can therefore estimate the desired average directly from those answers. Showing that generating and adjusting additional answers does not improve this estimate is a useful clarification, but the main mathematical result follows from a simple cancellation in the formula. The paper needs stronger experimental findings to establish a substantial contribution beyond that observation.

  • The main comparison is also strongly affected by the rule that limits how small or large the weights can be. Removing this rule reduces the reported error from 0.1027 to 0.0245 (§5.2). Meanwhile, using the proposed checks to choose between actions or allocate resources does not improve results over the comparison methods (§5.4). The large number of generated answers does not resolve this limited added value.

  • Some central descriptions remain difficult to follow. I could not determine exactly which samples are used in the experiment without weight limits in §5.2. Figure 1 makes a broad claim about unavoidable error that appears to conflict with §4.3. Figure 2’s titles, "T1 scale is tight, honest" and "T2 floor predictable," are very hard to parse. I could not understand what "honest" means here or how the figure supports that description.

Questions for authors
  1. Proposition 1 states that the adjusted estimate "equals the direct calibration mean exactly" under its stated conditions, but §5.2 reports different errors for the two estimates. Could this happen because the weights are calculated using one set of generated answers and then applied to a different set?
  2. Figure 1 describes "a bias floor that no reweighting removes", while §4.3 says that an unavoidable error in the estimated average does not always exist when answer scores are known. How should these two claims be reconciled?
Reviewer_oDpcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tackles the limitations of importance reweighting for calibrating LLM-generated survey responses using small human samples. Through theoretical analysis and empirical experiments, it identifies conditions under which additional LLM samples fail to improve estimation over human-only baselines.

Critical strengths
  • The paper tackles the important problem of LLM-based human response simulation.

  • The paper provides a systematic theoretical analysis of the limitations of reweighting.

  • It supports the theoretical findings with extensive experiments and human-only baselines.

Critical weaknesses
  • Unclear practical relevance of the analyzed methods. The paper focuses on reweighting LLM responses using human labels collected for each new question. However, it provides insufficient evidence that this approach is widely adopted in academia or industry. Concrete references or real-world applications are needed.

  • Limited generalizability of the theoretical results. Existing approaches combine human labels and model predictions through Prediction-Powered Inference [1], doubly robust estimation [2]. These approaches differ from the categorical importance reweighting analyzed in this paper. It remains unclear how broadly the theoretical findings apply to established calibration methods.

  • Unclear practical deployment scenario. The proposed setting requires new human responses for each question. Under this requirement, the practical benefit of incorporating additional LLM samples over human-only estimation is unclear.

[1] Prediction-Powered Inference, Angelopoulos et al., 2023. [2] Using Large Language Model Annotations for the Social Sciences: A General Framework of Using Predicted Variables in Downstream Analyses, Egami et al., 2023.

Questions for authors
  • Can the authors provide evidence that the analyzed reweighting methods are widely used in academic research or industrial applications?

  • To what extent do the theoretical results extend to established methods such as Prediction-Powered Inference and doubly robust estimation?

  • What practical scenarios justify collecting human responses for every new question while also incorporating additional LLM samples?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.