acceptodds
Under review as a conference paper at ICLR 2027

Rethinking the Evaluation and Optimization of LLM-based Social Simulation

Abstract

LLM-based social simulation is a promising complement to traditional social science methods such as surveys and behavioral experiments. A core question in this area is how to evaluate the fidelity of LLM-simulated human behavior and, in turn, how to optimize LLMs toward it. Prevailing practice evaluates by accuracy, checking whether the model selects the single response observed from a human, and accordingly trains the LLM to reproduce this one hard label. However, human behavior is inherently subjective: the same person in the same situation may reasonably act in different ways, so an observed response is only one draw from an underlying response distribution, which renders accuracy-based evaluation unreliable and hard-label training misleading. To address these problems, in this paper, we first introduce the subjectivity coefficient, an entropy-based quantity that distinguishes objective tasks such as coding from subjective tasks such as social simulation, and then use it to systematically analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows. Based on the subjectivity coefficient, we further propose Subjectivity-Adaptive soft-Label Training (SALT): it pools observed outputs from semantically nearby inputs into soft distributional labels, with an aggregation radius adapted to the estimated subjectivity of each input; in the near-objective limit the neighborhood shrinks, so SALT naturally falls back to standard single-label training. Moreover, since existing datasets record only single observed responses and thus cannot support distributional evaluation, we construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions. Since real-world data typically provide only a single observation per input, our experiments train models from single observed outputs while evaluating them against the full response distributions, thereby verifying the feasibility of our method in realistic settings. Extensive results on SUBJSIM demonstrate the advantages of our method.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
social simulation, llm, evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dzpYOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The authors present a methodology to address the underlying mismatch between continuous and shifting human opinions/annotations and the single-label training and evaluation setup. Their method primarily involves a principled way to create a neighborhood of similar annotator-content samples, and extend single annotations by averaging over the neighborhood, and then use this target then for training. They show how this better captures the uncertainty and the plurality of human opinions in a test set of their construction.

Critical strengths
  • The paper identifies, poses and solves a genuinely interesting and important problem.
  • The authors provide a principled approach to control the amount of neighbor information used to construct the new training distribution, controlled by the "subjectivity" of the problem (at least at the moment for the current model being trained), shown to be superior to simpler alternatives.
  • The paper's motivation, and hence the provided solution, are easy to follow, keep readers interested and engaged, and provide genuine insight.
Critical weaknesses
  • Lack of detail on the dataset construction and some basic statistics in the main paper, important since the takeaways rely on the soundness of the dataset. The authors should not delegate all of the discussion to the appendix.
  • Lack of other datasets and models to show generalization of findings: The findings are supported only by the test set of the authors and for one model. This brings up another point about the generalizability. The method was motivated as using a neighborhood to generate a distribution that is otherwise not present, yet the examined dataset does already have such a distribution (I understand it is not used during training, correct?). It would be interesting to see generalization in a dataset that does not have such a property, where the neighbors are used to construct a distribution that is not there. Moreover, the authors have annotated each and every sample with all annotators, which is usually prohibitively expensive in practice. Generalization in a setting where each item has a few annotations and each annotator only sees a fraction of the dataset (i.e., the usual case) has not been established.
Questions for authors
  • Why are the RL objectives used in the paper? My assessment is that, since this is only done for classification purposes with no reasoning, it appears as if this was done for the sake of their inclusion, without any actual expected benefit. Is this accurate?
  • It would be useful to describe the complexity of the each gradient descent step, as efficiency is a point of emphasis for any training algorithm.
  • A clarification: the grouping, since it happens on x, take into account the persona embeddings and the content embeddings? Therefore, the grouping can happen from similar personas and/or similar content. Empirically, what neighbors are fetched? In your dataset with its high density of annotations and a less dense dataset?
  • For the formulation in Section 3.1 and the preference of one model over the other for accuracy: What would the case be on average for a perfect underlying model with sampling (iii.k2) compared to a simpler model (iii.k1)? You presented a specific example with a failure case, and theoretically it establishes your point, but in a whole population, empirically and on average, wouldn’t phi_theta_2 (its observed k2s) be better?
  • I had a hard time wrapping my head around definition 2 and theorem 1. Could you provide more intuition? Why is the definition what it is? Why are the terms called bias and statistical error? etc.
7 comments
AuthorsOpenReview ↗

Response (1/3)

We sincerely thank the reviewer for the positive assessment and for recognizing the importance of the problem. We are also grateful for the reviewer's questions, which are inspiring and help us improve the paper. We address each point below.


Weakness 1: Lack of detail on the dataset construction and some basic statistics in the main paper, important since the takeaways rely on the soundness of the dataset. The authors should not delegate all of the discussion to the appendix.

Response: We thank the reviewer for this suggestion. We agree that our conclusions rely on the soundness of the dataset, so its construction should be explained in the main paper itself. In the revision, we will expand Section 5.1 to cover the question source, the annotation protocol and quality control, the annotator pool, and key statistics of the dataset. Full details will remain in Appendix G.


Weakness2: Lack of other datasets and models to show generalization of findings: The findings are supported only by the test set of the authors and for one model. This brings up another point about the generalizability. The method was motivated as using a neighborhood to generate a distribution that is otherwise not present, yet the examined dataset does already have such a distribution (I understand it is not used during training, correct?). It would be interesting to see generalization in a dataset that does not have such a property, where the neighbors are used to construct a distribution that is not there. Moreover, the authors have annotated each and every sample with all annotators, which is usually prohibitively expensive in practice. Generalization in a setting where each item has a few annotations and each annotator only sees a fraction of the dataset (i.e., the usual case) has not been established.

Response:

  1. On the dataset already having a distribution. Correct: no distribution enters training. Our modeling assumption is that every subjective context has an underlying response distribution, whether it is annotated or not. This is not a property specific to SubjSim; it is a property of this class of tasks. No existing dataset provides such distributions, which is why we constructed SubjSim ourselves. On the training side, each context carries only a single label, and the soft label is built entirely by neighborhood aggregation. From the algorithm's perspective, SubjSim is therefore indistinguishable from a dataset without distributions: they exist only behind the scenes, as the evaluation reference.

  2. Annotation density. SALT adapts to annotation density automatically. Sparse annotation shrinks the group size ngn_g, and the radius rule compensates on its own: r∗≍(Keff∗ngL2)1/(dX+2)r^* \asymp \left(\frac{K^*_{eff}}{n_g L^2}\right)^{1/(d_X+2)} As ngn_g shrinks, the radius grows as ng−1/(dX+2)n_g^{-1/(d_X+2)}, pooling more neighbors to keep the statistical term controlled. The cost is small. The optimized error bound also scales as ng−1/(dX+2)n_g^{-1/(d_X+2)}, so halving the annotations multiplies it by 21/(dX+2)2^{1/(d_X+2)}, and at our estimated dX=28d_X = 28, halving the annotations would increase the bound by only about 2%.

We further verified this empirically. We subsampled the SubjSim training set from 100% down to 10% (about 16 personas per question instead of 165) and retrained SALT:

Train set#SamplesKL ↓JSD ↓TVD ↓MMD ↓
10%1,6000.30690.08760.28170.1607
20%3,3000.30920.08790.28370.1616
40%6,6000.29560.08420.27520.1540
60%9,9000.29510.08440.27600.1540
80%13,2000.29220.08340.27340.1524
100%16,5000.29250.08370.27350.1523

Two observations:
a. Performance is insensitive to density: removing 90% of the training data changes each distribution metric by only 3% to 6% in relative terms (KL 0.2925 → 0.3069).
b. Even with only 10% of the data, SALT still clearly outperforms SFT trained on the full data (KL 0.3069 vs. 1.2871). This shows that the aggregation mechanism itself borrows statistical strength across contexts, consistent with the radius-rule analysis above.

We thank the reviewer for raising this question. The subsampling results will be added to the revised paper.

AuthorsOpenReview ↗

Response (2/3)

Question1: Why are the RL objectives used in the paper? My assessment is that, since this is only done for classification purposes with no reasoning, it appears as if this was done for the sake of their inclusion, without any actual expected benefit. Is this accurate?

Response: This is not entirely accurate. Our task is to learn from human actions, and RL and preference optimization are the standard training paradigm for exactly this; they are not specific to reasoning tasks. In social simulation and opinion alignment, they are widely used: Sotopia-RL trains agents with GRPO in simulated social environments [1], and SDPO optimizes social agents with a DPO-style objective [2]. Closest to our setting, preference optimization has also been applied to multi-choice opinion tasks: GPO learns group preference alignment on OpinionQA [3], and GDPO aligns the group opinion distribution of GlobalOpinionQA with a distributional variant of DPO [4]. Including DPO and PPO as baselines therefore follows established practice in this line of work.

[1] Yu et al., 2025. Sotopia-RL: Reward Design for Social Intelligence. arXiv:2508.03905.
[2] Kong et al., 2025. SDPO: Segment-Level Direct Preference Optimization for Social Agents. ACL 2025.
[3] Zhao et al., 2023. Group Preference Optimization: Few-Shot Alignment of Large Language Models. arXiv:2310.11523.
[4] Yao et al., 2024. GDPO: Group Distributional Preference Optimization. arXiv:2412.20299.


Question2: It would be useful to describe the complexity of the each gradient descent step, as efficiency is a point of emphasis for any training algorithm.

Response: The per-step cost of SALT is essentially the same as SFT. Each training step reads the soft label vector and then runs a standard forward and backward pass. The only difference from SFT is the loss. SFT computes cross-entropy against a single observed label, while SALT computes KL divergence over the K options. The extra cost is O(K), negligible relative to the network itself.

Neighbor retrieval and soft-label aggregation do not occur inside training steps. The soft labels are refreshed every 30 steps by one gradient-free inference pass over the training set, and embeddings are precomputed once before training.

Measured costs are reported in Appendix D (Computational Cost). Embedding precomputation is a one-time cost of 40 seconds. Each training epoch takes about 35 minutes for SALT and 31 minutes for SFT on 8 H20 GPUs, an increase of about 13%. We will add this per-step analysis to Appendix D in the revision.


Question 3: A clarification: the grouping, since it happens on x, take into account the persona embeddings and the content embeddings? Therefore, the grouping can happen from similar personas and/or similar content. Empirically, what neighbors are fetched? In your dataset with its high density of annotations and a less dense dataset?. Response: We thank the reviewer for this question.

  1. Embedding composition. Yes, the embedding of x contains both the persona and the question content, so both can in principle shape the neighbor ranking. In SubjSim, however, neighborhood retrieval is restricted to the same action-space group (Eqs. 11–12). A group corresponds to exactly one question. All contexts in a group share the same question text, so the content component is constant within a group. The neighbor ranking is therefore driven by the persona embedding (see Appendix D).

  2. What the fetched neighbors look like. The fetched neighbors are therefore contexts with the same question and the most similar personas. We verified that they are behaviorally meaningful: their response distributions are measurably closer than those of random personas (7.1% lower JSD for the nearest neighbor; details in our response to Reviewer MrP8, Weakness 3).

  3. Annotation density: In a less dense dataset, the radius rule automatically widens the neighborhood, so neighbors are fetched from a broader range of personas. Please see our response to Weakness 2 for the full analysis and the subsampling experiment.

AuthorsOpenReview ↗

Response (3/3)

Question 4: For the formulation in Section 3.1 and the preference of one model over the other for accuracy: What would the case be on average for a perfect underlying model with sampling (iii.k2) compared to a simpler model (iii.k1)? You presented a specific example with a failure case, and theoretically it establishes your point, but in a whole population, empirically and on average, wouldn’t phi_theta_2 (its observed k2s) be better?

Response: We thank the reviewer for this question. The cases in Section 3.1 are meant to show that accuracy can be wrong, not that it is always wrong. For a fixed model θ, its expected accuracy equals the true probability of its top action: E[acc(θ)]=ptop(θ)E[acc(θ)] = p_{top(θ)}. So on average, the ideal model does score highest under accuracy. We agree with the reviewer on this. The problem is that, for a metric meant to measure distributional fidelity, "can be wrong" is already a defect.

First, the failure probability grows with subjectivity. The reversal in case (iii) happens whenever the observed answer misses the mode, with probability 1−pmax1 − p_{max}. And since each context has only one observation, this risk cannot be averaged away by repeated sampling.

Second, even in expectation, accuracy only sees the top action. A perfect model and a point-mass model on the mode have identical expected accuracy (both p_max), while their true errors differ by 1−pmax1 − p_{max}.

So our claim is not that accuracy is always wrong, but that it is neither sensitive nor reliable, and fails more often on more subjective contexts which are exactly where a reliable metric matters most.


Question 5: I had a hard time wrapping my head around definition 2 and theorem 1. Could you provide more intuition? Why is the definition what it is? Why are the terms called bias and statistical error? etc.
Response: We thank the reviewer for this question.

Intuition behind Definition 2. KeffK_{\mathrm{eff}} measures how many actions the probability mass is effectively spread over. It has a calibration property: if the probability is uniform over mm actions, KeffK_{\mathrm{eff}} equals exactly mm. So it serves as the "equivalent number of equally likely actions": 11 for a point mass, KK for the uniform distribution.

Why this form. It was not chosen by us; it is exactly the combination that falls out of the derivation of the statistical error. The aggregated label of action kk averages nn independent Bernoulli observations with mean pkp_k (the neighborhood average), whose noise is pk(1−pk)/n≤pk/n\sqrt{p_k(1-p_k)/n} \le \sqrt{p_k/n}. Summing the per-action errors, as the TV distance does, gives 12∑kpk/n=12Keff/n\frac{1}{2}\sum_k \sqrt{p_k/n} = \frac{1}{2}\sqrt{K_{\mathrm{eff}}/n}, and the combination (∑kpk)2(\sum_k \sqrt{p_k})^2 appearing here is exactly KeffK_{\mathrm{eff}}.

Why "bias" and "statistical error". The names follow the bias–variance decomposition in kernel regression. The statistical error is the random fluctuation from finite sampling, and it vanishes as the neighborhood grows. The bias is the systematic shift from using neighbors whose distribution is ϕ∗(⋅∣x′)\phi^{\ast}(\cdot \mid x') rather than ϕ∗(⋅∣x)\phi^{\ast}(\cdot \mid x). By the Lipschitz assumption it is at most L⋅rL \cdot r, and collecting more neighbors does not remove it. The two terms move in opposite directions in rr, and balancing them gives r∗≍(Keff/(ngL2))1/(dX+2)r^{\ast} \asymp \left(K_{\mathrm{eff}}/(n_g L^2)\right)^{1/(d_X+2)}. Since the statistical error is controlled by KeffK_{\mathrm{eff}}, more subjective contexts call for a larger radius, which is the basis for SALT's subjectivity-adaptive radius.

Reviewer_hkhtOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The task of social simulation is a subjective one, i.e., there are multiple valid outcomes, yet it is usually treated objectively. This paper addresses social simulation from the lens of subjectivity. They first introduce the notion of a subjectivity coefficient. This measures the inherent randomness of a human response in a specific context and is based on entropy. This coefficient is then used to show the mismatch between using a single label to train on and accuracy to measure as a task gets more subjective.

The authors then introduce SALT: subjectivity-adaptive soft-label training, where the answers of semantically similar contexts with the same options are pooled together into a soft label (the human judgment distribution over the options). The neighborhood size is automatically adapted depending on how subjective a context is perceived to be. If a context is seen as objective, SALT defaults to a single label.

As there is no subjective social simulation dataset, the authors introduce SubjSim, a dataset with 100 questions that are annotated by 193 annotators.

Applying their methodology to Qwen3-8B, the authors find that SALT outperforms other alignment methods such as DPO, PPO, and SFT on various distance based metrics, specifically on mid- and high subjective contexts.

Critical strengths
  • I really enjoyed reading this paper as the task it addresses is highly relevant since the field has recognized the importance of taking subjectivity into account. But I also really like the idea behind SALT and I feel it can especially be very helpful in low-resource scenarios, which is especially the case for subjective tasks, where a lot of annotation data is needed but that can be expensive to get. I think SALT is a nice solution to this and I like how it is adaptable on a sample-level, it can also recognize when something is not subjective at all.
  • Furthermore, the paper was well-written and very easy to follow.
Critical weaknesses
  • There is no Related Work in the main body, which makes it difficult to really place the work agains related literature.
  • While interesting, I wonder what new perspective the subjectivity coefficient brings? This has already been (or a variation of it) used before [1] and while I understand the role it plays in SALT, I wonder what the added value is of this coefficient for showing how accuracy is a mismatch for subjective tasks. Or is the formalization the contribution? I am a bit unsure here, maybe putting subjectivity coefficient less as a strong contribution would work better and more as a supporting block for the narrative further.
  • I would have loved to see the transferability of results to other datasets, e.g., Scruples [2] dataset which is a bit of a related subjective task, in a social context determining who is at fault. Though this weighs less heavier than the other two points but I think such an experiment would really showcase the strength of SALT: does this pooling strategy also improve on related tasks.

[1] Richard Alies, Elena Merdjanovska, and Alan Akbik. 2025. Measuring Label Ambiguity in Subjective Tasks using Predictive Uncertainty Estimation. In Proceedings of the 19th Linguistic Annotation Workshop (LAW-XIX-2025), pages 21–34, Vienna, Austria. Association for Computational Linguistics.

[2] Lourie, N., Le Bras, R., & Choi, Y. (2021, May). Scruples: A corpus of community ethical judgments on 32,000 real-life anecdotes. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 35, No. 15, pp. 13470-13479).

Questions for authors
  • What are your thoughts on cross-dataset generalizability of SALT?
  • Do you have an idea for why DPO is so bad in comparison to also the other baselines?
5 comments
AuthorsOpenReview ↗

Response (1/2)

We sincerely thank the reviewer for the positive assessment and encouraging feedback. We are especially glad that the reviewer found SALT helpful for low-resource scenarios and appreciated its adaptability. We address the reviewer's questions below.

Weakness1: There is no Related Work in the main body, which makes it difficult to really place the work agains related literature.

Response: We thank the reviewer for the suggestion. Currently, the related work appears only in Appendix B. We agree that the main body should have a dedicated section to position our work against prior literature. In the revision, we will add a Related Work section to the main body.


Weakness2: While interesting, I wonder what new perspective the subjectivity coefficient brings? This has already been (or a variation of it) used before [1] and while I understand the role it plays in SALT, I wonder what the added value is of this coefficient for showing how accuracy is a mismatch for subjective tasks. Or is the formalization the contribution? I am a bit unsure here, maybe putting subjectivity coefficient less as a strong contribution would work better and more as a supporting block for the narrative further.

Response: We thank the reviewer for the suggestion and for pointing us to [1]. We agree with the reviewer's assessment on positioning. To answer directly: our contribution is not the coefficient itself, but the formalization built around it. We expand on this below.

  1. How our coefficient differs from [1]. Both are entropy-based, but they measure different things for different purposes. [1] measures cross-annotator disagreement on the same samples and uses it to flag ambiguous instances. Our ss is a property of a single decision context: it captures the inherent randomness of behavior in that context. It is also normalized by log K, so tasks with different candidate-set sizes can be compared on the same [0,1] scale.

  2. We also agree that normalized entropy is not a new statistic. What is new is that s serves as one shared variable connecting the diagnosis and the method. On the diagnosis side, Proposition 2 proves an explicit lower bound on the error of single-label training, and this bound grows with s. This turns "accuracy-style evaluation and training fail as subjectivity grows" into a quantifiable statement. On the method side, Keff≥KsK_{eff} ≥ K^s allows SALT's aggregation radius to be derived from subjectivity rather than tuned.

Following the reviewer's suggestion, we will demote the subjectivity coefficient from the list of core contributions to a supporting component in the revision, and bring the diagnosis, SALT, and SubjSim to the foreground. We will also cite [1] and add a comparison in the Human Label Variation part of Appendix B.


Weakness3: I would have loved to see the transferability of results to other datasets, e.g., Scruples [2] dataset which is a bit of a related subjective task, in a social context determining who is at fault. Though this weighs less heavier than the other two points but I think such an experiment would really showcase the strength of SALT: does this pooling strategy also improve on related tasks. [1]

Response: We thank the reviewer for suggesting Scruples Anecdotes. A cross-dataset test is indeed a more direct way to show the generality of SALT, and the suggested dataset fits our framework well. It is a subjective task: judging who is at fault in real-life anecdotes. All anecdotes share a single five-way action space (AUTHOR / OTHER / EVERYBODY / NOBODY / INFO). Each anecdote also comes with a distribution of community votes, which serves directly as the ground truth for evaluation.

Following this suggestion, we conducted experiment. We filter out anecdotes with fewer than 5 votes. Neighborhoods are built from the embeddings of the scenario texts. We train SALT and SFT, the strongest baseline. All metrics are computed against the vote distributions:

MethodKL↓JSD↓TVD↓MMD↓
SFT1.21730.14990.33820.3149
SALT (Ours)0.54800.11710.30380.2135
SALT outperforms SFT on all four metrics (KL reduced by 55.0%, MMD by 32.2%). The trend matches SubjSim. This shows that the gains of the pooling strategy do not depend on our specific dataset.

Two differences between the two experiments are worth noting.

  1. The pooling axis differs. SubjSim pools across personas within the same question, while Scruples has no persona information, so pooling happens across semantically similar anecdotes.
  2. The two datasets differ in what their distributions represent: in SubjSim, a distribution describes a single individual's behavioral tendency, while in Scruples it reflects disagreement across people. That SALT improves over SFT on both suggests that the pooling strategy is robust to different sources of subjectivity. We will include the full setup and results in the revision.
AuthorsOpenReview ↗

Response (2/2)

Question1: What are your thoughts on cross-dataset generalizability of SALT?

Response: We view cross-dataset generalizability as a strength of SALT. The method makes no dataset-specific assumptions. It builds soft labels from single observations, and it pools over semantic neighbors. So it applies to any dataset of this form. The Scruples experiment above directly tests this. The theory also indicates when the strategy should work. It helps whenever nearby contexts induce similar response distributions. For near-deterministic tasks, the derived radius simply shrinks. We agree that broader validation is valuable, and we plan to extend the experiments to more datasets.


Question2: Do you have an idea for why DPO is so bad in comparison to also the other baselines?

Response: We think the core reason is that DPO's preference pairs conflict with subjective behavior. In our setup, the observed option is the positive example. All other K−1 options become negatives. But in subjective contexts, the true distribution is spread out. These "negatives" often a substantial share of the true probability mass. Yet DPO's gradient actively pushes each of them toward zero.

AuthorsOpenReview ↗

Looking forward to feedback on the Responses.

Dear Reviewer:

Hello! As the discussion phase is short, we would like to ask you to review our responses and reevaluate our work based on our clarifications. We are looking forward to your insightful feedback if you have additional comments. Again, we appreciate your valuable time and consideration!

Reviewer_MrP8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the evaluation and optimization of LLM-based social simulation, arguing that accuracy-based evaluation using single responses fails to adequately capture the subjectivity of human behavior. The authors propose Subjectivity-Adaptive Soft-Label Training (SALT), which constructs soft labels by aggregating responses from similar individuals and adaptively adjusts the neighborhood radius based on the estimated effective number of actions. The authors also introduce SUBJSIM, a dataset containing 193 annotators and 100 subjective questions. Experimental results show that SALT outperforms the compared baselines across multiple distributional metrics.

Critical strengths
  1. Relevant research problem. The paper addresses the limitations of accuracy-based evaluation in capturing uncertainty in human behavior.

  2. Theoretically motivated approach. The authors propose a subjectivity-adaptive neighborhood mechanism and provide a corresponding theoretical analysis.

  3. Dataset and empirical results. The paper introduces a dataset containing individual response-propensity distributions, and SALT achieves strong performance across multiple distributional metrics.

Critical weaknesses
  1. Inconsistency between theoretical assumptions and experimental settings.The statistical error analysis in Appendix C.5 assumes that hard labels are independently sampled from latent response distributions. However, Section 5.1 constructs hard labels deterministically by selecting the highest-probability option. These mechanisms are not equivalent, and aggregating modal labels does not necessarily recover the underlying response distributions. Therefore, the applicability of Theorem 1's error bound and the resulting adaptive-radius derivation to the actual experimental setting remains insufficiently justified.

  2. Limited novelty of the adaptive mechanism.SALT's soft-label construction closely resembles conventional nearest-neighbor estimation and local smoothing. The ablation results show that fixed-radius aggregation alone achieves most of the performance gains. Although the adaptive mechanism provides further improvements, its substantive novelty beyond existing local estimation methods requires stronger justification.

  3. Insufficient validation of the persona similarity assumption.SALT assumes that individuals with similar semantic embeddings have similar response distributions. However, similarity in demographic attributes does not necessarily imply similarity in subjective preferences. The paper does not directly examine the relationship between embedding distance and response-distribution distance, leaving a key assumption underlying neighborhood aggregation insufficiently validated.

Questions for authors
  1. Given that the experiments use deterministic modal labels rather than randomly sampled responses, do the statistical error bounds in Lemma 2 and Theorem 1 still apply? Can the authors provide a corresponding proof or controlled experiments?

  2. Figure 12(a) reports topic proportions such as 21.9%, 18.4%, and 15.8%, which do not directly correspond to integer counts out of the stated 100 questions. Could the authors clarify how these proportions were calculated and whether they reflect multi-label categorization or a plotting inconsistency?

  3. Have the authors examined the correlation between persona embedding distances and response-distribution distances to validate the similarity assumption?

4 comments
AuthorsOpenReview ↗

Response (1/3)

We sincerely thank the reviewer for the thoughtful and constructive feedback, which are very inspiring and helpful for improving our work. We address your concerns one by one below.


Weakness1: Inconsistency between theoretical assumptions and experimental settings.The statistical error analysis in Appendix C.5 assumes that hard labels are independently sampled from latent response distributions. However, Section 5.1 constructs hard labels deterministically by selecting the highest-probability option. These mechanisms are not equivalent, and aggregating modal labels does not necessarily recover the underlying response distributions. Therefore, the applicability of Theorem 1's error bound and the resulting adaptive-radius derivation to the actual experimental setting remains insufficiently justified.

Response: We thank the reviewer for raising this point. The statistical error analysis in Theorem 1 assumes that hard labels are sampled from the latent response distributions, whereas Section 5.1 constructs modal labels deterministically; the two are not equivalent. The submitted version did not make this distinction explicit, and we agree it should have been addressed.

To examine its practical impact, we regenerate the training labels by sampling from these distributions, and retrain SALT and the strongest baseline (SFT) under identical data splits and training configurations, together with variants that replace the adaptive radius with a fixed K_eff:

MethodKL↓JSD↓TVD↓MMD↓
SFT0.74890.12670.36130.2667
SALT (Ours)0.28910.07780.26910.1500
Fix K_eff (2)0.30500.08710.27890.1594
Fix K_eff (3)0.30940.08840.28110.1617
Fix K_eff (4)0.30720.08780.27980.1608

Two observations follow.

  1. SALT's performance with sampled labels is almost identical to that with modal labels in the original paper (KL 0.2891 vs. 0.2880, TVD 0.2691 vs. 0.2720).
  2. It is worth noting that SFT improves under sampled labels (KL 1.2871 → 0.7489), since sampled labels themselves carry partial distributional information; even so, SALT maintains a clear advantage on all metrics.
  3. The adaptive radius still outperforms all fixed-K_eff variants, consistent with the theorem's prediction.

We now clarify how the two label constructions relate to Theorem 1.

  1. The statistical part of the bound does not require random labels. The derivation in Appendix C.5 only uses that the soft label averages independent 0-1 observations. With deterministic labels this still holds, but the randomness sits in the draw of contexts rather than in the labels: each answer is a fixed function of its context, so independent context draws again give independent 0-1 observations. The same calculation then gives a bound of the same form.

  2. The two constructions differ in what the soft label converges to. Random labels converge to the average distribution over the neighborhood. Deterministic labels converge to how often each option is the top choice there. These are two endpoints of one family: sampling from the distribution proportional to ϕ1/T\phi^{1/T} gives random labels at T=1T = 1 and deterministic labels as T→0T \to 0. In near-deterministic contexts the two coincide.

  3. The radius rule is unchanged in structure. It balances statistical error, which decreases in rr, against smoothness bias, which increases in rr, and both terms have the same form under either label type. There is one term the theorem does not cover: with deterministic labels, the top-choice frequency differs from the average distribution, and the bound does not control this gap. The experiment above measures how much this gap matters in practice.

In the revision, we will add a discussion of the relationship between the two label constructions to the appendix, together with the comparison experiment reported above.

AuthorsOpenReview ↗

Response (2/3)

Weakness2: Limited novelty of the adaptive mechanism.SALT's soft-label construction closely resembles conventional nearest-neighbor estimation and local smoothing. The ablation results show that fixed-radius aggregation alone achieves most of the performance gains. Although the adaptive mechanism provides further improvements, its substantive novelty beyond existing local estimation methods requires stronger justification.

Response: We thank the reviewer for this comment. The soft-label construction in SALT indeed resembles classical local smoothing (discussed in Appendix B), but our contribution does not lie in this construction itself.

1. SALT addresses a problem absent from the classical framework. In classical local estimation, the neighborhood size is chosen by cross-validation over candidate values. This is impossible in our setting: each context has only one response, and the target distribution is never directly observed, so there is no per-context signal to validate how large each neighborhood should be. Theorem 1 fills this gap by derivation: the optimal neighborhood grows with the subjectivity of the context.

2. "A fixed radius achieves most of the gains" is consistent with the theory, not evidence against adaptivity. The theorem decomposes the error into two parts. The first comes from hard-label training itself: whenever a context is subjective, treating the most frequent answer as the only correct one necessarily incurs error; aggregation essentially removes this part, which accounts for the bulk of the gains. The second depends on neighborhood size, too small is unstable, too large mixes in dissimilar contexts, and the radius adjusts only this part. Hence most gains come from aggregation and the rest from radius adjustment, exactly the error structure the theorem describes.

3. We further ran a stratified experiment to locate where adaptivity helps (SALT's error reduction over fixed-radius variants; positive = SALT better, lower is better for all metrics):

SubjectivityKLJSDTVDMMD
Low+0.0608+0.0161+0.0292+0.0337
Mid+0.0241+0.0071+0.0185+0.0123
High−0.0064−0.0016−0.0146−0.0039

SALT is clearly better in the low- and mid-subjectivity tiers; the high tier is comparable (differences within 0.015), as its gains come almost entirely from aggregation itself. Overall, SALT still outperforms all fixed-radius variants on all four metrics.

In summary, SALT formulates an estimation problem with unobservable target distributions and one observation per context, establishes the theory and radius rule for it, and validates them experimentally.


Weakness3: Insufficient validation of the persona similarity assumption.SALT assumes that individuals with similar semantic embeddings have similar response distributions. However, similarity in demographic attributes does not necessarily imply similarity in subjective preferences. The paper does not directly examine the relationship between embedding distance and response-distribution distance, leaving a key assumption underlying neighborhood aggregation insufficiently validated.

Response: We thank the reviewer for raising this question. Whether embedding-similar personas indeed have similar response distributions is a key assumption underlying neighborhood aggregation, and we have conducted a direct test.

Direct test. For each persona, we find its k nearest neighbors in the embedding space. For each pair of personas, we compute the distance (JSD/TVD) between their response distributions on the same question, averaged over all questions, and compare it with randomly paired personas:

NeighborhoodAvg. JSDJSD reductionAvg. TVDTVD reduction
Random pairs (baseline)0.3825—0.3521—
1 nearest neighbor0.35537.12%0.32627.35%
5 nearest neighbors0.36015.86%0.33135.93%
10 nearest neighbors0.36893.56%0.33933.63%
20 nearest neighbors0.37402.22%0.34422.24%

Two observations follow.

  1. The distribution distance of nearest neighbors is consistently lower than that of random pairs, showing that embedding distance does carry distributional information.
  2. The average distribution distance rises smoothly with neighborhood size and approaches the random baseline — personas closer in embedding space have closer distributions, and this relationship gradually weakens with distance.

Corroborating ablations. Two ablations already present in the paper support the same conclusion. Global-Freq, which ignores persona differences and pools all respondents of the same question, falls far behind SALT on all metrics; and in the Top-N experiment, performance degrades as N grows, showing that farther neighbors introduce larger bias.

AuthorsOpenReview ↗

Response(3/3)

Question1: Given that the experiments use deterministic modal labels rather than randomly sampled responses, do the statistical error bounds in Lemma 2 and Theorem 1 still apply? Can the authors provide a corresponding proof or controlled experiments?

Response: Please see our response to Weakness 1.


Question2: Figure 12(a) reports topic proportions such as 21.9%, 18.4%, and 15.8%, which do not directly correspond to integer counts out of the stated 100 questions. Could the authors clarify how these proportions were calculated and whether they reflect multi-label categorization or a plotting inconsistency?

Response: We thank the reviewer for the careful reading. These proportions come from multi-label topic annotation rather than single-label assignment: a considerable portion of the 100 questions in SubjSim belong to more than one topic domain and are counted toward each of their assigned topics. The denominator of each topic proportion is therefore the total number of topic assignments rather than 100, which is why the proportions do not correspond to integer counts of questions.

Examples from Table 6 in Appendix G.2:

  • "What is your view on genetically modified foods?" — annotated as both Technology & Society and Health & Well-being;
  • "Who should ensure young people acquire skills for good jobs?" — annotated as both Education and Economy & Labor;
  • "Who should protect personal information online?" — annotated as both Technology & Society and Social Issues.

Question3: Have the authors examined the correlation between persona embedding distances and response-distribution distances to validate the similarity assumption?

Response: Please see our response to Weakness 3.

AuthorsOpenReview ↗

Looking forward to feedback on the Responses

Dear Reviewer:

Hello! As the discussion phase is short, we would like to ask you to review our responses and reevaluate our work based on our clarifications. We are looking forward to your insightful feedback if you have additional comments. Again, we appreciate your valuable time and consideration!

Comments on the paper

2 comments

Dear Reviewers,

We sincerely thank all three reviewers for their time and constructive engagement. We are especially encouraged by their recognition of the importance of the problem and the idea behind SALT (Reviewers MrP8, hkht, and dzpY), its strong empirical performance (Reviewer MrP8), and its practical value in low-resource scenarios (Reviewer hkht). In response to the concerns raised during the discussion, we made several clarifications and added supplementary results. Specifically, we made the following revisions and additions:

  • Add a cross-dataset experiment on Scruples, where soft labels are built entirely through cross-context aggregation, together with four additional models. The results now cover 5 models and 2 datasets, and SALT outperforms SFT on all metrics for every model. (Reviewers hkht, dzpY)
  • Add a sampled-label retraining experiment and an analysis of how the two label constructions relate to Theorem 1, showing that SALT's performance is nearly identical under both. (Reviewer MrP8)
  • Add a stratified experiment locating where the adaptive radius helps most (low- and mid-subjectivity contexts), together with a clarified positioning against classical local smoothing. (Reviewer MrP8)
  • Add a direct validation of the persona-similarity assumption: embedding neighbors have measurably closer response distributions than random pairs. (Reviewer MrP8)
  • Add an annotation-density analysis and a subsampling experiment (10%–100% of the training set), showing that SALT's performance is insensitive to annotation density and that SALT with 10% of the data still outperforms SFT trained on full data. (Reviewer dzpY)
  • Restructure the paper: add a Related Work section to the main body, reposition the subjectivity coefficient as a supporting component (Reviewer hkht), and expand the dataset construction details in Section 5.1 (Reviewer dzpY).
  • Move the RL baselines to the appendix and keep the main comparison within the direct estimation family. (Reviewer dzpY)
  • Provide additional clarifications, including the intuition behind Definition 2 and Theorem 1, a per-step complexity analysis, and the multi-label annotation scheme behind Figure 12(a). (Reviewers MrP8, dzpY)

We hope these revisions and additions address the reviewers' concerns and improve the quality and readability of the paper.

Sincerely,
The authors

AuthorsOpenReview ↗

We would appreciate your reevaluation.

Dear Reviewer: Hello! As the discussion phase is about to close, we would like to kindly ask you to review our responses and reevaluate our work based on our clarifications. We are looking forward to your insightful feedback if you have additional comments. Again, we appreciate your valuable time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.