acceptodds
Under review as a conference paper at ICLR 2027

Socio‑Foundation: A Model for Generalizable Individual Behavior Simulation via Hierarchical Capability Distillation

Abstract

Simulating individual behavior requires large language models (LLMs) to preserve persona traits while adapting to dynamic social contexts. However, general-purpose LLMs often flatten distinct personas, while task-specific tuning suffers from fragmentation and generalization. To overcome these challenges, we organize individual simulation into the FONTS Taxonomy, comprising five complementary capability dimensions: persona fidelity (F), outcome realization (O), behavioral naturalness (N), trajectory coherence (T), and social grounding (S). Grounded in this taxonomy, we curate a standardized training corpus of approximately 10 million instances across 14 representative datasets and present Socio-Foundation. Socio-Foundation decouples specialization from integration via a three-stage pipeline: learning task experts via DAPO, consolidating them into capability experts via off-policy distillation, and unifying them via multi-teacher on-policy distillation (MOPD) in the probability space. We also establish IndiEval, consolidating 29 metrics across the FONTS dimensions. Experiments show that Socio-Foundation outperforms its Qwen3-8B base by 11.0 points and approaches frontier models such as GLM-5.2, with ablations and out-of-distribution evaluations further demonstrating the effectiveness and generalization of our model.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On policy distillation, human behavior simulation, LLM

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dFoNOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper examines the tension between persona fidelity and cross-context generalization in LLM-based simulation of individual behaviour. The authors first annotate over forty datasets with respect to their behavioural requirements and cluster them; they then appeal to the New Big Five personality theory to collapse the resulting preliminary clusters into a five-dimensional capability taxonomy (FONTS), on top of which they construct a standardized training corpus of roughly 10 million instances. Methodologically, the paper proposes Socio-Foundation, a three-stage recipe. In Stage 1, each task is trained separately with DAPO to yield mutually isolated LoRA-based task experts, so that behaviourally divergent specializations are allowed to mature independently in an environment free of cross-task interference. In Stage 2, exploiting the many-to-many affiliation between tasks and capabilities, the task experts are consolidated into five capability experts via off-policy forward-KL distillation, which effectively re-indexes knowledge from the task axis onto the capability axis. In Stage 3, the five frozen capability teachers supervise the student on its own self-sampled rollouts through multi-teacher on-policy distillation with weighted forward-KL objectives, ultimately delivering a single unified simulator. For evaluation, the authors construct IndiEval.

Critical strengths
  1. The pipeline design is grounded in clear mechanistic argumentation, and every design choice is supported by a corresponding ablation.

  2. After distillation, the capabilities of the 8B model approach those of general-purpose models. Moreover, the experiments provide interpretable behavioral evidence for the diagnosis that general-purpose models are systematically weak on the N dimension; at the same time, the observation that specialized models collapse to zero on OOD tasks also carries cautionary value.

Critical weaknesses

DeepSeek-V4-Flash simultaneously serves as: (1) the reward model for DAPO; (2) the dialogue counterpart in interactive tasks; (3) the judge for IndiEval; (4) one of the baselines compared in Table 1. The 8B model is optimized with RL against the preferences of DeepSeek-V4-Flash and is then scored by DeepSeek-V4-Flash, since a large number of metrics (CoSER, MirrorBench, HUMANUAL, SOTOPIA, among others) are LLM-judged. In essence, these scores reflect the subjective judgments of DeepSeek-V4-Flash, and this arrangement is therefore unfair to the other general-purpose large models used for comparison.

Questions for authors
  1. DeepSeek-V4-Flash simultaneously serves as the reward model, the interactive opponent, the judge, and one of the baselines.
  2. There is no second judge, human validation, or control for judge bias. What are the specific values of \lambda in Eq. 6 (equal weighting, weighting proportional to capability scale, or tuned values)?
  3. The metrics for N include termination F1, inverse intent overlap, and diversity, which reward stopping when the conversation should stop, whereas the metrics for O (goal, relation, knowledge, benefits, intent coverage, and task score) reward stating everything in full, pushing the interaction forward, and achieving the goal. Why, then, does Table 3 show that the score on the N dimension instead decreases when the O capability expert is removed?
3 comments
AuthorsOpenReview ↗

Response for Reviewer dFoN

We sincerely thank the reviewer for their careful reading and valuable feedback. We appreciate the recognition of our pipeline design and experimental analysis.

Socio-Foundation leads evaluated 8B models under all three judges (W1 & Q1 & Q2)

We re-evaluated the models using two additional judges, Kimi-K2.6 and GLM-5.2. We reused the saved trajectories and auxiliary-model outputs, recomputed LLM-judged metrics with the same benchmark prompts and scoring rules, and retained the original values for automatically computed metrics. The average IndiEval scores and ranking correlations are reported in General Response 1. Socio-Foundation remains the highest-ranked 8B model under all three judges. The complete per-benchmark and per-metric results are in Appendix D.6, Tables 13 (Kimi) and 14 (GLM).

Active teachers receive equal weights that sum to one per sample (Q2)

Thank you for asking about the teacher weights in Section 4.4, Eq. (6). Teachers activated by each sample's FONTS labels receive equal weights summing to one, while inactive teachers receive zero weight; no relative teacher weights are tuned. All active teachers supervise the same student-generated prefix. General Response 1 provides the weight definition and details of loss aggregation across positions and continuations.

Removing O improves termination but reduces overall behavioral naturalness (Q3)

We thank the reviewer for highlighting this interesting observation. We would like to clarify that N measures broader behavioral naturalness, rather than only whether the model ends a conversation at the appropriate time. Of the three N metrics mentioned in the review, only Termination F1 directly evaluates whether a conversation ends appropriately. Inverse intent overlap measures how little an utterance mechanically repeats the supplied intent at the lexical level; higher values indicate less surface overlap and can, to some extent, reflect autonomy and naturalness of expression. First-turn diversity measures unigram Jaccard distance between first-turn responses across samples, assessing whether initial responses are varied rather than templated or homogeneous. Neither metric directly rewards early termination.

N also includes MirrorBench GTEval (style and authenticity), CoSER anthropomorphism, AI-detector human likelihood, and role adherence. It therefore covers expression, interaction, initiative, role behavior, and termination decisions, rather than simply rewarding shorter responses.

We decomposed the N results for the full five-expert configuration and w/o O:

Rebuttal Table 12. N metrics for the full configuration and w/o O.

N metricFullw/o ODifference
Termination F152.5054.55+2.05
MirrorBench GTEval54.153.9-0.20
CoSER anthropomorphism31.628.75-2.85
Human likelihood47.942.65-5.25
First-turn diversity90.280.03-10.17
Inverse intent overlap97.196.39-0.71
Role adherence1001000
N aggregate54.4152.46-1.95

Difference denotes w/o O minus Full.

The local tension identified by the reviewer does exist, but declines in other metrics outweigh the termination gain and reduce the overall N score. Removing O improves Termination F1 by 2.05 points, consistent with reduced pressure to prolong an interaction. However, role adherence is unchanged and every other N metric decreases. These losses outweigh the termination improvement in the aggregate, which falls from 54.41 to 52.46.

A plausible explanation is that O contributes purposeful, context-sensitive interaction, not merely pressure to state everything. Removing it may reduce excessive conversational persistence while making responses more generic, homogeneous, or passive. This interpretation is consistent with the drops in first-turn diversity and human likelihood. CoSER's anthropomorphism rubric explicitly treats "lacks initiative and goals" as unnatural character behavior; initiative and goal-directedness are therefore themselves part of human-like behavior. MirrorBench GTEval likewise assesses similarity to human reference dialogue in style, tone, and authenticity, which can benefit from context-appropriate goal pursuit.

Moreover, not all O metrics reward exhaustive expression. Intent coverage is directly related to expressing needs, but SOTOPIA's goal, relationship, knowledge, and benefits, and AgentSense goal completion, evaluate interaction outcomes rather than response length. Natural human behavior includes both pursuing goals and stopping appropriately. Thus, Rebuttal Table 12 reflects an improvement in one termination-related component after removing O, alongside larger losses in other aspects of naturalness; it does not imply that O and N are inherently opposed.

I thank the authors for their detailed responses to my questions. The results in Table 12 of the rebuttal show that five of N's seven metrics are coupled with O, and that the capability gap between N and O is concentrated primarily in Termination F1. Once the metrics are integrated with equal weights, the weight assigned to the differing capability shrinks further; the disagreement between the two experts therefore appears more likely to be dissolved by "averaging over the next-token distributions" than to be adjudicated. Therefore, i would like to keep my initial assessment.

Thank you for your suggestions and comments. We also find this phenomenon interesting. In fact, addressing potential conflicts among training samples associated with different capabilities is one of the key motivations behind our efforts to build a unified foundation model. By organizing training samples according to their capability requirements, we have mitigated these conflicts to some extent. Nevertheless, at the capability level, there still exists some degree of mutual influence between capabilities, as you have observed. We believe that finding better ways to integrate and coordinate different capabilities is an important direction for future research. We plan to explore more fine-grained, capability-aware integration mechanisms, including architectural approaches such as Mixture-of-Experts (MoE).

Reviewer_aqfwOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Socio-Foundation, an 8B model for generalizable individual behavior simulation. It proposes the FONTS taxonomy, comprising persona fidelity, outcome realization, behavioral naturalness, trajectory coherence, and social grounding. The model is trained through a three-stage pipeline: task-expert reinforcement learning, capability-level off-policy distillation, and multi-teacher on-policy distillation. The authors also introduce IndiEval, an evaluation suite covering 29 metrics across the five dimensions. Socio-Foundation improves the average IndiEval score from 50.7 to 61.7 over Qwen3-8B and shows gains on additional out-of-distribution benchmarks.

Critical strengths
  1. The study covers preference prediction, role-playing, user simulation, and social reasoning. Its broad task coverage, data curation, and evaluation integration are valuable.

  2. The experimental evaluation is extensive and yields strong results. Compared with Qwen3-8B, the model improves the average IndiEval score from 50.7 to 61.7, with gains on 24 of 29 metrics, demonstrating the effectiveness of the training approach.

  3. The capability dimensions are well motivated, with useful pipeline and expert ablations and additional OOD evaluations. Given the competing demands of simulation tasks, the benchmark-level trade-offs are reasonable, and the model achieves a good overall balance. Appendix E also appropriately distinguishes evaluation stability from statistical significance.

  4. The illustrations are charming and thoughtfully designed, reflecting the authors’ care in presenting the work and making the paper engaging to read.

Critical weaknesses
  1. The current ablations do not isolate the benefit of the FONTS grouping itself. Comparisons with random or alternative groupings under matched training budgets would strengthen this claim.

  2. The data statistics and split procedures need clarification. Some counts in Tables 7, 8, and 10 are difficult to reconcile without further explanation of sampling units, replacement, and train–test separation.

  3. Using DeepSeek-V4-Flash for both training rewards and final evaluation raises concerns about evaluator-specific bias. Evaluation with an independent judge or human raters would strengthen the findings.

  4. Key distillation details remain underspecified, including teacher weights, the sampled forward-KL estimator, and the handling of top-64 truncation. Clarifying these details would improve reproducibility.

Questions for authors
  1. Could the authors conduct a small-scale user study (e.g., with at least 12 participants) to further validate the model’s effectiveness, particularly in perceived persona fidelity and behavioral naturalness?

  2. Can the authors provide qualitative examples showing how hierarchical distillation preserves individual differences rather than producing an averaged persona?

  3. What is the total training cost of the three-stage pipeline?

5 comments
AuthorsOpenReview ↗

Response for Reviewer aqfw Part 1

We thank the reviewer for their careful reading and valuable feedback. Besides, we highly appreciate the reviewer's acknowledgment of our efforts on drawing the illustrations. Your acknowledgment is a huge encouragement for us.

FONTS grouping outperforms random grouping across all five dimensions (W1)

We randomize dataset-to-group assignments across five groups while preserving both the number of datasets in each group and the number of groups to which each dataset belongs at the same training budget. Socio-Foundation outperforms random grouping on all five FONTS dimensions, improving the IndiEval average by 2.18 points. The advantage widens to 4.39 points on OOD benchmarks (66.90 versus 62.51), with higher scores on every reported metric across HiToM, Mistakes, AgentSense, UserLM-LiC, and τ-USI. The wider gap on OOD tasks suggests that potential conflicts within randomly formed groups can limit generalization in individual simulation. Taken together, the IndiEval and OOD comparisons support the soundness of the FONTS capability taxonomy and the advantage of its grouping strategy over random assignment.

Please refer to General Response 2, Rebuttal Tables 5 and 6, for the detailed IndiEval and OOD comparisons. Detailed per-metric IndiEval results have also been added to Table 2 in Section 5.3, Pipeline ablations.

Counting units and replacement explain data counts; evaluation cases are held out (W2)

Counting units. Appendix B.3, Table 8 reports retained training records after cleaning, while the Cases column of Appendix D.4, Table 12 reports final evaluation instances. Both count task instances. Table 12 also lists higher-level sampling units: FANToM question sets, CoSER scenes, and complete persona-specific BehaviorChain chains. For example, 100 FANToM question sets expand into 1,468 evaluation cases.

Sampling with replacement. Appendix C.4, Table 9 reports RL input records sampled with replacement from the corresponding training pools. These counts include repeated draws, particularly for smaller sources, and are not counts of unique original samples. OPD likewise samples with replacement exclusively from the training portion.

Evaluation structure and separation. Higher-level sampling preserves related instances and complete task structures. For example, retaining complete BehaviorChain chains enables evaluation of both individual-node accuracy and consistency across sequential decisions. All evaluation cases are selected outside the training portion and are excluded from RL and OPD. Where dataset size and resources permit, we additionally separate data at the higher-level sampling unit. In BehaviorChain, each evaluation chain and all its nodes are excluded from training, avoiding training on some nodes and testing on others from the same chain. Table 12 reports both sampled-unit and expanded case counts.

Socio-Foundation leads evaluated 8B models under all three judges (W3)

We re-evaluated the models with Kimi-K2.6 and GLM-5.2, using saved trajectories and auxiliary-model outputs. LLM-judged metrics were recomputed with the same prompts and scoring rules; automatically computed metrics retain their original values. General Response 1 reports the average scores and ranking correlations, and the complete per-benchmark and per-metric results are in Appendix D.6, Tables 13 (Kimi) and 14 (GLM). Socio-Foundation remains the highest-ranked 8B model under all three judges. General Response 2 provides further CoSER diagnostics. Human evaluation is discussed in Response for Reviewer aqfw Part 2.

AuthorsOpenReview ↗

Response for Reviewer aqfw Part 2

Thank you again for the detailed questions. Continuing our response, this post clarifies the distillation implementation and outlines our human-evaluation plan.

Distillation uses equal active-teacher weights and top-64-plus-tail forward KL (W4)

Teacher weights.

Each sample xx activates a teacher set A(x)A(x) according to its FONTS labels. All active teachers supervise the same student-generated prefix, with

\frac{1}{|A(x)|}, & d\in A(x), \cr 0, & d\notin A(x). \end{cases}$$ Details about Teacher weights please refer to **General Response 1**. ### Top-64 and tail mass. Given a student-generated prefix, we first apply a softmax to each active teacher's full-vocabulary logits, select the 64 tokens with the highest probability under that teacher, and retain their original probabilities. The probability mass of the remaining tokens is merged into a single other category. The student uses the same token set selected by that teacher to partition the vocabulary, with the other category defined as the sum of the student's probabilities assigned to all tokens outside this set. In this way, each teacher corresponds to a pair of normalized 65-way distributions, which preserves both item-wise supervision over the top-64 tokens and the total tail probability mass. [Ref 1, Ref 2] ### Forward KL. At each fixed prefix $h_t$, we explicitly sum over all 65 categories: $$\ell_{d,t}=\sum_{c=1}^{65}\bar q_d(c\mid h_t) \log\frac{\bar q_d(c\mid h_t)}{\bar p_{\theta,d}(c\mid h_t)}.$$ Writing $r_c=\log\frac{\bar p_{\theta,d}}{\bar q_d}$, the per-category log-ratio term is $-r_c$, as in $k_1$. The $k_3$ correction cancels in the complete sum: $$\sum_c\bar q_d(c)(e^{r_c}-1)=\sum_c\bar p_{\theta,d}(c)-\sum_c\bar q_d(c)=0.$$ We therefore compute the forward KL directly on the 65-way aggregated distributions. Relative to the full-vocabulary KL, the approximation arises from merging the tail tokens into a single category. We first aggregate the teacher losses at each generation position according to $\lambda_d(x)$, then average over the valid generation positions of each continuation, and finally average over the continuations within a batch. KL estimator definition follows prior work [Ref 3]. We have accordingly revised our manuscript in **Section 4.4 and Appendix C.2** **Ref 1:** Ma et al. *MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training.* Arxiv preprint 2026 [https://arxiv.org/abs/2606.30406](https://arxiv.org/abs/2606.30406) **Ref2:** Anshumann et al. *Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs.* ACL 2025 [https://aclanthology.org/2025.acl-long.885/](https://aclanthology.org/2025.acl-long.885/) **Ref3:** Wang et al. *Efficient KL Divergence Estimation via Truncated Top-K Integration for Large Language Models.* ACL 2026 [https://aclanthology.org/2026.acl-long.405/](https://aclanthology.org/2026.acl-long.405/) ## A three-stage human-evaluation plan is underway (Q1) We thank the reviewer for this suggestion. We are pursuing a three-stage plan to complement automatic evaluation with human judgments of persona fidelity and behavioral naturalness: 1. **Completed qualitative cases and analysis.** We have developed controlled cases and analyzed profile-dependent behavior, and have included these cases and analyses in **Appendix F.1, Tables 16, 17, and 19**. Details are discussed in **Response for Reviewer aqfw Part 3**. 2. **Recruit participants online.** We plan to recruit participants through an online platform to compare model responses under the same scenarios and persona profiles, with model identities hidden and response order randomized. 3. **Open an anonymous evaluation arena.** We plan to make the cases, evaluation protocol, and collected results available through an anonymous GitHub-hosted website with an arena for broader participation and feedback. This work is underway, and we aim to share initial results during the rebuttal period. Given the limited time, both the number of participants and the scope of the evaluation we can complete will be limited. We will continue recruiting participants and expanding the evaluation beyond the rebuttal period, and include more complete results in an updated version of the manuscript.
AuthorsOpenReview ↗

Response for Reviewer aqfw Part 3

We appreciate the reviewer's further suggestions. This post presents our qualitative and profile-diversity analyses and reports the training cost.

Controlled qualitative examples demonstrate profile-dependent decisions (Q2)

Controlled qualitative comparisons.

We thank the reviewer for the suggestion. To examine whether the shared model after hierarchical distillation can still distinguish between different personas, we fix the scenario and the interlocutor's wording on the same Socio-Foundation checkpoint, vary only the profile of the simulated persona, and compare their behavior when facing the same question. The revised appendix provide the full profiles, the fixed wording, and a three-model comparison at the same turn for all three cases in Appendices F.1.1–F.1.3. We compare Socio-foundation, its strongest ablation without hierarchical distillation Task SFT and Qwen3-8B.

In the earphone after-sales case, both customers prefer an exchange. Faced with the same five-day delivery option, Socio-Foundation has the customer who can wait a week accept the exchange, while the customer who needs the earphones in two days rejects it and insists on having them repaired or replaced before the event. On the same turn, Qwen3-8B has the latter customer accept a refund, whereas Task-expert SFT inquires about the refund procedure. This contrast shows how, after hierarchical distillation, our Socio-Foundation can translate the distinct constraints of two personas sharing the same initial preference into different decisions. We observe this differentiation in two further scenarios as well. In a fractions lesson, students start from their respective prior knowledge and then update their judgments in response to the same instructional explanation. In a bicycle bargaining scenario, one buyer is willing to wait until the next day to inspect the bike in person, while the other chooses same-day pickup conditional on the seller confirming that the brakes are functional. All these cases demonstrate that our Socio-Foundation can preserve individual differences with high fidelity.

Lexical diversity of generated profiles.

Rebuttal Table 10: Lexical diversity of generated profiles.

Model1−ROUGE-L;↑1-\mathrm{ROUGE\text{-}L}\\;\uparrow
Qwen3-8B0.762
Task-expert SFT0.741
Socio-Foundation0.775
DeepSeek-v4.10.838
GPT-5.6-luna0.683

In addition, we also try to provide some quantitative metrics to prove the persona variety. We have each model generate 100 fictional persona profiles from the same open-ended prompt and compute the average pairwise 1−ROUGE-L1-\mathrm{ROUGE\text{-}L} among the profiles generated by the same model. Socio-Foundation scores 0.775, higher than Qwen3-8B's 0.762 and Task-expert SFT's 0.741. This indicates that, on the same profile-generation task, the profiles produced by Socio-Foundation overlap less in vocabulary and word order, and show a stronger persona variety. Also, we surprisingly find that even the frontier GPT-5.6-luna struggles with persona diversity, which has strengthened our claim that these models fail in individual simulations because they are 'overly assistant-like'.

The three-stage pipeline costs approximately 1,840–2,640 H200 GPU-hours (Q3)

Training jobs use 8 NVIDIA H200 GPUs, each with 141 GB of memory. The estimated total GPU cost of the three-stage pipeline is 1,840–2,640 H200 GPU-hours.

Rebuttal Table 11. Three-stage training cost.

StageElapsed job-hours on 8 GPUsH200 GPU-hours
Stage 1: task-expert RL, summed over all jobs200–3001,600–2,400
Stages 2 and 3: capability distillation and final multi-teacher OPD, combinedAbout 30About 240
Total230–3301,840–2,640

Stage 1 is the most expensive. Individual task-expert jobs take approximately 4–30 hours on eight GPUs, depending on interaction length and the number of dialogue turns. The reported 200–300 hours sum the durations of all task-expert jobs. Stages 2 and 3 together take approximately 30 hours on eight GPUs. We also updated Appendix C.5, Table 10 to add these details.

Reviewer_ZAPBOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies individual behavior simulation where general-purpose LLMs tend to flatten distinct personas and task-specific tuning yields fragmented simulators. The authors organize the required capabilities into the five-dimension FONTS taxonomy, curate a large training corpus, and propose Socio-Foundation, trained from Qwen3-8B in three stages: task experts are trained with DAPO, consolidated into five capability experts by off-policy distillation, and unified into one student by multi-teacher on-policy distillation (MOPD). They also introduce IndiEval, which groups existing benchmark metrics by the FONTS dimensions. On IndiEval, Socio-Foundation improves on Qwen3-8B by 11.0 points on average and approaches frontier models such as GLM-5.2. Pipeline ablations and further out-of-distribution evaluations are also reported.

Critical strengths
  • The study is broad and its main result is useful. A single 8B student that consolidates many task experts improves the aggregate IndiEval score of its base model by 11 points, exceeds several open-weight models such as Kimi-K2.6 and DeepSeek-V4-Flash, and comes close to GLM-5.2 (Table 1). The comparison covers frontier, open-weight and specialized models on identical metrics, and shows that some specialized simulators score near zero on decision-oriented persona-fidelity metrics. Table 2 compares the pipeline with seven alternatives on the same metrics.
  • The paper addresses an important problem: one model that simulates a specific individual across persona-conditioned choice, role-play, user simulation and social reasoning. Most prior simulators target only one of these formats. The FONTS taxonomy gives these tasks a shared structure, and the three-stage post-training pipeline is a clear, reusable recipe for consolidating many task-specific adapters. IndiEval keeps each benchmark's native protocol and can serve later work independently of the proposed model.
Critical weaknesses
  • 11 out of the 13 IndiEval benchmarks are in-domain, evaluated on held-out splits of the training datasets, and most of the gain over Qwen3-8B comes from BehaviorChain and UserLM-S3. For example, BehaviorChain node accuracy rises from 34.9 to 86.4 (Table 1), while the two OOD benchmarks within IndiEval, AgentSense and UserLM-LiC, improve only modestly. The additional OOD benchmarks of Fig. 4 (Mistakes, HiToM and τ\tau-USI) are compared only with OSim 8B and Qwen3-8B. OOD results against all baselines are needed to support the generalization claim in the title.

  • The ablations do not establish that the intermediate stage of capability experts is needed. The strongest alternative without this stage, Task-expert SFT, reaches 59.5 against 61.7 for the full pipeline (Table 2), a small margin. The larger gaps to Task-expert averaging and Task-expert OPD may reflect combining all task experts at once rather than the FONTS grouping. Therefore a control that assigns the same task experts to random groups at matched compute would show whether the FONTS grouping is responsible for the improvement.

Questions for authors
  • which objective, number of final-stage steps and teacher weights λd\lambda_d produced the Socio-Foundation row of Table 1, and was the student a LoRA adapter or fully fine-tuned? How was the reported checkpoint selected (assuming training ran beyond 200 steps)?
  • can you evaluate all Table 1 baselines on Mistakes, HiToM and τ\tau-USI?
  • how does the full pipeline compare with a single random grouping of the same task experts into five groups at matched compute? This would show whether the gain comes from the FONTS grouping or from distilling from fewer teachers.
  • how do the judge-based results change when a sample of outputs is re-scored by a judge from a different model family, or by human raters?
3 comments
AuthorsOpenReview ↗

Response for Reviewer ZAPB Part 1

We sincerely thank the reviewer for their careful reading and constructive feedback. We have expanded the OOD comparisons and clarify the resulting evidence below. Training details and alternative-judge results are also provided.

Socio-Foundation achieves the highest OOD average among evaluated 8B models (W1 & Q2)

The following table reports results on five OOD benchmarks for the listed general-purpose models, 8B specialized simulators, and our base model. All metrics are higher-is-better.

Rebuttal Table 7. OOD results across models.

Model / VariantHiToM ↑Mistakes ↑AgentSense Goal (Maj.) ↑AgentSense Private Acc. ↑LiC Intent Coverage ↑LiC Task Score ↑τ-USI USI ↑Avg ↑
GPT-5.585.0073.0096.4778.0698.0996.6167.2681.97
Kimi-K2.679.0063.0095.2686.8293.4862.8259.4274.12
GLM-5.280.0057.0096.3482.4497.2679.2855.3874.01
DeepSeek-V4-Flash80.0045.0095.4275.7693.5755.5165.3070.08
Socio-Foundation62.0057.0093.5283.7476.7837.9269.5066.90
Qwen3-8B60.0052.0092.3182.7468.3830.7466.0263.02
OdysSim-8B54.0044.0087.8782.9080.3539.2663.6661.37
HumanLM-8B49.0037.0085.3577.0379.4337.4663.4057.81
CoSER-8B3.000.0060.3261.0011.842.6956.3025.45
USP-8B17.000.0012.5711.729.195.5548.3716.98
UserLM-8B0.0027.002.840.996.980.1150.7816.65

Socio-Foundation achieves the highest OOD average among the evaluated 8B models, scoring 66.90 and outperforming both the Qwen3-8B base model (63.02) and the strongest specialized 8B baseline, OdysSim-8B (61.37). It also leads this comparison on tau-USI (69.50).

AgentSense: gains from an already strong baseline.

Qwen3-8B already achieves 92.31 on Goal Achievement (Majority Vote) and 82.74 on Private Information Accuracy. Socio-Foundation improves these to 93.52 and 83.74. For context, the best scores among the listed frontier models are 96.47 and 86.82, respectively. Thus, even much larger models offer relatively modest gains over this baseline, especially on goal achievement, which is already close to the upper end of the scale. The modest absolute improvements should be interpreted in this context. Socio-Foundation also exceeds the evaluated specialized 8B models on both metrics, including OdysSim-8B (87.87 and 82.90) and HumanLM-8B (85.35 and 77.03).

UserLM-LiC: task understanding and multi-turn reliability.

This benchmark requires a simulated user and an assistant to jointly complete mathematics and coding tasks over multiple turns. Success depends on more than expressing the user's intent: the user model must understand the task, interpret the assistant's replies, and respond appropriately when the assistant makes a mistake or provides an unclear explanation. Limited mathematical and coding capabilities may make an 8B user model less able to identify and correct such problems, or more likely to follow an incorrect direction suggested by the assistant.

Socio-Foundation improves over Qwen3-8B on both intent coverage (76.78 versus 68.38) and task score (37.92 versus 30.74). The larger general-purpose models perform substantially better on LiC, consistent with a possible contribution from stronger underlying task capabilities.

A second limitation is output reliability over multiple turns. The following table reports dialogues assigned zero scores because the target model's output failed to meet the required format:

Rebuttal Table 8. Dialogues scoring zero due to output failure.

ModelDialogues scoring zero due to output failureRate
Socio-Foundation21021.00%
OSim 8B26126.10%
HumanLM 8B20220.20%
Qwen3-8B23223.20%
Qwen3-235B404.00%
DeepSeek20.20%
GLM-5.220.20%
GPT-5.500.00%

Output failures affect roughly one fifth to one quarter of dialogues for the listed 8B models, including 21.00% for Socio-Foundation, compared with 4.00% for Qwen3-235B and 0–0.20% for the other listed large models. Socio-Foundation reduces this failure rate relative to Qwen3-8B and OSim 8B, but the remaining failures still depress its aggregate score. Together, these results suggest that LiC exposes limitations in both task capabilities and sustained output reliability, alongside user-simulation ability.

AuthorsOpenReview ↗

Response for Reviewer ZAPB Part 2

Thank you again for the constructive feedback. Continuing our response, this post examines the contribution of the full training pipeline and the FONTS grouping through additional ablation comparisons.

The full pipeline outperforms Task-expert SFT and other ablations on average across OOD benchmarks (W2 & Q3)

Rebuttal Table 9. OOD results for pipeline ablations.

Model / VariantHiToM ↑Mistakes ↑AgentSense Goal (Maj.) ↑AgentSense Private Acc. ↑LiC Intent Coverage ↑LiC Task Score ↑τ-USI USI ↑Avg ↑
Socio-Foundation62.0057.0093.5283.7476.7837.9269.5066.90
Task-expert SFT60.0057.0091.5682.2670.7632.4768.5164.81
Task-expert OPD55.0053.0092.8581.8674.8535.3062.1362.51
Task-expert averaging53.0054.0091.2279.7268.8632.6567.6562.17
Capability-expert averaging61.0055.0091.8378.4675.4037.1471.4765.78
Capability-expert off-policy distill.58.0058.0091.3178.8860.9724.9761.8661.19
Pooled-data SFT55.0038.004.5128.8431.7624.6833.4934.28

The full pipeline achieves the highest OOD average among all listed ablations and outperforms Task-expert SFT, with an average score of 66.90 versus 64.81 (+2.09 points). The in-domain average also improves from 59.5 to 61.7.

Socio-Foundation also leads the listed ablations on the great majority of OOD metrics (5 of 7), achieving the best results on HiToM, both AgentSense metrics, and both LiC metrics.

For the FONTS grouping comparison (W2), General Response 2 presents the detailed experimental results. Socio-Foundation improves the IndiEval average over random grouping by 2.18 points.

The advantage further increases to 4.39 points on OOD benchmarks (66.90 versus 62.51), with higher scores on every reported metric across HiToM, Mistakes, AgentSense, UserLM-LiC, and τ-USI. The larger OOD gap suggests that conflicts arising from random assignments may impair transfer to unseen individual-simulation tasks. These findings reinforce the rationale for defining and organizing capabilities through FONTS and show the benefit of this structure over random grouping. Please refer to General Response 2, Rebuttal Tables 5 and 6, for the full IndiEval and OOD comparisons.

AuthorsOpenReview ↗

Response for Reviewer ZAPB Part 3

Thank you again for the detailed questions. This post clarifies the training configuration behind Table 1 and addresses the evaluation-judge concern.

Detailed setting declaration. (Q1)

We thank the reviewer for their careful reading and for raising this question. We provide the following clarifications.

  • In practice, the Socio-Foundation results reported in Table 1 come from the checkpoint at step=200\mathrm{step}=200 of the final MOPD stage. In this stage, we use the general forward-KL MOPD loss, namely

    Eh∼μθˉ∑d∈CλdDKL(qd(⋅∣h),∣,πθ(⋅∣h)),\mathbb{E}_{h\sim\mu_{\bar{\theta}}}\sum_{d\in\mathcal{C}}\lambda_d D_{\mathrm{KL}}\big(q_d(\cdot\mid h)\\,\\|\\,\pi_\theta(\cdot\mid h)\big),

    Compared with reverse KL, this loss is mean-seeking, and on the sampled data it encourages the student to cover the full support of the teacher. Although reverse KL is commonly used in general domains since the mean-seeking mode is believed to induce hallucination and low-quality generation. However such concerns are much less pronounced in the persona simulation domain. Instead, we consider it a better strategy to jointly leverage the guidance of experts across different dimensions and to generate more diverse responses. In reverse-KL MOPD, each sample is assigned to the expert that supervises it. In the persona simulation domain, however, a sample does not contain supervision from only a single FONTS dimension, and we therefore adhere to forward KL.

  • The final student model likewise uses a LoRA adapter rather than full fine-tuning as shown in our Figure 2(b), stage (3).

  • In practice, we observed that the loss had already converged around steps 150–200, so we did not train further; after comparing the performance at step=150\mathrm{step}=150 and step=200\mathrm{step}=200, we selected the step-200 checkpoint.

Sample-dependent teacher weights. Teachers activated by a sample's FONTS labels receive equal weights summing to one; inactive teachers receive zero weight. General Response 1 provides the weight definition and details of loss aggregation across positions and continuations.

Socio-Foundation remains the strongest evaluated 8B model across three judges (Q4)

We re-evaluated the models using Kimi-K2.6 and GLM-5.2 as alternative judges. We reused saved model trajectories and auxiliary-model outputs, recomputed LLM-judged metrics with the same benchmark prompts and scoring rules, and retained the original values for automatically computed metrics.

General Response 1 reports the average IndiEval scores and ranking correlations across all 13 models. Socio-Foundation scores 51.45 with Kimi and 52.44 with GLM, compared with 41.50 and 41.89 for Qwen3-8B, respectively. It remains the highest-ranked 8B model under all three judges. Pairwise Spearman correlations between the overall model rankings are 0.962, 0.978, and 0.989, supporting the robustness of the principal comparative conclusions to judge choice.

The complete per-benchmark and per-metric results are in Appendix D.6, Tables 13 (Kimi) and 14 (GLM). General Response 2 provides the CoSER zero-score and tie-rate diagnostics, which help explain why absolute scores can differ substantially between judges under the same protocol.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response 2

2. Evaluation robustness across different judges (continued)

DeepSeek provides finer differentiation between models under CoSER's evaluation protocol while balancing quality and cost. The protocol converts judge-identified flaws and severity ratings into length-corrected scores on a 0–100 scale. With identical trajectories, prompts, and scoring rules, Kimi and GLM produce higher cumulative severity assessments and more scores at the lower bound than DeepSeek.

Rebuttal Table 3 reports the zero-score rates among all valid trajectories across the 13 models for each CoSER metric. All reported scores come from successfully parsed assessments. The DeepSeek results in Rebuttal Tables 3 and 4 come from the additional re-evaluation with DeepSeek-V4-Flash. Compared with DeepSeek, Kimi shows a particularly pronounced concentration of zero scores, raising concerns about its ability to distinguish model performance under this scoring protocol; GLM also has substantially elevated zero-score rates.

Rebuttal Table 3. Zero-score rates under the CoSER scoring protocol.

CoSER metricDeepSeekKimiGLM
Character fidelity24.8%99.5%85.9%
Anthropomorphism13.1%99.2%79.5%
Storyline consistency0.9%94.6%69.2%
Storyline quality2.3%95.8%55.8%

We then paired the 13 models with one another within each scene and compared the scores assigned by each judge. For each metric, we included all valid model-pair comparisons. Rebuttal Table 4 shows how often the two models in a pair received exactly the same score on the same scene. A higher tie rate means the judge more often assigns identical scores to different models, making their performance harder to distinguish.

Rebuttal Table 4. Same-scene pairwise tie rates under the identical CoSER protocol.

CoSER metricDeepSeekKimiGLM
Character fidelity9.4%98.9%76.0%
Anthropomorphism2.9%98.4%65.2%
Storyline consistency1.6%89.7%52.5%
Storyline quality1.2%91.9%33.0%

The concentration at zero is accompanied by substantially higher tie rates, which substantially reduces score discrimination. Across all four metrics, Kimi and GLM distinguish between models less frequently than DeepSeek, with the loss of discrimination especially pronounced for Kimi. For example, GLM assigns both Socio-Foundation and DeepSeek-V4-Flash a character-fidelity score of zero in 176 of their 200 shared scenes.

Qualitative inspection also revealed concrete assessment issues. In one case, Kimi counted explicitly labeled environment narration as a character-language flaw, although the character-fidelity rubric restricted evaluation to the main characters. In a reference-dialogue control using a pre-race scene from Seabiscuit, we evaluated the same supplied reference conversation three times with each judge. DeepSeek and Kimi both assigned a character-fidelity score of 100/100 in all three runs, whereas GLM assigned 62, 67, and 82.

The complete Kimi and GLM per-benchmark and per-metric results are in Appendix D.6, Tables 13 (Kimi) and 14 (GLM).

3. Evaluating the contribution of FONTS grouping

We thank the reviewers for identifying this gap. We randomize dataset-to-group assignments across five groups while preserving both the number of datasets in each group and the number of groups to which each dataset belongs. Scores are on a 0–100 scale; the five dimensions are weighted equally.

Rebuttal Table 5. FONTS versus random grouping on IndiEval.

ModelFONTSAvg.
Random Grouping MOPD60.8162.4052.0548.6573.6759.52
Socio-Foundation (Full Pipeline)63.9363.8554.4150.5075.7761.69

Socio-Foundation outperforms Random Grouping MOPD on all five dimensions, with an overall gain of 2.18 points.

Detailed per-metric IndiEval results are in Table 2 in Section 5.3, Pipeline ablations. We additionally compare the two models on five OOD benchmarks:

Rebuttal Table 6. FONTS versus random grouping on OOD benchmarks.

ModelHiToMMistakesAgentSense Goal (Maj.)AgentSense Private Acc.LiC Intent CoverageLiC Task Scoreτ-USI USIAvg.
Random Grouping MOPD59.0053.0093.1377.4769.4130.4265.3062.51
Socio-Foundation (Full Pipeline)62.0057.0093.5283.7476.7837.9269.5066.90

The gap widens on OOD benchmarks: Socio-Foundation scores 66.90 versus 62.51 (+4.39 points), exceeding its 2.18-point advantage on IndiEval. These results suggest that potential conflicts introduced by random grouping may weaken generalization in individual-simulation tasks. They substantiate the rationale behind the FONTS capability definitions and demonstrate the superiority of its grouping mechanism over random grouping.

AuthorsOpenReview ↗

General Response 1

We sincerely thank all reviewers for their careful reading and constructive feedback. We appreciate their recognition of the clear problem formulation, broad task coverage, pipeline ablations, and comprehensive experiments. Below, we address three shared concerns: (1) underspecified distillation details; (2) the use of DeepSeek-V4-Flash for both training rewards and final evaluation; and (3) the benefit of the FONTS grouping itself. We address each reviewer's remaining points in the corresponding individual response.

1. Clarifying teacher weights in multi-teacher distillation

We clarify the teacher weights in Section 4.4, Eq. (6). We retain the weighted sum of per-teacher distillation losses used in multi-teacher on-policy distillation. While this formulation allows different weighting schemes, our implementation uses equal weights for the active teachers; exploring more sophisticated weighting strategies is left to future work.

For each training sample xx, we determine the active teacher set A(x)A(x) from its dataset's FONTS labels. All active teachers supervise the same student-generated prefix. The weights are defined as

\begin{cases} \frac{1}{|A(x)|}, & d \in A(x), \cr 0, & d \notin A(x). \end{cases}$$ For example, a sample that activates F, O, and N assigns a weight of $1/3$ to each of these three teachers. The weights therefore sum to one for each sample, without requiring additional tuning of relative teacher weights. We write them explicitly as $\lambda_d(x)$ in Eq. (6) to show that they depend on sample-wise routing. At each generated position, we sum the active teachers' distillation losses using these weights, then average over valid positions within each continuation and finally over continuations in the batch. We have updated Section 4.4 of the manuscript to clarify this implementation and improve reproducibility. ## 2. Evaluation robustness across different judges We thank the reviewer for highlighting the importance of judge choice and the robustness of our evaluation conclusions. We originally selected DeepSeek as the best practical balance between evaluation quality and computational cost for our evaluation budget. To address this concern within the available time and resource budget, **we conducted an additional re-evaluation with Kimi and GLM as alternative judges, using the saved model trajectories and auxiliary-model outputs.** We recomputed the LLM-judged metrics under the same benchmark prompts and scoring rules, while retaining the main-table values for metrics computed automatically without an LLM judge. **Socio-Foundation continues to demonstrate strong overall performance under both alternative judges.** It outperforms all other 8B specialized simulation models and the Qwen3-8B base model, as well as several substantially larger general-purpose models, including Qwen3-Next-80B, Qwen3-235B-A22B, and DeepSeek-V4-Flash. Rebuttal Table 1 reports the average IndiEval score, calculated by averaging metrics within each benchmark and dimension, averaging benchmarks equally within each dimension, and then weighting the five dimensions equally. Detailed per-benchmark and per-metric results are in **Appendix D.6, Tables 13 (Kimi) and 14 (GLM)**. This comparison is now included in **Section 5.2, Main results, Table 3**. **Rebuttal Table 1. Average IndiEval scores under different judges.** | Model | DeepSeek judge | Kimi judge | GLM judge | |---|---:|---:|---:| | GPT-5.5 | 67.2 | 56.17 | 58.61 | | DeepSeek-V4-Flash | 59.7 | 46.93 | 48.89 | | GLM-5.2 | 63.2 | 51.99 | 54.23 | | Kimi-K2.6 | 61.2 | 52.10 | 52.82 | | Qwen3-Next-80B | 50.9 | 43.96 | 43.52 | | Qwen3-235B-A22B | 55.6 | 47.62 | 47.88 | | OSim 8B | 55.4 | 48.51 | 49.54 | | CoSER 8B | 38.2 | 28.52 | 30.22 | | HumanLM 8B | 48.6 | 40.32 | 39.95 | | UserLM 8B | 21.5 | 12.32 | 15.68 | | USP 8B | 19.3 | 12.17 | 14.65 | | Qwen3-8B | 50.7 | 41.50 | 41.89 | | **Socio-Foundation** | **61.7** | **51.45** | **52.44** | The three judges are DeepSeek-V4-Flash, Kimi-K2.6, and GLM-5.2. We further assessed whether changing the judge alters the overall ordering of the evaluated models. Rebuttal Table 2 shows the pairwise Spearman correlations between the rankings of all 13 models in Rebuttal Table 1. All correlations exceed 0.96. Socio-Foundation remains the highest-ranked 8B model under every judge, and the ordering of all seven 8B models is identical across the three columns. Together, the high ranking agreement and Socio-Foundation's consistent advantage over all other evaluated 8B models and several larger general-purpose models support the robustness of our overall comparative conclusions to judge choice. **Rebuttal Table 2. Agreement between overall model rankings under different judges.** | Judge pair | Spearman rank correlation | |---|---:| | DeepSeek–Kimi | 0.962 | | DeepSeek–GLM | 0.978 | | Kimi–GLM | 0.989 | *Continued in General Response 2: CoSER judge diagnostics and the FONTS grouping ablation.*
AuthorsOpenReview ↗

General response to the reviewers and Area Chair

We sincerely thank the reviewers and the Area Chair for their time and thoughtful feedback. We appreciate the reviewers’ recognition of the importance of individual behavior simulation, the breadth of the evaluation, our clear problem formulation, and comprehensive experiments.

During the rebuttal, we added the following experiments and clarifications.

  • Evaluation with independent judges (ZAPB Q4, aqfw W3, dFoN W1, Q1–Q2). We rescored the same saved outputs with Kimi-K2.6 and GLM-5.2. Socio-Foundation ranked first among the evaluated 8B models under all three judges. And different judges can reach a high correlation score.

  • Matched random-grouping ablation (ZAPB W2, Q3, aqfw W1). At the same training budget, FONTS grouping improved over the tested random grouping by 2.18 points on IndiEval and 4.39 points on the OOD average.

  • Expanded OOD comparisons (ZAPB W1–W2, Q2–Q3). We evaluated additional baseline models and pipeline ablations on the OOD benchmarks. The full pipeline scored 66.90 on average, compared with 64.81 for the strongest baseline, Task-expert SFT.

  • Final-stage training details (ZAPB Q1, aqfw W4, dFoN Q2). We clarified the choice of the step-200 LoRA checkpoint, the equal weights assigned to each sample’s active teachers, and the forward-KL computation over each teacher’s top 64 tokens and remaining probability mass.

  • Data accounting and evaluation splits (aqfw W2). We clarified the units counted in the data tables, when training samples were drawn with replacement, and how evaluation examples were held out from training.

  • Individual differences, persona diversity, and anonymous human evaluation (aqfw Q2). We added three controlled dialogue cases in the appendix that vary the persona while keeping the interlocutor script fixed, and measured lexical diversity across 100 generated profiles per model. Furthermore, we launched an anonymous online arena for human evaluation and collected feedback from 20+ participants on 10 cases involving four models to assess our model’s persona fidelity and behavioral naturalness.

  • Training cost (aqfw Q3). We reported a stage-wise breakdown and estimated the total cost at 1,840–2,640 H200 GPU-hours.

  • Behavioral naturalness after removing the outcome teacher (dFoN Q3). We reported the component scores showing that termination F1 improves while the aggregate naturalness score declines.

Taken together, these additions strengthen the evidence for Socio-Foundation’s effectiveness in individual behavior simulation. The consistent results across independent judges support both the robustness of the IndiEval evaluation framework and the reliability of the reported gains, while the matched grouping ablation and expanded OOD comparisons further substantiate the contribution of FONTS grouping and the generalization of the full pipeline. The persona analyses and human feedback provide complementary evidence of behavioral fidelity and behavioral naturalness, and the additional training and data details improve transparency and reproducibility.

Thank you again for the constructive discussion. Your suggestions will greatly help strengthen our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.