acceptodds
Under review as a conference paper at ICLR 2027

Learning to Simulate Individuals from Macro Social Signals

Abstract

Large language models are increasingly used to simulate how individuals respond to new situations, yet the behavioral reasoning behind these responses is either inherited from pretraining or learned from individual-level annotations, which offer limited behavioral diversity and little supervision of the reasoning itself. We propose to learn behavioral reasoning from prediction markets, whose price trajectories record how populations respond to real-world events at scale. We introduce macro2mind, which trains a language model with GRPO using market signals. A social behavioral decomposition makes behavioral reasoning an explicit step of forecasting: the model infers representative groups of market participants, predicts how each interprets the news and updates its beliefs, reasons about their interactions, and aggregates these responses into a price. A hindsight-regret curriculum then focuses training on transitions where representative groups identified in hindsight most improve the forecast, downweighting those dominated by noise or already mastered. The learned reasoning applies to user simulation without further training. On SWM-Bench, macro2mind achieves state-of-the-art directional accuracy and correlation on Polymarket. Trained on market data, it transfers zero-shot to four user-simulation benchmarks (Humanual, OvertonBench, PRISM, and CAD) and has leading performance among zero-shot methods. Used as a data generator, macro2mind also raises a downstream simulator's accuracy on unseen users by 15.5 points, outperforming data generated by its backbone by 13.2 points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, social reasoning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_PYx1OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes macro2mind, a framework that learns transferable behavioral reasoning from aggregate prediction-market signals rather than individual-level behavioral annotations. The method combines social behavioral decomposition, which explicitly models the belief updates and interactions of representative market participants, with a hindsight-regret curriculum that prioritizes informative market transitions during GRPO training.

Critical strengths
  1. The paper explores an interesting and relatively novel direction: learning individual behavioral simulation capabilities from aggregate social signals. This potentially reduces reliance on expensive individual-level behavioral annotations.

  2. The evaluation covers multiple downstream benchmarks and includes a synthetic data generation experiment, providing relatively broad empirical evidence.

  3. The ablation studies suggest that the hindsight-regret curriculum plays an important role in transferring forecasting capabilities to user simulation.

Critical weaknesses
  1. The central claim that macro-level market supervision teaches transferable individual behavioral reasoning is not sufficiently supported. The training reward only evaluates the final market forecast, without verifying whether the inferred constituent groups, belief updates, or social interactions correspond to actual human behavior. Moreover, the SFT warm-start uses teacher-generated reasoning traces conditioned on the realized market outcomes, potentially introducing post-hoc rationalizations. Consequently, the observed improvements could reflect better event understanding or structured reasoning rather than genuine behavioral reasoning. More direct evidence is needed to establish the claimed mechanism.

  2. The contribution of the hindsight-regret curriculum is not sufficiently isolated from generic difficulty-based sampling. Since the proposed sampling weights combine hindsight regret with a difficulty filter, it remains unclear whether the improvements arise from identifying behaviorally informative transitions or simply selecting examples that provide more effective GRPO training signals. A controlled comparison against difficulty-only sampling is necessary. Furthermore, the transfer improvements are not consistent across benchmarks: the gains on PRISM are relatively small, and the main component ablations are conducted primarily on Humanual. This limits the evidence for a generalizable behavioral reasoning capability.

Questions for authors
  1. How can the authors distinguish genuine behavioral reasoning from general event understanding or structured reasoning? Is there any independent evidence that the inferred constituent behaviors and belief updates reflect actual human behavior?

  2. Can the authors compare hindsight-regret sampling against difficulty-only sampling under the same training budget and report the results across all four downstream benchmarks?

4 comments

We sincerely thank the reviewer for careful reading and constructive feedback. Below, we address each concern and question in detail. We will incorporate the relevant clarifications and additional analyses and experimental results into the revised manuscript. We would be happy to provide further clarification during the discussion period.

[Distinguishing Behavioral Reasoning from General Event Understanding] (1) Our evidence comes from zero-shot transfer to four downstream benchmarks that directly evaluate the ability to predict individual human behavior rather than event forecasting. Specifically, these benchmarks (Humanual, OvertonBench, PRISM, and CAD) require modeling individual perspectives, reactions, and preferences. The consistent zero-shot transfer improvements across these distinct tasks suggest that the learned capabilities extend beyond general event understanding. (2) Also, our ablation shows that simply imposing a structured reasoning format is insufficient to explain the transfer gains, with the strongest improvements emerging when behavioral reasoning is combined with curriculum-based training.

Reasoning StructureCurriculum data selectionHumanual State ↑Humanual Resp. ↑CAD Acc ↑Overton MAE ↓Overton Spearman ↑PRISM MAE ↓PRISM Pairwise Acc ↑
Free-form ReasoningNo6.11.50.36600.62300.647023.0000.5390
Free-form ReasoningYes5.51.30.35700.61960.647221.8200.6203
Behavior ReasoningNo6.30.90.31850.68460.625521.4780.6284
Behavior ReasoningYes9.53.40.42200.63500.653021.2000.6330

[Independent Evidence of Actual Human behavior: Human Evaluation] We also conduct human evaluations (Section 6, Figure 5), where five annotators assess the plausibility of inferred constituent behaviors, event interpretations, and belief updates. macro2mind achieves 72.2, outperforming Qwen3-4B (47.1) and Qwen3.5-397B (51.4) under the same reasoning structure. This provides independent human-judged evidence that our model learns more plausible behavioral reasoning rather than merely following a structured format.

[Independent Evidence of Actual Human Behavior: Synthetic Data] On Twin-2K-500, training with macro2mind-generated data achieves 60.0% accuracy on unseen users, compared to 46.8% with Qwen3-4B-generated data. This provides further evidence that the learned capabilities can generate useful supervision for predicting real human decisions in behavioral economics and consumer preferences, as evaluated against held-out responses from actual participants.

[Comparison of Hindsight-Regret and Difficulty-Based Sampling] We compare hindsight-regret and difficulty-only sampling with the same GRPO optimization settings and further evaluate different sampling weights across all four downstream benchmarks. Both sampling strategies improve performance on different metrics, while combining them achieves the best results across all four benchmarks, with our default weighting (0.4 regret / 0.6 difficulty) performing best on all seven evaluation metrics.

Weight of Regret TermWeight of Difficulty TermHumanual State ↑Humanual Resp. ↑CAD Acc ↑Overton MAE ↓Overton Spearman ↑PRISM MAE ↓PRISM Pairwise Acc ↑
017.52.00.37700.65760.630021.9530.6262
0.20.88.32.30.38650.70250.614921.6460.6263
0.4 (Ours)0.6 (Ours)9.53.40.42200.63500.653021.2000.6330
0.80.27.11.80.40550.71250.590022.3300.6194
106.41.60.40500.72690.636522.7380.6222

[More analysis on the different roles of hindsight-regret sampling against difficulty sampling] These terms serve complementary purposes: the regret term evaluates whether a transition is informative, while the difficulty term determines whether it is currently learnable.

Specifically, the regret term prioritizes transitions whose outcomes hinge on critical constituents that the policy currently overlooks. The difficulty term accounts for the optimization dynamics of GRPO. Because GRPO learns from the relative differences among M responses to the same transition, the advantage signals are strongest when a group contains a mix of successes and failures. This variance peaks at intermediate success rates and vanishes when the policy consistently succeeds or fails, regardless of the regret magnitude.

[Ablation on SFT Initialization and Post-hoc Rationalization] We use SFT on SWM-Bench to initialize the behavioral reasoning structure, without any downstream individual-level training data. Subsequent GRPO training generates reasoning without access to realized market outcomes in its rollout prompts.

We further compare different SFT initializations across four downstream benchmarks. Our full pipeline achieves the best performance across all seven metrics. Importantly, outcome-blind SFT with the same GRPO curriculum also achieves strong transfer performance, suggesting that the gains do not solely depend on outcome-conditioned teacher traces. We believe access to realized outcomes helps the teacher generate more informative, outcome-consistent behavioral reasoning traces, providing a stronger initialization for subsequent RL.

SFT InitializationGRPOCurriculumHumanual State ↑Humanual Resp. ↑CAD Acc ↑Overton MAE ↓Overton Spearman ↑PRISM MAE ↓PRISM Pairwise Acc ↑
Outcome-conditioned BR-SFTNoNo7.51.40.33350.69580.598422.5920.6248
Outcome-conditioned BR-SFTYesFull9.53.40.42200.63500.653021.2000.6330
Outcome-blind BR-SFTYesFull8.12.80.41100.65960.634621.6110.6262
Without SFTYesFull7.01.60.39150.64580.634721.9160.6185

Thank you for the review. We truly appreciate the time and expertise you dedicated to it. As the discussion period draws to a close, we would appreciate it if you could take a moment to review our responses. If you have any further feedback or additional questions on any point, we would be happy to discuss it.

If our responses have addressed your concerns and questions, we kindly request a reconsideration of the merit of our work and an adjustment to the overall assessment score. Thanks again for your time!

Thank you for the new ablations. They address my sampling concern and ease my concern about post-hoc rationalization. I encourage the authors to temper their claims about behavioral reasoning pending direct validation against actual market participant behavior. I keep my original rating.

Reviewer_eKjUOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes macro2mind, a framework that learns to simulate individual human behavior using aggregate prediction-market signals rather than individual-level behavioral annotations. The approach addresses two challenges: learning meaningful behavioral reasoning instead of relying on forecasting shortcuts, and identifying informative training examples from noisy market trajectories. To address these, the authors introduce Social Behavioral Decomposition, which models representative participant groups, their belief updates, and interactions, along with a Hindsight-Regret Curriculum that prioritizes market transitions where identifying relevant groups improves forecasting. The framework is trained using supervised fine-tuning followed by GRPO on prediction-market data (SWM-Bench) and transferred zero-shot to individual user simulation. Experiments show strong directional accuracy and correlation on Polymarket, competitive results on Kalshi, and improvements across four user-simulation benchmarks (Humanual, OvertonBench, PRISM, and CAD). The authors also demonstrate that synthetic data generated by macro2mind improves a downstream simulator's accuracy on unseen users by 15.5 percentage points (44.5% to 60.0%).

Critical strengths
  1. The paper explores an interesting direction by learning individual behavioral reasoning from aggregate prediction-market signals without requiring individual-level behavioral annotations during training. The combination of Social Behavioral Decomposition and Hindsight-Regret Curriculum provides a way to structure behavioral reasoning and prioritize informative market transitions. I find this approach promising because it offers an alternative to collecting expensive and privacy-sensitive individual behavioral data.

  2. The experimental results provide evidence for macro-to-micro transfer. The method achieves strong directional accuracy on Polymarket (0.674) and improves zero-shot performance across four user-simulation benchmarks. I find the synthetic-data experiment particularly interesting, where training on macro2mind-generated data improves accuracy on unseen users from 44.5% to 60.0%, compared with 46.8% using backbone-generated data. The ablation study (Figure 3) shows that the Hindsight-Regret Curriculum is important for the observed improvements, with Humanual response alignment increasing from 0.9 (without the curriculum) to 3.4 (full method). These results support the effectiveness of the proposed approach.

Critical weaknesses
  1. I am not fully convinced that accurate aggregate market forecasting necessarily implies learning the underlying individual behavioral mechanisms. For example, the same market price increase could result from many traders becoming slightly more optimistic or a few influential traders making large purchases while others remain skeptical. Although the authors show improvements on user-simulation benchmarks, they do not directly validate whether the inferred constituent beliefs and interactions reflect actual participant behavior.

  2. The model is required to generate six constituent groups (three upward-moving and three downward-moving), but the paper does not directly measure whether these groups represent meaningfully different behavioral perspectives. Different groups could produce nearly identical interpretations of the same event. I think measuring pairwise differences in their belief updates and responses would help establish whether the model learns genuine behavioral diversity rather than simply generating different group descriptions.

  3. The paper does not provide a sensitivity analysis for several design choices: the number of constituent groups (3 upward and 3 downward), interaction depth (second order), curriculum weights (0.4 and 0.6), and GRPO sampling temperature (0.6).

  4. The paper reports overall improvements on individual-simulation benchmarks but does not provide a detailed analysis of how performance varies across different user groups. Since the method learns from prediction-market participants, it is unclear whether the improvements generalize equally to users with different backgrounds and preferences. I think evaluating simulation accuracy across demographic or preference-based subgroups, where annotations are available, would provide stronger evidence for the generalizability of the learned behavioral reasoning.

Questions for authors
  1. Could the authors clarify how they assess whether the inferred constituent groups and their belief updates reflect actual participant behavior? Is there any evidence beyond aggregate forecasting accuracy and downstream simulation performance that supports the behavioral validity of these inferred groups?

  2. Have the authors measured the diversity of beliefs, interpretations, or responses across the six generated constituent groups? If such analysis is available, it would help establish whether the groups capture meaningfully different behavioral perspectives rather than similar reasoning expressed through different descriptions.

  3. Could the authors explain the rationale for selecting exactly three upward-moving and three downward-moving groups and limiting interactions to second-order reasoning? Were alternative group counts or interaction depths explored during development, and if so, how did they affect performance?

  4. Were any sensitivity experiments conducted for the curriculum weights (0.4 and 0.6) or GRPO sampling temperature (0.6)?

  5. Do the authors have any analysis of individual-simulation performance across demographic or preference-based user groups? It would be useful to understand whether the improvements are consistent across different populations or concentrated in particular types of users.

3 comments

We sincerely thank the reviewer for the careful reading and constructive feedback. Below, we address each concern and question in detail. We will incorporate the relevant clarifications and additional analyses and experimental results into the revised manuscript. We would be happy to provide further clarification during the discussion period.

[More evidence supports the behavioral validity of these inferred groups: Human Evaluation of Behavioral Validity] We provide independent human evaluation of the inferred behavioral reasoning (Section 6, Figure 5). Five human annotators evaluate whether the inferred constituent groups, their event interpretations, and belief updates provide plausible and coherent explanations of observed market movements. macro2mind achieves a score of 72.2, substantially outperforming Qwen3-4B (47.1) and Qwen3.5-397B (51.4) under the same structured prompt. These results provide independent human-judged evidence of the behavioral plausibility of our inferred groups and belief updates, beyond forecasting accuracy and downstream simulation performance.

[Diversity Analysis Across the Six Constituent Groups]

We measure diversity across all 4,243 SWM-Bench test traces, containing 25,458 generated groups. To distinguish meaningful behavioral diversity from different group labels, we compare the prior beliefs, interpretations, and updated beliefs of groups on the same side (both "up" or both "down").

FieldVerbatim / Near-duplicate Same-side Pairs*Distance: Same-side vs. Opposite-side
Prior belief1.2% / 1.9%0.54 vs. 0.53
Interpretation0.1% / 0.7%0.36 vs. 0.37
Updated belief6.9% / 12.1%0.46 vs. 0.43

Near-duplicate: cosine similarity > 0.9 using all-MiniLM-L6-v2. Distance is measured as cosine distance.

The results show that prior beliefs and interpretations are rarely duplicated, and same-side groups exhibit semantic distances comparable to opposite-side groups. This suggests that the generated groups capture distinct perspectives rather than merely repeating the same reasoning under different labels.

We further examine how groups use evidence. Among traces with at least three news items, 85.5% of same-side pairs cite different news items, while 97.9% of traces contain at least one news item interpreted in opposite directions. Updated beliefs show lower diversity, partly because their directions are constrained by the up/down split. The diversity therefore lies primarily in why different groups update their beliefs, rather than simply the direction of their updates. We will include this analysis in the appendix.

[Rationale for Constituent Count and Interaction Depth] We choose three upward and three downward groups to balance coverage of diverse behavioral perspectives and reasoning length, avoiding both missing important groups and introducing excessive redundancy.

We limit interactions to the second order for two reasons: (1) prediction markets often respond rapidly to news, making immediate reactions and their most consequential interactions particularly relevant; and (2) modeling higher-order interactions introduces additional computational costs and uncertainty from recursively simulating group responses.

Our ablations further show that 3+3 groups provide a strong balance across downstream tasks, while second-order reasoning generally outperforms first-order reasoning, supporting these design choices.

[Ablations on the Number of Upward and Downward Moving Groups]

Upward GroupsDownward GroupsHumanual State ↑Humanual Resp. ↑CAD Acc ↑Overton MAE ↓Overton Spearman ↑PRISM MAE ↓PRISM Pairwise Acc ↑
1110.93.70.43300.66960.637721.8440.6256
3 (Ours)3 (Ours)9.53.40.42200.63500.653021.2000.6330
5510.53.70.43200.64320.645321.8100.6183
UnconstrainedUnconstrained8.72.60.11200.68690.627021.5310.6097

[Ablations on the interaction depth]

Interaction DepthHumanual State ↑Humanual Resp. ↑Overton MAE ↓Overton ρ ↑PRISM MAE ↓PRISM Pairwise Acc. (%) ↑CAD Acc. (%) ↑
18.52.50.6260.65222.163.332.4
2 (Ours)9.53.40.6350.65321.263.342.2

[Sensitivity experiments on curriculum weights] We add sensitivity experiments on the weight of regret term and difficulty term.

Weight of Regret TermWeight of Difficulty TermHumanual State ↑Humanual Resp. ↑CAD Acc ↑Overton MAE ↓Overton Spearman ↑PRISM MAE ↓PRISM Pairwise Acc ↑
017.52.00.37700.65760.630021.9530.6262
0.20.88.32.30.38650.70250.614921.6460.6263
0.4 (Ours)0.6 (Ours)9.53.40.42200.63500.653021.2000.6330
0.80.27.11.80.40550.71250.590022.3300.6194
106.41.60.40500.72690.636522.7380.6222

[Analysis of individual-simulation performance across demographic and preference-based user groups] We further analyze individual-simulation performance across demographic and preference-based user groups on PRISM, OvertonBench, and CAD, comparing macro2mind with Qwen3-4B using the same test data.

The results show that improvements are broadly distributed rather than concentrated in specific populations. On OvertonBench, MAE improves across all 17 demographic and viewpoint subgroups, with particularly large gains for minority and singleton viewpoints (MAE reductions of 0.159 and 0.236, compared to 0.080 for majority viewpoints). On PRISM, pairwise accuracy improves across most of the 37 examined subgroups. On CAD, accuracy improves in five of six subgroups.

These results suggest that macro2mind generalizes across diverse user populations, with particularly strong gains for viewpoints that are harder for the backbone to simulate. We will include the complete subgroup results and confidence intervals in the appendix.

Overall Consistency Across User Subgroups

BenchmarkSubgroup DimensionsImproved SubgroupsKey Finding
OvertonBenchParty, sex, age, ethnicity, viewpoint17/17 (MAE)Larger gains for minority viewpoints
PRISMAge, gender, region, education, English proficiency, ethnicity, religion, conversation type33/37 (Pairwise Acc.)No significant heterogeneity across dimensions
CADCountry, political leaning5/6 (Accuracy)Gains across countries and most preference groups

All 17 OvertonBench subgroups show statistically significant MAE improvements. None of the four non-improving PRISM subgroups or the one declining CAD subgroup shows a statistically significant decrease.

Representative Subgroup Results

BenchmarkSubgroupMetricQwen3-4B → macro2mindImprovement
OvertonBenchMajority viewpointMAE ↓0.693 → 0.613−0.080
OvertonBenchMinority viewpointMAE ↓0.812 → 0.653−0.159
OvertonBenchSingleton viewpointMAE ↓0.922 → 0.686−0.236
OvertonBenchAge 60+MAE ↓0.834 → 0.617−0.217
PRISMValues-guided conversationsPairwise Acc. ↑—+1.4 pp
PRISMControversy-guided conversationsPairwise Acc. ↑—+1.1 pp
PRISMNon-native English speakersPairwise Acc. ↑—+1.5 pp
CADIndiaAccuracy ↑39.4% → 43.4%+4.0 pp
CADUnited StatesAccuracy ↑38.3% → 40.6%+2.4 pp
CADRight-leaningAccuracy ↑40.2% → 37.8%−2.4 pp (n.s.)

pp = percentage points; n.s. = not statistically significant. PRISM subgroup results report accuracy improvements rather than absolute scores.

Thank you for the review. We truly appreciate the time and expertise you dedicated to it. As the discussion period draws to a close, we would appreciate it if you could take a moment to review our responses. If you have any further feedback or additional questions on any point, we would be happy to discuss it.

If our responses have addressed your concerns and questions, we kindly request a reconsideration of the merit of our work and an adjustment to the overall assessment score. Thanks again for your time!

Reviewer_9Cs7OpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces macro2mind, a framework for learning individual behavioral simulation from aggregate prediction-market signals. It trains a language model to forecast market prices by identifying participant groups, modeling their responses to news and one another, and aggregating their reactions. A hindsight-regret curriculum prioritizes training examples where outcome-informed group hints improve forecasting (this is the key methodology to me). The resulting model transfers to individual user simulation without additional training. Validation covers prediction-market forecasting, four user-simulation benchmarks, and synthetic-data generation for downstream simulators. Results show improved forecasting and user simulation. The authors have done useful ablations on their methodology.

Critical strengths
  • The authors provide solid evidence of generalization through zero-shot evaluation on four user-simulation benchmarks and downstream training with synthetic data. These results go beyond market forecasting and support the central thesis that learning from aggregate market signals can improve predictions of individual responses.

  • The insight that not all behavioral data provides useful learning signals is insightful. The ablations show that market-forecasting training without the curriculum does not improve user simulation, while the full method yields clear gains. This makes data selection a consequential part of the approach.

Critical weaknesses
  • The transfer mechanism is not clearly established. The ablations show that the curriculum is important, but do not distinguish gains from better data selection from those attributable to the proposed behavioral reasoning. A comparison using the same curriculum with free-form reasoning, alongside difficulty-based sampling without hindsight hints, would help clarify what drives transfer.
Questions for authors
  • What evidence distinguishes the contribution of the prescribed behavioral reasoning structure from that of data selection? Have you compared the same curriculum with free-form reasoning? These comparisons would help clarify what drives transfer.
  • What do you expect would change with a larger LLM? Would the transfer gains and the importance of the reasoning structure prior and curriculum persist, and what informs that expectation?
2 comments

We sincerely thank the reviewer for the careful reading and constructive feedback. Below, we address each concern and question in detail. We will incorporate the relevant clarifications and additional analyses and experimental results into the revised manuscript. We would be happy to provide further clarification during the discussion period.

[More Comparisons on Data Sampling Curriculum and Reasoning Form] We conduct a 2×2 ablation to examine the effects of reasoning structure and curriculum-based data selection. With structured behavioral reasoning, the full curriculum substantially improves Humanual State/Resp. scores from 6.3/0.9 to 9.5/3.4 and CAD accuracy from 31.85% to 42.20%. In contrast, applying the same curriculum to free-form reasoning does not improve performance on these tasks. The combination also achieves the best Overton Spearman correlation and PRISM MAE. These results suggest that structured behavioral reasoning and curriculum-based data selection complement each other, their combination yielding the strongest overall transfer performance.

Reasoning StructureCurriculum data selectionHumanual State ↑Humanual Resp. ↑CAD Acc ↑Overton MAE ↓Overton Spearman ↑PRISM MAE ↓PRISM Pairwise Acc ↑
Free-form ReasoningNo6.11.50.36600.62300.647023.0000.5390
Free-form ReasoningYes5.51.30.35700.61960.647221.8200.6203
Behavior ReasoningNo6.30.90.31850.68460.625521.4780.6284
Behavior ReasoningYes9.53.40.42200.63500.653021.2000.6330

[Ablation of Data Selection Curriculum Components] We further ablate hindsight regret and difficulty-based sampling across four downstream benchmarks. Both components contribute to performance improvements, while combining them yields the strongest overall results.

CurriculumHindsight RegretDifficultyHumanual State ↑Humanual Resp. ↑CAD Acc ↑Overton MAE ↓Overton Spearman ↑PRISM MAE ↓PRISM Pairwise Acc ↑
Uniform SamplingNoNo6.30.90.31850.68460.625521.4780.6284
Difficulty-onlyNoYes7.52.00.37700.65760.630021.9530.6262
Hindsight-onlyYesNo6.41.60.40500.72690.636522.7380.6222
Full CurriculumYesYes9.53.40.42200.63500.653021.2000.6330

[Scalability of Transfer Gains and Method Components] We expect the transfer gains to persist with larger LLMs, although the relative improvements may decrease as base models become stronger.

(1) Scaling alone does not guarantee better behavioral reasoning. In our human evaluation (Fig. 5), Qwen3.5-397B scores only 51.4 under the same structured prompt, compared to 72.2 for macro2mind-4B. (2) The reasoning structure remains important because market rewards supervise only the final forecast, without directly supervising how the model reasons about people's beliefs and responses. Our 2×2 ablation supports this: with the same curriculum, structured BR achieves 9.5/3.4 on Humanual State/Resp., compared to 5.5/1.3 with free-form reasoning. (3) The curriculum is adaptive to model capabilities. It continuously prioritizes transitions based on the current policy's difficulty and hindsight regret, allowing it to identify informative examples as the model improves. We expect larger models to benefit less from SFT initialization and potentially show smaller relative gains, while the adaptive curriculum may remain useful as model capabilities improve.

The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models. Pan et al. 2022.

Measuring Faithfulness in Chain-of-Thought Reasoning. Lanham et al. 2023.

Thank you for the thorough response and the new ablations.

Comments on the paper

1 comment

General Comments

We sincerely thank all reviewers for their constructive feedback and recognition of our work. To address their concerns, we conducted additional ablations and analyses on behavioral validity, training components, sensitivity, and generalization. The table below summarizes the key feedback and our responses.

DimensionReviewer 9Cs7Reviewer eKjUReviewer PYx1Action / Summary
Novelty & Significance"The insight that not all behavioral data provides useful learning signals is insightful.""The paper explores an interesting direction...""an interesting and relatively novel direction"Strength: Reviewers recognize the novelty of learning behavioral reasoning from macro signals.
Evaluation & Generalization"solid evidence of generalization""evidence for macro-to-micro transfer""relatively broad empirical evidence"Strength: Reviewers recognize the extensive zero-shot evaluations and synthetic-data results.
Behavioral Validity"The transfer mechanism is not clearly established.""they do not directly validate whether the inferred constituent beliefs and interactions reflect actual participant behavior.""the observed improvements could reflect better event understanding or structured reasoning..."Human Evaluation: 72.2 vs. 47.1/51.4. Synthetic Data: 60.0% vs. 46.8%. SFT Ablation: Outcome-blind initialization also transfers.
Reasoning Structure & Curriculum"A comparison using the same curriculum with free-form reasoning..."—"A controlled comparison against difficulty-only sampling is necessary."Additional Ablations: 2×2 reasoning/curriculum comparison, difficulty-only vs. hindsight-only sampling, and curriculum-weight sensitivity across four benchmarks.
Constituent Diversity & Design—"Different groups could produce nearly identical interpretations..." and "sensitivity analysis for several design choices"—Diversity Analysis: 25,458 groups. Design Ablations: Group counts, interaction depth, and curriculum weights.
Generalization Across Populations—"different backgrounds and preferences""the gains on PRISM are relatively small"Subgroup Analysis: Improvements in 17/17 OvertonBench, 33/37 PRISM, and 5/6 CAD subgroups; expanded ablations across four benchmarks.
Scalability"What do you expect would change with a larger LLM?"——Scaling Analysis: Discuss expected gains and component importance using existing evidence and policy-adaptive curriculum design.

Overall, these additional analyses strengthen the evidence for transferable behavioral reasoning, the complementary roles of reasoning structure and curriculum-based learning, and generalization across diverse user populations. We will incorporate the additional results and clarifications into the revised manuscript and would be happy to address any remaining questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.