acceptodds
Under review as a conference paper at ICLR 2027

Handle With CARE: Can LLMs Reproduce How Online Communities React?

Abstract

Large language models (LLMs) are increasingly used as proxies for computational social analysis, yet faithfully representing the “thick descriptions” (Geertz, 1973) of human communities remains a critical challenge. Current evaluations often reduce social identity to static labels, sidelining how real-world groups navigate social shifts. To bridge this gap, we introduce CARE: Community-Aware Reaction Evaluation, a reaction-centered framework that benchmarks LLM-simulated discourse against the authentic, event-contingent responses of distinct communities to real-world news.  Spanning 207 Reddit communities, 2,194 news-driven posts, and 9,947 authentic reactions, CARE evaluates leading LLMs using a hierarchical taxonomy covering coarse attitudes and fine-grained communicative tones. Our empirical findings expose two critical failure modes in prevailing community-conditioning paradigms. First, while community context and targeted reasoning significantly enhance macro-level attitudinal and tonal profiling, these gains largely collapse at the instance level when predicting reactions to specific events. Second, the benefits of community conditioning are remarkably uneven: prompting strategies yield non-uniform shifts, where fidelity gains in some communities are offset by performance drops in others. This micro-macro divergence and community-level instability demonstrate that standard conditioning enables models to approximate static baseline profiles without capturing dynamic or equitable event reactions, establishing CARE as an essential diagnostic tool for community-aware social simulation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, LLM evaluation, online communities

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_avQdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CARE, a benchmark for evaluating how well LLMs reproduce reactions from online communities. The authors collect Reddit comments about COVID-19 news from 207 communities and compare them with simulated reactions from Gemini models. They evaluate both general attitudes and more specific tones, such as sarcasm or frustration.

Critical strengths

I think this is an interesting research question, especially given how often LLMs are used to simulate human behavior and opinions.

I also appreciate the data that consists of 207 Reddit communities. The distinction between overall community behavior and reactions to specific events is interesting.

Critical weaknesses

The authors use the five most upvoted comments to represent a community's reaction to a news article. However, I'm not sure these comments necessarily represent the community as a whole. Highly upvoted comments may reflect popular opinions, but they may also be influenced by timing, moderation, or Reddit voting behavior. (UPD: I still see no answer from authors, so again I highlight the weakness is that only highly upvoted comments are taken into account)

I'm also concerned that the authors generate only one simulated comment per news article. Real communities can have very different reactions to the same event. I wonder whether generating multiple responses and comparing their distributions would be more appropriate.

The authors use GPT-5 to annotate tones and then map them to broader attitudes. I appreciate that they include a human validation study. However, I'm not sure that assigning a fixed attitude to each tone is always appropriate.

For example, an assertive or sarcastic comment can express either a positive or negative opinion depending on the context. I wonder how much this mapping affects the results.

I also noticed that the community-informed CoT condition receives more information than the regular community-informed condition. It includes the subreddit description, while the other condition only provides the subreddit name. This makes it difficult to tell whether the differences come from reasoning or simply from having more information.

UPD: I still stand by these points and believe that the idea of the paper is great, but for clear acceptance, the points I mentioned must be addressed.

Questions for authors

Why did you choose only the five most upvoted comments to represent each community? Have you considered using a more diverse sample of reactions?

Have you tried generating multiple comments per article? I'm curious whether this would change the instance-level results.

How do you handle tones that can express different attitudes depending on context? For example, sarcasm is not necessarily negative.

UPD: For changing my score to 3, I still wait for answers

1 comment
AuthorsOpenReview ↗

Reply to avQd

Thank you for following up. Below are answers to each point, with pointers to the revised PDF.

1. Why the five most upvoted comments? (W1, Q1) Votes are the community's own signal of which reactions it endorses. A member who would say the same thing tends to upvote an existing comment, or reply under it, rather than post another top-level comment, so the votes on a top-level comment also carry the agreement of members who did not write it. We use top-level comments because nested replies tend to drift from the article (§3.1). The cap of five came from inspecting the raw data: even among posts with at least ten comments, many have only a few top-level comments (in the final set, 24% of posts have fewer than five), so a larger cap would mostly shrink the pool of eligible posts or leave more posts below it. We agree that votes are also shaped by timing and moderation: the five comments are the reactions the community promoted, not a sample of the whole thread, and we will state this scope explicitly in the final version.

2. Multiple comments per article? (W2, Q2) Yes, for Gemini 3.1 Pro: two additional full runs of every condition on all 2,194 posts (App. D, Table 10). The instance-level results barely change: across the three runs, the standard deviation is at most 0.005 for AttHit and 0.010 for per-post Spearman ρ and EM_syn, and the orderings between conditions hold in all 3 × 3 pairings of runs. On comparing distributions: the paper evaluates LLM simulation as it is commonly used, one simulated reaction per prompt at the provider's default temperature. How many voices should represent a community, and how many reactions a prompted LLM should produce, are separate design choices; we see distribution-level simulation as an open question the benchmark invites, and the coverage design of AttHit is what lets a single reaction be scored against a plural community.

3. Tones whose attitude depends on context (W3, Q3) The tone-to-attitude mapping is context-independent by design: each tone receives one attitude, so the attitude label never depends on a second, contextual judgment (§4.2). Assigning one of three values (positive +1, negative −1, neutral/mixed 0) to 364 tones is necessarily coarse, which is why attitude is only the coarsest layer of our hierarchical evaluation. The tone-level results (EM, EM_syn, JSD, sarcasm share, tonal diversity and structure) work on the tones and synonym groups themselves and do not use the mapping at all. For sarcasm, if one value must be chosen, we believe −1 fits best: people rarely sound sarcastic when their reaction is truly positive, and at most a sarcastic comment may read as mixed (Contempt Mockery & Wit → Sharp Sarcasm, Table 4), unlike playful (Humor Playfulness & Sincerity → Amusement & Wit, Positive, Table 5).

Assertive, on the other hand, raises a point well taken. Its attitude was debated among the authors when we built the taxonomy, and your comment helped us finally decide: assertive and its synonyms emphatic and confident (the Strong Conviction subgroup) are now mapped to Mixed instead of Positive (App. B, Table 7), and all attitude results use the updated mapping. The change moves AttHit by at most 0.005 and reverses no comparison between conditions in the paper.

4. Comm-Inf.-CoT receives more information (W4) That is right: the condition adds context (the subreddit description and the 2020 framing) and written reasoning at once. We separated the two (App. G): a full 2 × 2 on Claude Sonnet 5.5 (all posts) and a preliminary context-only run on Gemini 3.1 Pro (1,000 posts). For Sonnet, the reasoning step drives the change (AttHit +0.03, EM_syn +0.035; context alone: no clear change); for Gemini 3.1 Pro, the added context drives it (EM_syn −0.034, sarcasm share −8.2 points; reasoning on top: no clear change). Since the answer depends on the model, the main text now attributes these effects to the Comm-Inf.-CoT condition as a whole, and §4.1 describes everything it adds. The new prompts are in App. H.4–H.5.

We hope these clarifications and new analyses address your concerns, and we would be grateful if you would take them into account in your final assessment. We are happy to clarify anything further.

Reviewer_fUq8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether an LLM, told which subreddit it is speaking for, can write the comment that subreddit actually left on a specific news article i.e. the reaction for a particular community. The dataset is 2,166 news articles from 2020, each with the five most-upvoted real comments it received in each of 207 subreddits, 9,947 comments in total. Two Gemini models write one comment per article under three prompts: the article alone, the article plus the subreddit name, and the article plus the subreddit’s self-description with step-by-step reasoning. GPT-5 labels every real and generated comment with a tone (such as sarcastic or hopeful) and an attitude (positive, negative, or mixed). Two results. First, telling the model the subreddit name makes its average attitude per subreddit line up much better with the real average, but the share of individual articles where the model’s attitude matches the community stays at 59 to 63 percent no matter the prompt. Second, under every prompt change some subreddits get better and some get worse . The paper concludes that prompting teaches models what a community is like in general but not how it reacts to a given event.

Critical strengths
  1. Clear motivation for the problem that's trying to be addressed and constructing a dataset to answer it. Earlier benchmarks check whether a model knows what a community believes in general, from survey questions (Santurkar et al., ICML 2023) or multiple-choice questions built from subreddit posts (Chen et al., EMNLP 2025). From what I can tell, this is the first to pair real news articles with the comments a community actually left on them, across ~200 subreddits. That lets a model be scored on a community’s reaction to a specific event rather than on its general stance, which is the question people using LLMs as stand-ins for communities need answered. The dataset would be useful even if the paper’s findings changed.

  2. Scoring the same generated comments two ways, averaged per subreddit and article by article, is what makes it possible to see that a model can get a community’s average right while getting each article wrong. The sarcasm result shows this concretely: with reasoning, the model’s overall sarcasm rate matches the real rate of 22 percent while its match rate on individual articles goes down.

Critical weaknesses
  1. I am trying to understand if the main finding, that per-article accuracy “collapses,” holds. On each article the model gets a point if its comment’s attitude matches the community’s, and the models score 63 percent regardless of prompt. But Table 2 also shows what a real person scores: the most-upvoted real comment on each article matches the community’s attitude only 68 percent of the time. A community member, writing about that exact article, misses on a third of articles because the five comments disagree with each other. So the models at 63 may be near the best anyone can do, and the flat number may say that communities do not react in one voice, not that models fail. The paper never measures how well the real comments agree with each other, so we cannot tell these apart. Can the authors provide leave-one-out agreement among the real comments scored like the model is or clarify if I misunderstood this.
  2. The second finding, that conditioning helps some subreddits and hurts others, is not separable from chance. Each subreddit has about ten articles, so its score moves by a tenth whenever one article flips, and the model samples each comment once, so re-running the same prompt would flip some articles on its own. If the subreddit name did nothing, re-rolling 207 scores from ten articles each would send about half up and half down with an average change near zero. The paper reports 79 better, 85 worse, average change -0.009, and in another transition 88 and 88 (Figure 4). That is the no-effect pattern. Without a measure of how much a subreddit’s score moves on its own, the counts do not support the claim.
Questions for authors

Were comments generated once per article at a fixed temperature? If you run the model several times per article, how many of the 85 subreddits that got worse under the subreddit name move by more than that run-to-run variation? Google has licensed Reddit data for training since February 2024. Did you check whether generated comments overlap with the real 2020 comments (for example, n-gram overlap)?

1 comment
AuthorsOpenReview ↗

Reply to fUq8

Thank you for the close reading; both weaknesses led to changes in the paper.

1. Does the instance-level finding hold if the top comment reaches only 0.68? (W1) We think this comes from how AttHit is defined, and the revision makes it explicit (§4.4, p. 5; Table 2, p. 7). AttHit is not a hit or miss per article: for each post, it is the fraction of the real comments whose attitude the prediction shares. The top-voted comment's 0.68 therefore does not mean that it misses on a third of articles; it means that one real voice, scored on its whole thread (itself included), shares the attitude of about two thirds of the voices in it. We agree with the premise behind the question: communities do not react in one voice, and they differ in how much. All comments share one attitude in fewer than three in ten posts, and across communities the average share of comments holding a post's majority attitude ranges from 0.50 to 0.96 (median 0.74). This is why AttHit measures coverage. It also gives the attainable maximum for a single prediction: the post's majority attitude, which covers 0.75 on average (now in the Table 2 footnote, p. 7). The models reach 0.60–0.64, and per-post Spearman ρ stays at 0.27–0.35 for all four models, while the same majority prediction reaches 0.63; no prompt moves the models toward these values. We have also replaced "collapse" with "do not carry over" / "stagnation" throughout, which describes this result more accurately.

2. Are the community shifts separable from chance? (W2, Q1) You are right that the up/down counts alone do not separate an effect from noise: re-running an unchanged condition also moves communities up and down in similar numbers (67–89 up, 72–88 down). We therefore ran Gemini 3.1 Pro twice more in every condition and tested the shifts directly (App. E and Table 11, p. 22; Fig. E3, p. 23):

  • With the community name, a community's shift in one run correlates +0.38 with its shift in an independent run; 41% of the variance of a single-run shift belongs to the community; and post-level shifts cluster by community (permutation p = 0.0002).
  • To your exact question: 22 communities get worse and 14 get better in all 3 × 3 pairings of runs; without any effect, about 21 would be separated in either direction combined.
  • The step from Comm-Informed to Comm-Inf.-CoT shows no such pattern (correlation +0.02, 7%, p = 0.15): it lowers tone match by a similar small amount everywhere. §5.3 (p. 8) now states this. Gemini 2.5 Pro has a single run, so its Fig. 4 counts cannot be tested this way. On temperature: comments were generated once per post per condition at the provider's default temperature, the most commonly used setting; App. D (p. 21) reports how much every metric moves across three runs.

3. Overlap with the real 2020 comments (Q2) We measured word n-gram overlap between each simulated comment and the real comments of the same post, excluding text from the article shown in the prompt. Simulated comments share 0.03–0.15% of their 4-grams with the real comments of their post, less than real comments share with each other (0.19%). The longest shared run is 11 words, and the eight runs of eight or more words (out of about 26,300 simulated comments) are common idioms ("a light at the end of the tunnel"). We find no sign that the real comments' wording is reproduced, and we will add this check to the appendix of the final version.

We hope these clarifications and new analyses address your concerns, and we would be grateful if you would take them into account in your final assessment. We are happy to clarify anything further.

Reviewer_uh9ROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CARE (Community-Aware Reaction Evaluation). This is a diagnostic benchmark and evaluation framework designed to measure how well large language models (LLMs) simulate the authentic, event-contingent discourse of online communities. Rather than evaluating the socio-cultural alignment against the static demographic labels or survey responses that was aggregated, CARE grounds evaluation in real-world reactions to external events.

The authors create a dataset of 9,947 top-level comments reacting to 2,166 COVID-19 news articles (from 2020) across 207 diverse Reddit communities. To evaluate open-ended reactions systematically, the authors develop a Speech Act Theory-inspired hierarchical taxonomy that maps 362 fine-grained communicative tone descriptors into 141 synonym groups (L2) and coarse attitudes (L1 of positive; negative; and neutral/mixed), labeled via a human-validated GPT-5 annotation pipeline.

Evaluating two models (Gemini 2.5 Pro and Gemini 3.1 Pro Preview) across three prompting paradigms (Community-Blind, Community-Informed, and Community-Informed-CoT), the authors uncover two primary empirical findings of the Macro-Micro Divergence and Community-Level Volatility.

Critical strengths

(1) The paper's most consequential contribution is demonstrating that aggregate distributional alignment (macro-level profile matching) creates a deceptive illusion of community awareness. By pairing corpus-level metrics (e.g., Spearman correlation of community attitude means and Jensen-Shannon divergence) with instance-level event metrics (e.g., Attitude Hit Rate and Exact Tone Match), the authors prove that CoT and persona conditioning merely calibrate global output priors (such as suppressing overall sarcasm rates to match human baselines of ~22–23%) without knowing which specific events trigger those tones. This is a vital methodological takeaway for computational social science and LLM simulation research.

(2) The CARE benchmark is thoughtfully constructed, using bipartite keyphrase salience scoring and temporal down-sampling to ensure cohesive cross-community event overlap across 207 subreddits. Grounding the evaluation in illocutionary speech acts (via a two-level tone hierarchy validated with high inter-annotator agreement, Gwet's AC1=0.944AC_1 = 0.944) provides a scalable, interpretable middle ground between brittle surface-text similarity metrics and overly coarse sentiment analysis.

Critical weaknesses

(1) Narrow Model Coverage.

The empirical evaluation relies solely on two models from a single proprietary family (Gemini 2.5 Pro and Gemini 3.1 Pro Preview). Evaluating open-weights models (e.g., Llama, Qwen) and at least one other frontier family (e.g., Claude, GPT) is necessary to verify whether the observed micro-level stagnation and negativity bias are universal limitations of community conditioning or specific artifacts of Google's alignment pipeline.

(2) Confounded Prompt Design and Single-Sample Evaluation.

Transitioning from Community-Informed to Community-Informed-CoT simultaneously introduces the subreddit's self-description, the year (2020), CoT reasoning, and JSON formatting constraints. Bundling these variables makes it impossible to isolate whether CoT reasoning itself causes the observed performance regressions, or if models are simply over-indexing on the literal subreddit description. Additionally, generating only a single simulated reaction per post to compare against up to 5 heterogeneous human comments limits distributional evaluation at the instance level.

Questions for authors

(1) Can you provide evaluation results on at least one open-weights model family (e.g., Llama-3/4 or Qwen) and another major frontier model family to confirm that the macro-micro divergence and negativity bias are not specific to the Gemini Pro model series?

(2) In Appendix D.3 vs. D.4, Comm-Informed only provides /r/{subreddit}, whereas Comm-Inf.-CoT adds {subreddit_description}, the year 2020, CoT reasoning, and JSON formatting. Did you run an ablation where Comm-Informed also receives the subreddit_description and year without CoT? How do you know the regression in CoT tone fidelity is caused by reasoning rather than over-conditioning on the literal text of the subreddit description?

(3) In Figure 1, the ground-truth attitude score is computed by averaging {−1,0,+1}\{ -1, 0, +1 \} across the 5 comments (e.g., −0.6-0.6). Since only a single simulated reaction is generated per post (yielding a discrete attitude in {−1,0,+1}\{-1, 0, +1\}), how is AttHit defined when the ground-truth community average is fractional or mixed? Additionally, please clarify how the 0:100 (Random:Top) reference in Tables 2 and 3 is constructed—since the top-voted comment is one of the ≤5\le 5 ground-truth comments for that post, isn't EM guaranteed to be 1.001.00 by construction (making it an oracle upper bound rather than a predictive baseline)?

1 comment
AuthorsOpenReview ↗

Reply to uh9R

Thank you for the detailed review and the concrete suggestions. The general response summarizes the new results; here we answer your questions directly, with page numbers in the revised PDF.

How AttHit handles mixed ground truth (Q3) AttHit is a coverage score (§4.4, p. 5). For each post, it is the share of the post's real comments (up to five) whose attitude the simulated reaction shares; the reported value is the mean over posts. A post with three negative and two neutral comments gives 0.6 to a negative simulation and 0.4 to a neutral one, so a fractional or mixed ground truth is scored directly, without averaging the comments into one label first.

How the reference rows are built, and the 0:100 reference (Q3) Each reference row assigns every post a tone without any model. The Top Ref. (0:100) takes the tone of the post's most upvoted real comment; Random (100:0) draws a tone uniformly from the 364 tones of the taxonomy; the 50:50 and 25:75 rows choose between the two per post in that proportion. All rows are scored exactly like a simulated reaction, against all of the post's real comments. You are right that the Top Ref. therefore has EM = 1 by construction, since its tone comes from one of those comments. It is not meant as a predictor or an upper bound (for AttHit, the upper bound is each post's majority attitude, 0.75 on average; Table 2 footnote, p. 7), but as a set of multi-tiered references, from chance (Random) to one real voice of the thread (Top Ref.).

More model families (W1, Q1) We added Claude Sonnet 5.5 and the open-weights Qwen3.5-27B on the full benchmark (App. F, p. 23; Table 12, p. 23; Fig. F4, p. 24). Per-post attitude correlation stays between 0.27 and 0.35 for all four models under every prompt. The macro–micro divergence holds for Sonnet 5.5; Qwen3.5-27B differs, as the community name does not improve its community-level profile either. The negativity bias holds for both Gemini models and Sonnet 5.5, while Qwen3.5-27B errs the other way on negatively received posts.

Confounded Comm-Inf.-CoT condition (W2, Q2) We ran the ablation you asked for (App. G, pp. 23–25; Tables 14–16, p. 25): a full 2 × 2 on Claude Sonnet 5.5 and a preliminary context-only run on Gemini 3.1 Pro (due to budget constraints). Which component drives the change depends on the model (reasoning for Sonnet 5.5, the added context for Gemini 3.1 Pro), so the main text now attributes these effects to the Comm-Inf.-CoT condition as a whole, and §4.1 (p. 4) lists everything the condition adds. The JSON output stays bundled with the reasoning step.

Single sample per post (W2) Two more full runs of every Gemini 3.1 Pro condition (App. D, p. 21; Table 10, p. 22): instance-level metrics move by at most 0.013 (SD) across runs, and the orderings we discuss hold in all 3 × 3 pairings of runs.

We hope these clarifications and new analyses address your concerns, and we would be grateful if you would take them into account in your final assessment. We are happy to clarify anything further.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revised PDF, new results, and one correction

We thank the reviewers for their careful reading. The reviewers found the question timely (avQd), the event-level grounding new and the dataset useful in its own right (fUq8), and the pairing of community-level and instance-level scoring a key methodological takeaway (uh9R). We have uploaded a revised PDF; all changes are in blue.

The central claim, as it now stands. Community conditioning changes how well a model's simulated reactions match a community's overall profile, but this does not carry over to reactions to individual events: per-post attitude correlation stays between 0.27 and 0.35 for all four models we now evaluate, under every prompt, while the most a single simulated reaction per post could cover is well above what models reach (AttHit ceiling 0.75; Table 2 footnote). The orderings behind this claim hold across three independent runs of Gemini 3.1 Pro (App. D).

Concerns raised by more than one reviewer (short answers here, details in each reply and the revised PDF):

  1. More models (uh9R). Claude Sonnet 5.5 and the open-weights Qwen3.5-27B on the full benchmark (App. F). Instance-level stagnation holds for both; Sonnet shows the same divergence between community-level and instance-level results; Qwen behaves differently, and we report how.
  2. One sample per post (all three reviewers). Two more full runs of every Gemini 3.1 Pro condition (App. D). Instance-level metrics move little between runs (SD at most 0.013 for AttHit, RMSE, EM, EM_syn and per-post Spearman ρ), and the orderings we discuss hold in all 3 × 3 pairings of runs. The community-level shifts of Fig. 4 are tested against run-to-run noise in App. E.
  3. Context vs. reasoning in Comm-Inf.-CoT (uh9R, avQd). A full 2 × 2 ablation on Claude Sonnet 5.5 and a preliminary context-only run on Gemini 3.1 Pro (App. G). Which component matters depends on the model, so the main text now describes these effects as effects of the Comm-Inf.-CoT condition as a whole, and §4.1 lists everything the condition adds.
  4. Reading the metrics (uh9R, fUq8). AttHit is a coverage score (§4.4); the reference rows are explained in the caption of Table 2; the majority-attitude ceiling is added as a footnote.

Correction. While preparing the additional runs, we found that part of the Gemini 3.1 Pro Comm-Blind outputs had been matched to the wrong posts when batched results were read back. We re-matched them and verified the order of every other batch; no other condition or model is affected. The corrected values are in Tables 2–3 and Figs. 2–4 (revision note under Table 3). The correction removes the "initial performance slump" we described for Gemini 3.1 Pro under Comm-Blind and makes its community-level gains from the community name moderate rather than dramatic (e.g., attitude level 0.59 → 0.66); the conclusions are unchanged.

Other changes in the PDF.

  • "Collapse" is replaced by "do not carry over" / "stagnation" (Abstract, §1, §5), and the readings in §5.1–5.3 follow the corrected numbers.
  • The strong-conviction tones (emphatic, confident, assertive) are now mapped to Mixed instead of Positive (App. B; see our reply to avQd).
  • New appendices D (run-to-run stability), E (community shifts vs. noise), F (additional models) and G (ablation); the ablation prompts are in App. H.4–H.5.
AuthorsOpenReview ↗

Summary of rebuttal and revisions for the AC

Dear Area Chair,

Thank you for your time in handling our submission. As the discussion period comes to a close and we have not yet heard back from the reviewers, we are writing to give you a short summary of our responses and the revised submission. In particular, Reviewer avQd’s latest update (Nov 17) states that they are waiting for our answers before changing their score to 3. That update was made before our responses were posted. We have since answered each of their questions, with pointers to the revised PDF, and are still awaiting their response.

What we addressed. We answered every concern the reviewers raised. Where a concern could reasonably be settled by more experiments, we ran them within the rebuttal period:

  • More models (uh9R). We added Claude Sonnet 5.5 and the open-weight Qwen3.5-27B on the full benchmark (App. F). The micro-level stagnation holds for all four models. Sonnet 5.5 also shows the same macro-level gains, and with them the micro–macro divergence, while Qwen3.5-27B, a much smaller model, scores lower on most metrics, and community information does not improve even its macro-level fit.

  • One sample per post (all reviewers). We ran two more full runs of every Gemini 3.1 Pro condition (App. D). The results vary little across runs, and the main orderings discussed in Section 5 hold across all pairs of runs. We also tested the community-level shifts against run-to-run noise (App. E; fUq8).

  • Context vs. reasoning in Comm-Inf.-CoT (uh9R, avQd). We ran a full 2 × 2 ablation on Sonnet 5.5 and a preliminary context-only run on Gemini 3.1 Pro (App. G).

  • Reading the metrics (uh9R, fUq8). §4.4 now explains AttHit as a coverage score. The Table 2 caption explains how the reference rows are built, and a footnote adds the majority-attitude ceiling.

  • Tone-to-attitude mapping (avQd). We moved the strong-conviction tones (emphatic, confident, assertive) from Positive to Mixed (App. B) and updated all affected results.

Extensive PDF revision. We uploaded a substantially revised PDF. All existing tables, figures and their readings were updated, and four new appendices (D–G) were added for the new experiments and analyses. Each of our responses points to the exact sections and pages, so every change can be checked against the paper.

We would be grateful if you could take the revised PDF and our responses into account. If any reviewer comes back with further questions, we are happy to answer them.

We would also like to take this opportunity to thank all three reviewers for their careful reading and concrete suggestions. The additional experiments, clarified metrics, and revised tone mapping all stem directly from the reviewers' constructive feedback, substantially improving the clarity and rigor of our paper.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.