acceptodds
Under review as a conference paper at ICLR 2027

Rehydration: Fixed-Semantics Counterfactual Replay for Mechanistic LLM-Agent Social Simulation

Abstract

Counterfactual experiments with large language model (LLM) agents are diffi- cult to interpret when an intervention changes the platform state, thereby altering subsequent prompts and generated language. Standard replay comparisons can therefore mix structural policy response with post-intervention semantic adaptation, even when branches share the same decoding seed. We formalize behavioral and platform interventions through the Behavior–Platform Decomposed Mechanistic Transition Framework (BDMTF) and introduce Rehydration, a paired coun- terfactual replay protocol that generates and seals semantic opportunities before policy assignment and reuses them across intervention branches. This construction defines a fixed-semantics structural effect that complements the conventional adaptive-semantic effect. In an analytic setting with known ground truth, Re- hydration recovers the fixed-semantics target, whereas common-seed adaptive replay does not recover that target. Controlled factorial experiments further re- veal a behavior-by-ranking interaction: controversy-oriented ranking increases participation substantially more under conflict-responsive behavior while redirect- ing activity toward shallower reply structures, a regime we call Shallow Swarm. Mechanism-deletion and execution-trace analyses link the additional volume to conflict-responsive participation and the depth shift to ranking-conditioned expo- sure and reply targeting. The interaction is replicated across independently frozen model-family semantic catalogs and held-out simulator posts, while activation- form and boundary analyses identify settings in which parts of the effect weaken or reverse. Held-out platform evidence supports the conflict-to-volume pathway, whereas evidence for the shallower-depth component remains more limited. Over- all, Rehydration provides a reproducible way to isolate fixed-semantics structural effects from policy-induced semantic adaptation and to test mechanistic claims in generative-agent social simulations.

Primary area:
causal reasoning
Keywords:
counterfactual simulation, LLM agents, causal inference, social simulation, policy evaluation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_KWBDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a replay protocol that freezes semantic opportunities before a simulated policy intervention. Policy branches share the frozen pool and keyed randomness, while activation, exposure, and reply targeting can change. The resulting fixed-semantics effect is distinguished from an adaptive effect that permits new state-dependent language generation. A factorial study finds that controversy ranking increases participation more under conflict-responsive behavior and can produce shallower reply trees. The paper studies channel deletions, alternative response forms, learned topology, semantic catalogs, and external platform data, with stronger external support for volume than depth.

Critical strengths
  1. The protocol specifies what stays fixed across policy branches and what may change. The pre-treatment pool, separate consumption within each branch, keyed random streams and exhaustion rule make this distinction executable. Proposition 1 gives the corresponding unbiasedness result and states the covariance condition for a variance reduction. I find this useful for studies that want to separate structural policy response from newly generated semantic content.

  2. The mechanism study uses a four-cell interaction rather than interpreting the joint behavior-plus-ranking contrast as an interaction. Channel deletions, volume-matched comparisons, and the learned-topology experiment provide several checks on the implemented explanation. Reporting depth reversals and the weak external depth evidence makes the scope of the result easier to assess.

Critical weaknesses

1. The motivation and advantages of fixed-semantics replay are not sufficiently established.

It is unclear why fixing semantic opportunities is particularly useful for studying LLM-agent societies, where semantic adaptation is an essential part of the interactions. While the proposed method provides more controlled experiments, it is not obvious that the resulting counterfactual effect is more meaningful than the effect measured through conventional adaptive simulation.

The paper does include evaluations of simulation fidelity, including distributional similarity, intervention prediction, and semantic compatibility. However, these results do not clearly establish in which aspects Rehydration is preferable to existing methods. In particular, it would be helpful to demonstrate a scientific insight that becomes possible specifically because of the proposed control.

2. The comparison with conventional methods and the interpretation of the empirical findings are not fully convincing.

The theoretical justification largely follows standard ideas from causal inference and paired simulation. In the synthetic experiment, the ground truth is defined as the fixed-semantics effect, so it is not surprising that Rehydration estimates this quantity more accurately than adaptive replay, which targets a different effect. The empirical comparison demonstrates reduced variance, but the practical implications of this improvement remain unclear.

Similarly, while the authors provide mechanism ablations and robustness tests for Shallow Swarm, it is difficult to determine how much of this phenomenon arises from the specific behavioral and ranking mechanisms implemented in the simulator. The external evidence supports increased participation more clearly than the reduction in discussion depth. I would like to better understand the generality and significance of this finding beyond the chosen simulation settings.

3. The presentation could be substantially improved.

Several important definitions and distinctions are difficult to follow. For example, the random variables ϵ0\epsilon_0 and ϵ1\epsilon_1 appear in Section 3.1 without a formal definition of their distributions or coupling across interventions. The presentation also moves between fixed-semantics and adaptive effects, additive and multiplicative interaction measures, and different simulation configurations without always making their relationships sufficiently explicit.

Questions for authors

See Weaknesses section

4 comments

Response to Reviewer KWBD: W1

Thank you for asking what scientific insight the proposed control provides. We do not argue that fixed-semantics replay is universally more meaningful than adaptive simulation. Rather, it addresses a complementary question: how do policies change conversational outcomes when semantic opportunities are held constant?

1. Why fix semantic opportunities?

The key distinction is between a policy changing which replies are generated and changing which available replies are actually realized. Conventional adaptive simulation captures both effects, since different policy-induced states may lead to different generated content. Rehydration holds semantic opportunities constant across policy branches, allowing us to isolate policy-dependent changes in realized interactions under a shared semantic reference.

This distinction matters for moderation. Hiding a comment may prevent not only that comment from being seen but also subsequent replies to it. In adaptive simulation, policy branches may generate different semantic content, making it difficult to attribute differences in subsequent conversations to the moderation rule alone. Fixed-opportunity replay makes these downstream effects traceable independently of newly generated content. It complements rather than replaces adaptive simulation.

2. A scientific insight enabled by this control.

We conducted a five-thread multi-round moderation experiment to ask: Can two policies produce nearly identical activity levels but different conversations?

Each thread received two independently generated, parent-bound catalogs, each containing six first-layer opportunities and twelve possible continuations. All 180 reply texts and parent links were sealed before policy assignment.

Within each catalog and paired seed, both policy branches shared the initial state, frozen opportunities, participation draws, and moderation allowance. Only the moderation rule changed: coarse-label moderation used a shared hash to break ties among proposals with the same antagonism label, whereas semantic moderation prioritized replies classified as containing attacks. Both policies could hide up to two replies after the first six opportunities, with matched realized allowances. A reply could be published only if its fixed parent existed and remained visible. Phase II published the sealed text verbatim, with no new generation or rewriting.

Across five threads, two catalogs, two policies, and ten paired seeds, we obtained 200 trajectories (100 matched policy pairs). Both policies hid 200 reply instances and published nearly identical totals (954 vs. 952), yet 27/100 pairs produced different realized reply sets, across three threads. All 180 distinct replies were published at least once. Independent Qwen and DeepSeek evaluations yielded minimum acceptability rates of 94.44% and 98.33%, respectively.

This shows that similar activity levels do not imply conversational equivalence.

To explain why these differences arise, we held each realized action history fixed and changed only the visibility mask, separating immediate masking from subsequent interaction-path effects.

Table R1. Exposure contrasts and fixed-history decomposition.

Reference / outcomeTotal contrast [95% interval]Direct maskSubsequent path
Qwen / attack exposure−0.06667 [−0.19667, 0.00000]−0.08000+0.01333
DeepSeek / attack exposure−0.10556 [−0.36000, +0.04333]−0.12000+0.01444
Qwen / acceptable rebuttal exposure−0.11000 [−0.29239, +0.01333]−0.08667−0.02333
DeepSeek / acceptable rebuttal exposure+0.12556 [+0.02222, +0.22889]+0.16667−0.04111

Note. Contrasts are Semantic minus Coarse-label moderation. Exposure is the cumulative count of visible new replies over 18 scheduled opportunities, divided by 18, not a percentage. Acceptable rebuttals must satisfy the respective evaluator's minimum acceptability criterion. Intervals use paired bootstrap resampling of five threads and are descriptive; catalogs and seeds are repeated measurements. The decomposition is a simulator diagnostic, not a real-world mediation estimate.

The subsequent path component reduced acceptable-rebuttal exposure under both evaluators, suggesting that hiding a comment can also prevent substantive replies to it. Although total rebuttal-exposure effects differed across evaluators, this analysis reveals how similar activity totals can conceal different conversational continuations. Rehydration makes these policy-dependent effects traceable under shared semantic opportunities, providing a content-level mechanism analysis complementary to adaptive simulation.

Response to Reviewer KWBD: W2 (1) Theoretical Contribution, Synthetic Validation, and Practical Implications

Thank you for this important comment. We clarify the scope of our theoretical contribution, the role of the synthetic experiment, and the practical implications of our additional variance analyses.

1. Theoretical contribution.

We agree that the unbiasedness argument and common-random-number covariance identity follow established probability results. Our contribution is not a new pairing theorem, but an operational protocol for fixed-semantics counterfactual experiments in generative-agent simulations. Rehydration specifies how to generate and seal semantic proposals before intervention, share them across policy branches, maintain branch-local consumption states, couple structural randomness by operation, and prevent post-intervention semantic regeneration from changing the replay target.

2. Role of the synthetic experiment.

We agree that the synthetic experiment does not establish superiority over adaptive simulation. It verifies whether Rehydration recovers its declared fixed-semantics estimand under a known ground truth. Adaptive replay instead targets an effect that includes policy-induced semantic adaptation. Therefore, their difference reflects distinct estimands, rather than estimation error by adaptive replay. We use the synthetic experiment for estimator verification, not as a general ranking of the two workflows.

3. Practical implications of uncertainty and repeatability.

We conducted additional crossed-catalog, same-estimand pairing, and matched-budget comparisons to distinguish semantic-generation uncertainty from replay variability and workflow-level repeatability.

Table R3. Uncertainty decomposition and matched-budget repeatability.

ComparisonResultInterpretation
Between-catalog variance share34.9% [29.3%, 40.2%]Catalog generation contributes additional uncertainty
Independent / paired variance ratio, same estimand1.015 [0.994, 1.038]No clear precision gain from pairing
Original volume variance ratio4.89Original workflow comparison
Same threads and statistic, original comparison3.68 [3.25, 4.67]Advantage persists after alignment
Matched-budget volume variance ratio3.07 [2.48, 3.36]Lower volume-effect dispersion under freezing
Matched-budget depth variance ratio0.97 [0.84, 1.30]No clear stability advantage

For the crossed-catalog experiment, we used 25 threads, with 10 independently generated catalogs and 20 replay seeds per catalog. Catalog variation accounted for 34.9% of paired log-volume effect variance. Under the same fixed-semantics estimand and branch marginals, the independent-to-paired variance ratio was 1.015, indicating no clear precision gain from pairing itself.

For matched-budget repeatability, both workflows used 78 usable generation requests and 156 paired policy contrasts per workflow on the same 10 threads. We matched the model snapshot, temperature, and requested seeds, although actual token counts differed. The adaptive-to-frozen variance ratio was 3.07 [2.48, 3.36] for individual log-volume policy contrasts, with lower variance under freezing on all 10 threads. For leaf depth, the corresponding ratio was 0.97 [0.84, 1.30], showing no clear advantage.

The previously reported 9.61 [7.58, 12.37] measures the ratio of average per-thread variances of budget-averaged log-volume effects, a different statistic from the 3.07 comparison. It should not be interpreted as an increase from the original 4.89 ratio.

The practical implication is that researchers should allocate experimental budgets across independently generated semantic catalogs as well as replay seeds, rather than repeatedly sampling only one catalog. Rehydration also provides more repeatable volume-effect estimates under the matched workflow, although this advantage does not extend clearly to depth.

Overall, Rehydration contributes an operationally specified experimental control, verifies recovery of its declared estimand, and provides practical guidance for separating semantic-generation uncertainty from replay variability. It complements adaptive simulation by enabling a different, explicitly defined counterfactual analysis rather than replacing conventional adaptive effects.

Response to Reviewer KWBD: W (2) Shallow Swarm — Mechanism, Robustness, and Generality

Thank you for raising this concern. We first clarify what Shallow Swarm means in the original paper, and then report additional experiments examining its mechanisms and generality.

1. Original definition of Shallow Swarm.

In the original manuscript (Section 5.1, Figure 3), we define Shallow Swarm as a behavior–platform interaction: controversy-oriented ranking is nearly neutral for baseline populations but amplifies participation under conflict-responsive behavior, while the resulting reply structures become shallower. Thus, it is not simply an increase in comment volume, but a joint pattern involving participation and discussion structure. Section 5.2 further investigates the mechanisms behind this pattern.

2. Mechanism separation.

On 79 held-out Reddit threads, the original policy reproduced both comment-volume amplification (1.230 [1.206, 1.253]) and mean leaf-depth compression (−1.517 [−2.615, −0.366]). Matching simulated trajectories at a common comment-count prefix retained the negative depth interaction (−1.692 [−2.906, −0.409]), indicating that volume differences alone do not explain depth compression.

An eight-cell activation-by-fatigue-by-ranking ablation further distinguished the behavioral pathways.

Table R4. Mechanism ablation and depth-tail robustness.

ConditionVolume interaction [95% CI]Depth interaction [95% CI]
Activation varied; fatigue off1.234 [1.228, 1.240]+0.013 [−0.010, +0.038]
Activation fixed on; fatigue varied—−0.327 [−0.353, −0.299]
Common depth cap, 97-thread split1.240 [1.236, 1.244]−0.332 [−0.373, −0.291]

These results suggest that conflict-responsive activation primarily contributes to volume amplification, whereas depth fatigue contributes to the negative depth interaction in the learned-topology implementation. The common depth cap reduced maximum-depth normalized Wasserstein distance from 3.263 to 1.170, while both interactions persisted.

3. Dependence on decision policies.

We replaced the original hand-designed decision rule with alternative selectors to examine whether the joint pattern was specific to that rule.

Table R5. Fixed-pool decision-policy comparisons.

Decision policyThreadsVolume interaction [95% CI]Depth interaction [95% CI]
Original rule791.230 [1.206, 1.253]−1.517 [−2.615, −0.366]
Earlier learned selector791.000 [0.987, 1.012]−0.155 [−0.481, +0.150]
Qwen-calibrated bounded selector501.057 [1.015, 1.098]−0.112 [−0.181, −0.049]
Live-Qwen pilot120.691 [0.600, 0.800]−0.287 [−0.389, −0.183]

The earlier learned selector reproduced neither interaction. The exploratory 50-thread selector reproduced both, whereas the 12-thread Live-Qwen pilot retained negative depth interaction but not positive volume amplification. Thus, the joint pattern is not exclusive to the original hand-designed rule, but is not guaranteed under autonomous LLM decisions.

4. External evidence and generality.

EvidenceFindingInterpretation
Reddit conflict–volume regression10.375 [3.815, 16.982]Supports conflict–participation association
1,415-thread lexical-proxy analysisNo established depth compressionDoes not validate the depth pathway
HN/Lemmy observational samplesNo established depth compressionLimited external support for depth effects
r/anime ranking auditNo Best-versus-Controversy comparisonTarget ranking effect not identified

The observational evidence supports an association between early conflict and subsequent participation, but does not directly establish ranking-conditioned depth compression.

Overall, we characterize Shallow Swarm as a conditional behavior–platform interaction, not a universal empirical law. Its volume amplification and depth compression arise through distinct mechanisms, depend on behavioral and decision policies, and have different levels of external support. Its significance is to reveal why participation and discussion structure must be evaluated jointly: increased activity does not necessarily imply deeper discussion.

Response to Reviewer KWBD W3 : Definitions — Randomness, Contrasts, and Effect Scales

Thank you for identifying these presentation issues. We agree that the mathematical definitions and relationships among experimental quantities should be explicit in the main text rather than left implicit in the implementation. We will revise Section 3.1 and the experimental sections accordingly.

(1) Structural randomness and coupling.

Conditional on the initial context and frozen semantic catalog, each base structural draw is modeled as an independent Uniform(0,1) variable across distinct operation keys. The symbols ϵ0\epsilon_0 and ϵ1\epsilon_1 denote the collections of these draws for the two policy arms, rather than single scalar errors.

Paired replay uses the same draw for each matching operation key in both arms, whereas independent replay uses separate draws with identical marginal distributions. In the 79-thread implementation, keys include the post-level seed, simulation step, agent, and operation, but exclude the decision method, behavioral condition, and ranking. Thus, an abstention in one branch does not shift subsequent matching draws. Categorical choices transform shared base draws using branch-specific probabilities, so shared randomness does not force identical actions.

(2) . Relationships among estimands, contrasts, and simulation configurations.

We will explicitly distinguish these quantities in the main text (Table R5).

Table R5. Distinctions needed to interpret the policy comparisons.

DistinctionMeaning
Fixed-semantics vs. adaptive effectShared pregenerated semantic proposals vs. policy-dependent semantic generation
Joint contrast vs. interactionChanging behavior and ranking together vs. measuring whether the ranking effect differs across behavioral conditions
Additive depth interactionDifference between ranking-induced depth changes across behavioral conditions; measured in layers, with null value 0
Multiplicative volume interactionRatio of ranking effects across behavioral conditions; with null value 1
Simulation configurationThe decision, exposure, timing, and other rules defining each experimental comparison

Let Db,rD_{b,r} and Vb,rV_{b,r} denote the corresponding depth and volume outcomes, where bb indexes behavioral conditions and rr indexes ranking conditions.

The interactions are defined as follows:

ID=(D1,1−D1,0)−(D0,1−D0,0).I_D=(D_{1,1}-D_{1,0})-(D_{0,1}-D_{0,0}).

IV=V1,1/V1,0V0,1/V0,0.I_V=\frac{V_{1,1}/V_{1,0}}{V_{0,1}/V_{0,0}}.

Thus, the additive depth interaction has a null value of zero, whereas the multiplicative volume interaction has a null value of one. For volume, we calculate the interaction on the log⁡(1+new replies)\log(1+\text{new replies}) scale and exponentiate the resulting contrast, yielding a ratio-of-ratios rather than a ratio of aggregate arithmetic means.

We will also add a setup-and-contrast key alongside the main results, distinguishing local robustness tests, global phase-boundary analyses, and policy-replacement experiments. These configurations differ in their decision rules and execution settings and will not be presented as interchangeable replications.

These revisions will make the relationship between each research question, its estimand, effect scale, and simulation configuration explicit, allowing readers to interpret the mathematical formulation and empirical results consistently.

Reviewer_YXipOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the Behavior–Platform Decomposed Mechanistic Transition Framework (BDMTF), developed to distinguish effects between behavioral and platform interventions in LLM-agent simulation experiments. Base on this framework, the authors introduce Rehydration, a protocol that allows the reuse of semantic opportunities and thus a fixed-semantics structural effect, compared to the conventional adaptive-semantic effect one. Using Rehydration, the authors reveal the state of shallow swarm, where controversy-oriented ranking leads to more shallower replies under a thread.

Critical strengths
  1. The authors distinguish the effects coming from the recommendation ordering and the one from the semantic change caused by the ordering switch. By fixing the semantic opportunity space, the authors can reveal how ordering itself affects the behavior of the agents, where they discover the pattern of shallow swarm.
  2. The authors have conducted substantive experiments and analysis about the revealed ordering effect and the shallow swarm pattern, as well as whether Rehydration really achieve the goal of splitting behavioral and platform intervention effects.
Critical weaknesses
  1. In Rehydration, the phase II operation may introduce unexpected bias related to the policy step. Rehydration generates candidate response types (semantic opportunites) that don't depend on live condition, then for every context choose one of them every time based on a separated policy, from which a deterministic response is rendered accordingly. However, the candidate phase II policies are manually designed and may not align with what agents actually want to choose by themselves. Therefore, the experiment results, including the discovered shallow swarm phenomenon, are limited to agents that adopt this externally designed policy, and thus may not apply to the more general agents that directly respond to live status.
  2. The paper has offloaded many critical details to the appendix, which makes the main sections rather difficult to understand. For example, the experiment configurations are nearly omitted in the main text, making the experiment Section 5 full of unexplained notations and difficult to interpret. The arrangement of figures is also confusing: one experiment figure is inserted in Figure 1 but the detailed analysis is in Section 5.3.
Questions for authors
  1. How can you guarantee that the observations can transfer to real scenarios or dynamic simulations?
  2. Is it possible that the pre-generated semantic opportunities cannot fit the live status where replies are already built?
4 comments

Response to Reviewer YXip: W1

We would like to clarify that Shallow Swarm was not intended to describe a universal behavior of autonomous agents. As stated in the Abstract (p. 1) and demonstrated in Section 5.1 (p. 7), it describes a specific phenomenon: controversy-oriented ranking has little effect on baseline agents, but increases participation and produces shallower discussions when agents are conflict-responsive. Section 5.4 (pp. 8–9) further examines the conditions under which this pattern holds or disappears, explicitly clarifying that it is not a universal law of the simulator.

To address whether this finding depends on the hand-designed Phase II policy, we conducted additional decision-policy replacement experiments. Importantly, Rehydration fixes the available semantic proposals, not the agents' decisions about whether to reply or which comment to target. These decisions can still depend on the evolving discussion state.

First, we collected multi-turn Qwen decisions from 12 training threads and evaluated the learned decision models on 8 held-out threads (512 recorded states). Exact parent-selection agreement with Qwen increased from 36.0% to 69.2%, while the bounded-feedback model predicted a mean reply probability of 13.54%, close to Qwen's observed 15.04%.

We then replaced the original scripted decision rule while keeping semantic pools, initial conditions, platform settings, and random draws fixed.

Table R1. Behavior-by-ranking interactions under different decision policies.

Decision policyThreadsVolume interaction [95% CI]Leaf-depth interaction [95% CI]
Original scripted rule791.230 [1.206, 1.253]−1.517 [−2.615, −0.366]
Earlier learned selector791.000 [0.987, 1.012]−0.155 [−0.481, +0.150]
Qwen-calibrated bounded-feedback selector501.057 [1.015, 1.098]−0.112 [−0.181, −0.049]

Note. Volume interaction has a null value of 1; leaf-depth interaction has a null value of 0.

Results. The earlier learned selector did not reproduce either interaction, whereas the Qwen-calibrated bounded-feedback selector reproduced both directions. In a separate live-Qwen pilot on 12 threads, the negative depth interaction remained (−0.287 [−0.389, −0.183]), although the positive volume interaction did not persist (0.691 [0.600, 0.800]). This pilot used fewer decision opportunities per step and should not be treated as a matched replacement experiment.

Implication. These results show that Shallow Swarm is not independent of the decision policy. Its emergence depends on how agents respond to the evolving discussion and how the platform ranks content. The results support the negative depth interaction across multiple decision implementations, while the positive volume amplification is more sensitive to the decision policy. We therefore clarify that Shallow Swarm is a conditional behavior–platform interaction, not a universal outcome of autonomous-agent simulation.

Response to Reviewer YXip: W2

We thank the reviewer for pointing out these presentation issues. We agree that the main text should provide sufficient experimental context without requiring readers to repeatedly consult the appendix.

(1) Experimental configurations and notation. We will add a compact experimental overview table at the beginning of Section 5, specifying the simulator, intervention, fixed inputs, sample sizes, replay seeds, and primary outcomes for each experiment. We will also define key quantities when they first appear, including the volume interaction (RoR, null = 1) and depth interaction (DiD, null = 0), and explicitly identify the statistical unit used for each confidence interval. Full parameter settings and reproduction details will remain in the appendix.

(2) Figure organization. We will separate the two panels currently combined in Figure 1. The conceptual schematic (Figure 1b) will remain in the Introduction, while the empirical dispersion comparison (Figure 1a) will be moved to Section 5.3, alongside its experimental setup and detailed analysis. Its caption will explicitly report the 25-thread, 5-seed setting and the median live-to-frozen variance ratio of 4.89 [3.33, 9.61].

These revisions will make the experimental settings, reported statistics, and corresponding interpretations accessible directly in the main text, while retaining the appendix for implementation and reproducibility details.

Response to Reviewer YXip: Q1

We thank the reviewer for this important question. Our goal is not to fully reproduce real-world social dynamics, but to provide a controlled framework for identifying how behavioral and platform policies affect discussion outcomes under explicitly defined conditions. Rehydration enables controlled counterfactual comparisons under fixed semantic opportunities. We therefore examine the portability and boundary conditions of the resulting observations across independent implementations, alternative exposure mechanisms, different discussion structures, and observed platform interventions.

We conducted the following additional evaluations:

Table R2. Transfer and robustness evaluations across implementations and environments.

EvaluationExperimental settingVolume interaction (RoR)Leaf-depth interaction (DiD)
Independent implementationReddit / graph-based engine1.057 [1.015, 1.098]−0.112 [−0.181, −0.049]
Alternative exposureReddit / highest-ranked only1.000 [0.986, 1.016]−0.121 [−0.222, −0.015]
Cross-environmentHN / source viewport1.044 [0.976, 1.116]−0.041 [−0.146, 0.064]
Cross-environmentHN / highest-ranked only0.997 [0.977, 1.019]−0.139 [−0.245, −0.030]

Note. All four settings use 50 threads, 50 agents, 24 simulation steps, and three paired seeds. Volume interaction is multiplicative (null = 1); leaf-depth interaction is additive in layers (null = 0). The independent graph-based implementation additionally reproduced all 600/600 Reddit source-viewport outcomes exactly.

First, implementation-level reproducibility. An independently implemented graph-based replay engine reproduced all 600 Reddit source-viewport outcomes exactly under matched conditions. This shows that the replay results are not specific to a single software implementation.

Second, environmental robustness. Under highest-ranked-only exposure, the volume interaction disappeared, whereas the negative depth interaction persisted. On 50 held-out Hacker News discussion trees, neither interaction was statistically distinguishable from zero under the original exposure setting. Under highest-ranked-only exposure, the negative depth interaction remained, while the volume interaction was not supported. These findings indicate that depth reduction is more persistent across the tested configurations than volume amplification.

Importantly, Hacker News provides real initial text and reply topology, but agent profiles and semantic catalogs remain Reddit-derived, and voting is simulated. This is therefore a cross-environment simulation, not direct validation of Hacker News user behavior.

Third, external evaluation on real platform interventions. We evaluated 103 chronologically held-out Lemmy moderation events against observed reply trees.

Evaluation metricBDMTFBaseline
Aggregate MAE1.6341.208 (Additive Ridge)
Direction accuracy73.1%73.1% (Majority-sign)
Event-path composite error0.0440.029 (Always-no-reply)

BDMTF did not outperform these baselines. We therefore treat Lemmy as an external evaluation of intervention-response prediction, rather than evidence of established real-platform predictive validity.

Regarding behavioral-policy dependence, our Qwen-calibrated policy replacement experiments provide additional evidence that Shallow Swarm is not exclusive to the original hand-designed Phase II policy (see our response to W1).

Overall, our results support a conditional rather than universal interpretation of Shallow Swarm. Volume amplification is more sensitive to behavioral and exposure conditions, whereas depth reduction persists across more of the tested configurations. Rehydration provides a controlled framework for identifying these policy effects and their boundaries, rather than a guarantee of unrestricted real-world generalization.

Response to Reviewer YXip: Q2

We thank the reviewer for raising this important concern. Rehydration freezes semantic templates and opportunity tokens, not the evolving discussion or the obligation to publish every opportunity. A pre-generated semantic opportunity may remain applicable to newly emerging viewpoints, provided that the selected target satisfies its predefined compatibility conditions. After target selection, the deterministic renderer resolves the parent reference without regenerating the semantic content. However, this applicability is not unlimited: opportunities that fail the compatibility conditions should not be forced into the discussion.

We examine contextual compatibility at two levels: whether intent-constrained expressions fit selected parent comments, and whether frozen semantic opportunities can be executed in evolving reply trees.

First, we evaluated contextual expression on 200 matched parent–reply cases. GPT-4o-mini generated intent-constrained adaptations and polarity-matched direct replies for the same selected parent. Two independent human evaluators assessed anonymized candidates, yielding 400 judgments per method.

Table R3. Same-parent reply quality.

OutcomeIntent-constrained adaptationDirect generation
Minimum acceptability86.25%80.25%
Strict acceptability58.25%45.00%
Obvious mismatch3.50%3.50%
Minimum acceptability and intent retention51.75%44.25%

These results show that adapting the expression to a selected parent while preserving the original intent improves acceptability, although contextual fit and intent retention are not always achieved simultaneously.

Second, we tested frozen-content compatibility in parent-bound replay. We constructed parent-bound frozen opportunity graphs from real comment prefixes on five development threads. Each thread had two independently generated catalogs, each containing six first-layer opportunities and twelve possible continuations. All 180 reply texts and their parent links were fixed before policy assignment. During replay, a continuation was published only if its predetermined parent had appeared and remained visible; otherwise, it was not emitted. Phase II used the frozen text verbatim, without further LLM rewriting.

Table R4. Compatibility of frozen replies in parent-bound replay.

Independent evaluatorMinimum acceptanceStrict acceptancePublication-weighted minimum acceptance
Qwen94.44%89.44%92.60%
DeepSeek98.33%79.44%98.16%

Note. The evaluation covered 180 distinct replies across 200 trajectories and 1,906 publication instances. All 180 replies appeared at least once. No low-rated proposal was removed. Both evaluators met the predeclared overall acceptance threshold of 80% and per-thread threshold of 60%.

Overall, these results show that freezing semantic opportunities does not require forcing every pre-generated reply into an evolving discussion. Intent-constrained adaptation improves contextual expression, while parent-bound replay demonstrates that frozen replies can be executed when their designated parents become available. We acknowledge that compatibility with arbitrary dynamically selected parents has not yet been established; this remains a limitation of the current implementation.

Thank you for the detailed responses and additional experiments. The Qwen-calibrated policy replacement alleviates the concern about dependence on the original hand-designed Phase II policy. The mixed results across decision policies also clarify the boundaries of Shallow Swarm and support interpreting it as a conditional behavior--platform interaction. The proposed experimental overview, notation clarifications, and figure reorganization directly address the presentation concerns.

The compatibility experiments provide useful additional evidence, although LLM-based adaptation and parent-bound replay do not fully establish compatibility under the original deterministic rendering procedure with dynamically selected parents. Real-platform generalization also remains limited.

I've adjusted my ratings accordingly.

Reviewer_bTHLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the difficulty of interpreting counterfactual experiments in LLM-agent social simulations when a policy intervention changes both platform dynamics and subsequently generated language. To put this in a simple question, the work asks how a discussion would change under a different policy if the semantic opportunities available to agents were held fixed. To solve this problem, the work introduces the Behavior–Platform Decomposed Mechanistic Transition Framework (BDMTF), together with Rehydration, a two-phase replay protocol that generates and seals semantic opportunities before treatment assignment and reuses them across policy branches with matched keyed random streams. This construction defines a reference-distribution fixed-semantics effect that complements the conventional adaptive-semantic effect. To evaluate the approach, the work combines an analytic experiment with known ground truth, a paired behavioral-by-ranking factorial over 1,500 replay blocks, mechanism-deletion analyses, learned-topology experiments, model-family sensitivity tests, and external platform diagnostics. The results show that controversy ranking changes comment volume by 0.988× under baseline behavior but 1.200× under conflict-responsive behavior, yielding a multiplicative interaction of 1.214. Under the tested configurations, increased participation coincides with shallower reply trees, a regime termed “Shallow Swarm.” As the work states, the contributions are (1) an explicit fixed-semantics estimand and intervention decomposition, (2) an executable paired replay protocol, and (3) a mechanistic investigation of behavior-conditioned ranking amplification, including robustness tests and failure boundaries.

Critical strengths

The work has at least three strengths: (1) a clear distinction between counterfactual targets and a carefully specified replay protocol, (2) substantial mechanistic investigation of the behavior–ranking interaction, and (3) extensive diagnostic evaluation with transparent reporting of limitations.

First, the paper carefully distinguishes the fixed-semantics effect from the adaptive-semantic effect and consistently acknowledges that adaptive language generation is a meaningful response to an intervention. This distinction is technically reasonable: in COSMOS, for example, common-seed generation still conditions on branch-specific sensory information and moderation memory, allowing the generated content to diverge after intervention. Rehydration makes its alternative target operational through a treatment-blind proposal pool, branch-local consumption, keyed random streams, and explicit exhaustion rules. Proposition 1 also states the conditions required for its variance result, including identical marginal outcome distributions and nonnegative cross-branch covariance. These details make the intended experimental contrast inspectable and reduce ambiguity about what is being held fixed.

Second, the paper supports its central observation with several complementary mechanism tests. The main factorial distinguishes the behavior–ranking interaction from the larger joint-versus-baseline contrast, which is necessary for establishing that ranking effects depend on behavioral response. On the separate 25-post deletion panel, removing the complete conflict-response pathway reduces the volume interaction from 1.414 to 1.059, with the latter interval including one. The negative depth interaction persists in the complete depth-channel deletion experiment and in a separate comment-count matching analysis. Execution traces further connect the depth shift to the exposed candidate set and subsequent parent selection. The learned-topology experiment provides an additional check, reproducing a volume interaction of 1.234 and a negative depth interaction on held-out posts. Together, these analyses connect the observed outcomes to identifiable components of the simulator.

Third, the evaluation makes a considerable effort to expose the scope of the results. The paper includes independently frozen model-family catalogs, alternative activation functions, pool-capacity audits, a human semantic-compatibility study, chronological evaluation partitions, and explicit failure-region analyses. It also reports unfavorable findings, including the learned topology’s excessive depth tail, inconclusive external evidence for the shallow-depth component, and the sparse Lemmy event-path benchmark. The documented configuration hashes, parameter-freeze boundaries, and offline replay design further strengthen the auditability of the work. This transparency is particularly valuable in a setting where apparently plausible social behavior can otherwise be difficult to distinguish from consequences of simulator design.

Critical weaknesses

However, there are concerns too. The major concerns are: (1) the restricted role of semantic content and the resulting uncertainty about the protocol’s additional scientific value, (2) limitations in the empirical comparison used to establish the benefits of semantic freezing, and (3) insufficient independent support for the shallow-depth mechanism.

First, the restriction placed on semantic feedback substantially narrows what the experiments establish. Sections 3.1–3.3 and Appendix L specify that rendered text cannot influence subsequent activation, targeting, or timing. Appendix R.2 and Table 22 then show that replacing the model-family surface catalogs produces exactly zero change in the volume and depth interactions when operational coordinates are held fixed. The nonzero reference sensitivity comes from changing the mapping of reply and antagonism prevalence into response-policy coordinates. The authors correctly distinguish these tests, but it remains unclear what the semantic layer contributes beyond supplying operational inputs to a structured agent-based simulator. A matched non-LLM proposal baseline, or a concrete example in which substantive frozen message content changes a mechanistic conclusion, would clarify the additional value.

This concern also affects the interpretation of semantic compatibility. The human audit is useful, but its 77.7% acceptability rate uses the minimum criterion of relevance and coherence scores of at least three. Under the stricter threshold of four, only 8.1% of rendered replies qualify, and mean relevance is 2.95/5 (Appendix H). The comparison with verbatim transplantation establishes an improvement over literal transfer, but does not establish comparability with context-sensitive generation. These findings do not invalidate the structural estimator, since rendered text is excluded from transitions. They do, however, leave a gap between a mathematically well-defined fixed-input experiment and a semantically credible conversational counterfactual.

Second, the evidence for the practical benefits of freezing does not fully separate estimand choice from differences in randomness and catalog reuse. The analytic experiment in Table 12 successfully verifies the declared fixed-semantics target, but the adaptive baselines deliberately include a policy-dependent semantic shift. Their deviation from that target therefore demonstrates the difference between the estimands; it does not establish superior estimation of a shared quantity. The authors acknowledge this distinction.

The empirical dispersion comparison has an additional limitation. Appendix O.2 states that the adaptive arm generates new batches from branch-specific 12-step pilot snapshots for each repeat, without fixing the API decoding seed, whereas the frozen arm reuses one condition-blind catalog per thread across replay seeds. Consequently, the reported median variance ratio of 4.89 compares variability conditional on a frozen catalog with variability that also includes fresh semantic generation. This is informative about rerun dispersion under the two implementations, but does not isolate the contribution of the information barrier or establish lower total uncertainty for the expectation over Qref. Crossing several independently generated frozen catalogs with replay seeds, together with a paired-versus-independent comparison targeting the same fixed-semantics effect, would provide a more informative evaluation.

Third, the shallow-depth component is supported much more strongly within the simulator than by independent evidence. The learned-topology experiment is a useful addition, but its behavioral intervention still changes both conflict-responsive activation and depth fatigue (Appendix O.6). It therefore does not independently establish that the same depth interaction follows from conflict responsiveness while the other behavioral components remain fixed. Moreover, the learned model produces a mean maximum depth of 14.888 compared with 6.640 in the observed data, failing the prespecified maximum-depth fidelity criterion (Table 16). This discrepancy matters because reply depth is central to the proposed phenomenon.

The external evidence does not resolve this uncertainty. Held-out Reddit supports an association between early conflict and subsequent volume, while the depth intervals include zero. On the Lemmy aggregate benchmark, BDMTF’s MAE of 1.634 is worse than the additive Ridge baseline’s 1.208, and its 73.1% direction accuracy equals the majority-sign baseline (Table 18). The event-path benchmark contains only four observed replies across three of 103 test interventions, so it cannot provide substantial evidence about active discussion trajectories. The authors appropriately report these limitations. Nevertheless, the independent evidence currently provides limited support for the substantive shallow-structure claim or for the practical predictive value of the resulting simulator.

The minor concerns are: (1) insufficient separation of different robustness claims in the main narrative, and (2) incomplete specification of the principal external regression result.

First, positive volume interaction, a joint higher-volume/lower-depth direction, and a statistically supported negative depth interaction are different findings. Appendix Q explicitly states that the additive and multiplicative activation variants have depth-interaction intervals spanning zero, even though their volume interactions remain positive. The main text should consistently preserve this distinction when describing robustness across activation forms. Similarly, the local and global boundary panels use different simulators and contrasts, so their support counts should remain clearly associated with their respective experimental questions.

Second, the reported early-conflict–subsequent-volume slope of 10.375 [3.815, 16.982] plays an important role in the external-support argument, but I could not locate a complete specification of that analysis in the manuscript or its included appendices. The sample size, regression equation, adjustment variables, coefficient scale, and uncertainty procedure should be stated explicitly. These details are necessary to assess how much this observational result supports the proposed activation pathway.

Questions for authors

The paper addresses an important methodological problem in LLM-based social simulation: specifying what a policy comparison means when generated language changes along with the intervention. Its explicit estimand, detailed replay contract, factorial design, and extensive mechanism diagnostics constitute a useful contribution. The authors also show considerable care in distinguishing simulator-conditional effects from claims about human platforms and in reporting failure cases.

However, the current evidence does not sufficiently establish the additional scientific value of the semantic replay layer beyond a structured agent-based simulator with fixed operational inputs. The practical variance comparison leaves catalog-generation uncertainty insufficiently separated, while the semantic audit and external depth evidence provide limited support for interpreting the resulting trajectories as meaningful conversational counterfactuals. My concern is therefore about the demonstrated scope and significance of the contribution. On balance, I lean marginally below the acceptance threshold. A convincing demonstration in which substantive frozen semantics contributes to a useful mechanistic conclusion, accompanied by a better-controlled uncertainty comparison, would substantially strengthen the case for acceptance.

Here are several questions: First, Appendix R.2 shows that replacing the surface catalogs has exactly zero effect when operational coordinates remain fixed. Can the authors clarify what inferential or explanatory value the frozen semantic proposals add beyond an agent-based simulator using the same action and response coordinates? A matched baseline or a concrete policy-analysis example where the semantic representation affects the scientific conclusion would be particularly helpful. A parent-conditioned live-generation reference would also help interpret the absolute quality of the semantic audit.

Second, can the authors cross multiple independently sampled frozen catalogs with replay seeds to distinguish between-catalog uncertainty from within-catalog simulation variance? How much of the reported dispersion advantage remains under matched generation budgets and controlled decoding randomness, and what is the gain from pairing when both compared methods target the same fixed-semantics estimand?

Third, in the learned-topology experiment, does the negative behavior-by-ranking depth interaction persist when conflict responsiveness changes while depth fatigue remains fixed? Please also provide the full specification of the external conflict–volume regression and clarify whether any observed-data analysis directly tests the proposed ranking-conditioned depth pathway.

5 comments

Response to Reviewer bTHL: Q1-a Inferential Value of Frozen Semantic Proposals(1/2)

We thank the reviewer for this important question. We agree that replacing surface text while keeping operational coordinates fixed does not change the original volume or depth interactions. This is the intended information barrier of Rehydration.

1. A concrete policy-analysis example.

To clarify what frozen semantic proposals add beyond operational inputs, consider two antagonistic replies: one challenges an argument with evidence, while the other attacks its author. Although both share the same coarse antagonism label, they may require different moderation decisions.

An ABM relying only on the original action/response coordinates cannot distinguish these cases. Frozen semantic proposals preserve this distinction, allowing us to investigate whether the content of a reply changes how it should be moderated.

We conducted two evaluations to examine whether semantic information captures these differences and whether they affect moderation decisions under identical budgets.

2. Can semantic content distinguish personal attacks from substantive rebuttals?

We evaluated 147 frozen proposals from five held-out threads, comparing three methods:

  • Coarse: Original action labels and community information.

  • Lexical: Non-LLM lexical features.

  • Semantic: Classification based on frozen message content.

Table R1. Identifying content differences hidden by coarse action labels.

MethodPure attacks detected ↑Correct on agreed cases ↑
Coarse5/27 (18.5%)66/96 (68.8%)
Lexical2/27 (7.4%)33/96 (34.4%)
Semantic18/27 (66.7%)82/96 (85.4%)

Note. Among 147 proposals, Qwen and DeepSeek independently agreed that 27 were pure personal attacks and agreed on the content categories of 96. These are model-based reference labels, not human ground truth.

Results. The semantic method identified 18/27 pure personal attacks, compared with 5/27 for Coarse and 2/27 for Lexical. Among the 96 evaluator-agreement cases, it correctly classified 82, versus 66 and 33 for the baselines.

The same antagonistic-reply label can therefore conceal attacks and substantive rebuttals that the original operational coordinates cannot distinguish.

3. Does this distinction change moderation decisions?

We tested whether semantic information changes moderation decisions under identical budgets. All three methods selected 15 replies from the same 147 frozen proposals, without access to reference labels.

Table R2. Moderation targeting under the same intervention budget.

ReferenceMethodAttacks selected ↑Rebuttals not selected ↑
QwenCoarse7/3262/64
Lexical8/3261/64
Semantic12/3264/64
DeepSeekCoarse10/5864/66
Lexical8/5863/66
Semantic15/5866/66

Note. Each method selects 15 proposals. Denominators indicate the total reference-labeled attack-containing or substantive-rebuttal-containing proposals. “Not selected” means not targeted for moderation, not necessarily preserved in subsequent simulation.

Results. Under the same budget, Semantic selected more attacks (12 versus 7/8 under Qwen; 15 versus 10/8 under DeepSeek), while selecting no reference-labeled substantive rebuttals. At a larger budget of 30, it continued to target more attacks, although under Qwen it also selected one additional rebuttal-containing reply compared with Coarse (4 versus 3).

Thus, semantic targeting is not uniformly superior across budgets. Instead, it makes the trade-off between targeting attacks and avoiding unnecessary moderation of substantive disagreement explicit.

Scientific implication. These experiments show that semantic information offers more than finer content descriptions: it changes moderation choices even when the candidate replies and intervention budget are identical.

Response to Reviewer bTHL: Q1-a Inferential Value of Frozen Semantic Proposals(2/2)

Rehydration explicitly defines and implements a fixed-semantics counterfactual comparison protocol for LLM-agent social simulations. Under this protocol, frozen semantic proposals provide a shared reference across policy interventions, enabling us to examine not only structural outcomes such as reply volume and depth, but also how policies affect personal attacks, substantive rebuttals, and subsequent discussions.

We illustrate this additional explanatory value through two analyses.

1. Can similar reply counts conceal different discussions?

We compared coarse-label and semantic moderation on five development threads, using identical frozen reply opportunities, intervention budgets, and paired random seeds. No LLM calls were made during replay.

Table R3. Similar reply counts, different discussions.

OutcomeCoarse policySemantic policy
Published reply instances954952
Phase II LLM calls00

Results. Across 100 matched comparisons, the policies produced different published reply sets in 27 cases, despite nearly identical total reply counts (954 vs. 952). Thus, similar reply volumes can conceal differences in which conversations actually occur.

2. Can direct moderation benefits be offset by subsequent discussion changes?

We further examined how moderation affected different types of content. We separated direct masking (which replies were directly hidden) from subsequent paths (how earlier moderation decisions affected later replies).

Table R4. Content-exposure differences (Semantic→Coarse\text{Semantic} \rightarrow \text{Coarse}).

Reference / OutcomeTotal difference [95% interval]Direct maskSubsequent path
Qwen / Attack exposure ↓−0.067 [−0.197, 0.000]−0.080+0.013
DeepSeek / Attack exposure ↓−0.106 [−0.360, +0.043]−0.120+0.014
Qwen / Rebuttal exposure ↑−0.110 [−0.292, +0.013]−0.087−0.023
DeepSeek / Rebuttal exposure ↑+0.126 [+0.022, +0.229]+0.167−0.041

Note. Differences are normalized by 18 scheduled opportunities, not percentages. Intervals are descriptive estimates from five development threads.

Results. Under DeepSeek, semantic moderation preserved more substantive rebuttal exposure through its direct choices (+0.167), but subsequent discussion changes offset part of this benefit (−0.041), leaving a smaller overall gain (+0.126). Under Qwen, both components were negative. Attack exposure was lower in both point estimates, but the intervals do not establish a robust reduction.

These results illustrate a policy trade-off that aggregate reply counts and the original coarse action labels cannot directly reveal: a policy's effect depends not only on which replies it directly targets, but also on what kinds of discussion remain afterward.

By holding semantic opportunities fixed across policies, Rehydration makes these content-level consequences observable and comparable under the same reference conditions. This extends controlled policy analysis beyond aggregate volume and depth without requiring semantic text to directly drive the original structural kernel.

The findings are exploratory and simulator-conditional; we do not claim that semantic moderation is uniformly better or that the results generalize to real platforms. Full details are provided in revised paper.

Response to Reviewer bTHL: Q2 — Uncertainty and Repeatability

(1) Distinguishing Sources of Uncertainty

We thank the reviewer for this suggestion. Rehydration allows us to distinguish uncertainty arising from independently generated semantic catalogs from simulation randomness within a fixed catalog.

We conducted a crossed experiment on 25 threads, with 10 independently generated catalogs per thread and 20 replay seeds per catalog.

Table R5. Uncertainty decomposition across catalogs and replay seeds.

Setting / ResultValue
Threads25
Catalogs per thread10
Replay seeds per catalog20
Between-catalog variance share34.9% [29.3%, 40.2%]

Results. Between-catalog variation accounted for 34.9% of the variance in paired log-volume policy effects, showing that replaying a single catalog with different seeds cannot capture all uncertainty. By fixing semantic proposals within each policy comparison and varying catalogs independently, Rehydration enables these two sources of uncertainty to be separately quantified.

(2) Dispersion Under Matched Generation Requests

We agree that the original dispersion advantage should be reassessed under matched generation budgets and controlled decoding settings. We therefore repeated the comparison using matched generation requests and the same variance statistic as the original experiment.

Both workflows were evaluated on the same 10 threads, with 78 usable generation requests and 156 paired policy contrasts per workflow. We matched the model snapshot, temperature, and requested random seeds.

Table R6. Dispersion under matched generation requests.

Setting / OutcomeAdaptive / Frozen variance ratio [95% CI]
Original experiment (25 threads)4.89
Original experiment (same 10 threads)3.68 [3.25, 4.67]
Matched requests: log volume (10 threads)3.07 [2.48, 3.36]
Catalog-aware variance check3.55 [2.87, 3.88]
Matched requests: leaf depth0.97 [0.84, 1.30]

Note. Ratios use the median across threads of Adaptive/Frozen sample-variance ratios for individual policy contrasts. The catalog-aware result is an additional variance-component check.

Results . Using the same threads and variance statistic, the Adaptive/Frozen ratio decreased from 3.68 to 3.07 under matched generation requests, with lower log-volume variance for Frozen on all 10 threads. The catalog-aware check showed a consistent advantage (3.55), but no clear advantage was found for leaf depth (0.97). These results support Rehydration's improved comment-volume repeatability, rather than a universal gain in statistical efficiency or accuracy. Decoding settings were matched, although generated outputs and actual token counts could differ.

(3) Statistical Gain from Paired Replay

We agree that pairing should be evaluated separately when both methods target the same fixed-semantics estimand. We therefore compared paired and independent replay while holding the semantic reference, policy branches, and estimand fixed.

Table R7. Statistical gain from pairing.

ComparisonIndependent / Paired variance ratio [95% CI]
Same fixed-semantics estimand1.015 [0.994, 1.038]

Results . The independent-to-paired variance ratio was 1.015 [0.994, 1.038] , indicating no clear variance reduction from pairing. Thus, the observed workflow repeatability advantage cannot be attributed simply to paired random seeds . Rehydration's primary benefit remains the ability to compare policies under the same frozen semantic conditions.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We sincerely thank all three reviewers for their constructive feedback. We have uploaded a revised manuscript, with changes highlighted in blue, incorporating the additional experiments and clarifications from our rebuttal.

The revisions address four key concerns:

  1. Scientific value of fixed-semantics replay. A new multi-round moderation experiment shows that two policies can produce nearly identical reply volumes (954 vs. 952) while yielding different realized reply sets in 27/100 paired comparisons. This demonstrates how Rehydration enables content-level analysis of policy-dependent discussion pathways.

  2. Uncertainty and repeatability. We separated semantic-catalog uncertainty from replay variability (34.9% between-catalog variance share). Under matched generation-request budgets, frozen replay exhibited lower comment-volume effect dispersion across all 10 tested threads (median variance ratio: 3.07). We also clarified that pairing alone provides no significant precision advantage.

  3. Mechanism robustness and limitations. Additional experiments separate conflict responsiveness from depth fatigue and evaluate alternative agent decision policies, including Qwen-calibrated and live-Qwen variants. The results clarify that Shallow Swarm is a conditional behavior–platform interaction rather than a universal phenomenon.

  4. Semantic compatibility. New same-parent and parent-bound replay experiments evaluate whether frozen replies remain contextually appropriate and executable in evolving discussions, while explicitly identifying the remaining limitations.

We have also improved the presentation, clarified the estimands and experimental configurations, and revised the claims to better reflect the available evidence.

Overall, the revised manuscript provides stronger evidence for the scientific utility of fixed-semantics counterfactual replay, more rigorous uncertainty analysis, and clearer boundaries on the mechanistic findings.

We sincerely appreciate the reviewers' feedback, which has substantially strengthened the paper.

Sincerely,
The Authors of Submission #38459

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.