acceptodds
Under review as a conference paper at ICLR 2027

DynaCAST: Intervention-Grounded State for Long-Horizon Language Agents

Abstract

Long-horizon language agents need compact persistent state that preserves distinctions in past experience relevant to future decisions. Existing memory objectives emphasize retention or prediction without specifying which state differences should matter when goals, world conditions, or plans change. Learning such a criterion also requires comparing open-ended language actions across histories and keeping component-specific targets consistent with a global recursive metric. We introduce \method, which grounds persistent goal, world, and plan registers in controlled futures generated by executable goal, physical-world, and route interventions in resettable environments. We make these futures comparable through canonical action classes and frozen argument-binding distributions, and derive global and component Bellman targets from shared transport couplings and a common maximizing control, ensuring exact target compatibility while establishing contraction and covered-value preservation for the exact operator. Experiments across eight benchmarks with nine matched baseline/control comparisons across four model families—Qwen2.5-7B, Llama-3.1-8B, Gemma-2-9B-IT, and Mistral-7B-Instruct—isolate –-point macro-averaged task-score gains from matched finite-horizon Bellman bootstrapping over direct Monte Carlo metrics in both state architectures, with intervention and register ablations identifying the contributions of executable controls and modular state, and exact checks on 24 enumerated delivery SCMs validating operator implementation.

Primary area:
reinforcement learning
Keywords:
long-horizon language agents, agent memory, state abstraction, executable interventions, bisimulation metrics, recurrent state representations

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_m1pfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DynaCAST framework for learning compact persistent states in long-horizon language agents through separate Goal (G), World (W), and Plan (P) registers. Training uses paired futures from executable goal, physical-world, and route interventions in resettable environments. Canonical action classes and frozen argument bindings make actions comparable across histories, while shared transport couplings and a common maximizing control produce compatible global and register-specific Bellman targets. For the exact operator on the covered process, the paper establishes contraction and agent-value bounds. Experiments across eight benchmarks and four model families include existing agent and memory baselines, matched Monte Carlo and future-prediction controls, and mechanism ablations. The authors report 1.6–3.1-point macro-averaged gains from recursive bootstrapping over matched Monte Carlo objectives, alongside long-horizon, deployment coverage and calibration analyses, and exact operator checks on 24 delivery SCMs. Deployment retains the recurrent state and policy without simulator resets or transport.

Critical strengths

The paper gives persistent agent states a concrete behavioral criterion. Executable goal, world, and route interventions define relevant history differences, while canonical actions and frozen bindings make comparisons possible. A shared transport witness keeps Bellman targets consistent, with contraction and covered-value guarantees for the exact operator on the covered process. This grounds G/W/P in a formal objective rather than architectural labels alone.

The evaluation is carefully controlled. A matched 2 × 2 comparison tests recursive learning under single-state and G/W/P architectures, while DAS, intervention, and coupling ablations examine separate mechanisms. Across eight benchmarks and four model families, recursion improves aggregate scores over matched Monte Carlo objectives. Long-horizon, deployment-coverage, and finite-SCM checks provide additional validation.

Critical weaknesses

The training setup limits applicability beyond well-instrumented environments. DynaCAST requires resettable environments, validated intervention pairs, benchmark-specific adapters, and frozen action-binding distributions. Table 19 reports only 67.3% raw visited-state coverage and 84.3% final-policy visited-state coverage on the core four, despite higher policy-mass coverage. The theoretical guarantees also remain conditional: the contraction result concerns the exact covered operator, while the learned-distance bounds require approximation and missed-control errors to be bounded. Proposition 3 further requires uniform policy-shift bounds on a common forward-invariant set, which the finite-sample statistics in Table 37 do not establish. The coverage and calibration audits provide useful empirical support, but the guarantees and gains remain insufficiently established outside the evaluated instrumented settings.

The independent benefit of the Plan register is not sufficiently established. H2 measures only goal/world selectivity, while Plan-specific outcomes remain secondary. Validated Plan pairs cover only 28.6% of visited states, 22.7% of states in the longest-horizon quartile, and 12.4% of states on failed reference trajectories (Table 20). Table 43 provides a capacity-matched register ablation, but adding P to G+W improves the Core macro score by just 0.7 points, with no paired uncertainty interval reported for this difference. Closed-loop route diagnostics show that the policy uses P, but this does not establish a substantial incremental benefit from maintaining a separate Plan register. Its contribution in difficult states outside validated Plan-pair support remains unclear.

Questions for authors

H2 does not test Plan selectivity, and valid Plan pairs cover only 28.6% of visited states (Table 20). Given the 0.7-point gain over G+W (Table 43), what establishes P's independent contribution to long-horizon performance and failure recovery?

Table 19 reports 67.3% raw-state and 84.3% final-policy visited-state coverage. Does DynaCAST retain its advantage over matched baselines on tasks involving under-covered states?

Does adaptation to new environments require constructing new executable intervention pairs and adapters (E.10)? What are the associated development and simulator/executor costs beyond the reported forward-equivalent compute budget (E.21)?

6 comments
AuthorsOpenReview ↗

Response to Reviewer m1pf (1/3): independent contribution of the Plan register

Thank you for the thoughtful review. We understand your main concerns to be P's independent contribution, and the scope of the training requirements and guarantees. We address P first.

Summary of changes:

  1. Paired CI for Table 43 (Sec. 4.3), plus P analyses in App. F.2: horizon strata, route-failure recovery and Plan selectivity.
  2. Coverage strata (F.3) and native adaptation costs (F.5).
  3. Occupancy-form Proposition 3′ with an exact check (App. D, F.5), and a Scope and limitations paragraph (Sec. 6).

Given the 0.7-point gain over G+W (Table 43), what establishes P's independent contribution to long-horizon performance and failure recovery?

Short answer. We agree that P's aggregate increment is small. Its contribution concentrates where your question points: the longest-horizon quartile and recovery after route failures. Resetting P at the failure point removes more than half of Full's recovery advantage over G+W.

Paired CI for Table 43. Total capacity is fixed at 12 tokens (6+6 for G+W, 4+4+4 for G/W/P). P receives the typed attribution term. As a descendant of every intervention, it receives no invariance term (Eq. 31), and Core has no route head or route loss. The increment is therefore not explained by register capacity.

Contrast (core four)Δ macro [95% CI]
G+W → G/W/P (Core)+0.70 [0.12, 1.29] (+0.5/+0.7/+0.3/+1.3 per benchmark)
Core → Core + route extension+0.7 [0.1, 1.3]
G+W → Core + route+1.4 [0.6, 2.2]

Point estimates are positive on all four benchmarks, largest on ComplexMCP, which has the longest dependency chains.

Long horizons (frozen quartiles). P's gain grows from +0.09 [−0.72, 0.91] in Q1 to +1.58 [0.46, 2.70] in Q4, an interaction of +0.50 per quartile [0.06, 0.93]. Full − G+W in Q4 is +3.51 [2.00, 5.02].

Failure recovery (new evaluation of existing checkpoints). At 50% of the intrinsic horizon, we inject a route failure with the App. E.15 physical-world programs: a moved object, an out-of-stock item, a changed order state or a failed tool call. Every injection point admits an alternative route to the same terminal effect. Goals, injection points and prefixes are paired across methods (5 seeds).

MethodRecovery %Repeated failed actions %
G+W48.719.4
G/W/P Core51.616.9
Full54.813.4
Full, P reset at injection51.317.6

Core − G+W is +2.9 [1.0, 4.9], and resetting P lowers Full by 3.6 [1.3, 5.9], so recovery relies on the route information held in P. A reset also perturbs the state distribution, so we do not read this as P alone causing recovery.

Plan selectivity (secondary; outside H1→H2→H3). On validated plan pairs, G/W/P exceeds the capacity-matched DAS subspace in path response by +3.4 [0.9, 6.0] under MC and by +4.6 [1.7, 7.6] under recursion. Off-target G/W changes stay at 3–5%.

Outside plan support, the point estimates remain positive with wider intervals:

  • +0.43 [−0.31, 1.21] when the reference episode never reaches a supported state;
  • +0.59 [−0.54, 1.72] on failed reference trajectories;
  • +1.5 [−0.4, 3.5] for recovery outside support.

We therefore state P's established contribution as long-horizon performance and route-failure recovery within validated support (Sec. 4.3).

AuthorsOpenReview ↗

Response to Reviewer m1pf (2/3): under-covered states and adaptation cost

Does DynaCAST retain its advantage over matched baselines on tasks involving under-covered states?

Short answer: largely yes. In the least-covered third of tasks, Full's advantage over the matched G/W/P MC cell is +2.08 [0.61, 3.69], nearly the +2.27 of the best-covered third. Against the equal-budget predictor, the point estimate stays positive (+1.55), but its interval just includes zero.

Existing evidence. WA-Lite and AppWorld have the lowest final-policy coverage (88.7/79.8 and 90.4/82.4; Table 31). On these two benchmarks, Full exceeds the equal-budget predictor on all 8 backbone–benchmark pairs, by +1.6 to +3.1 points (Table 1). The Qwen paired intervals are +2.0 [0.1, 3.9] and +2.4 [0.1, 5.0] (Table 7).

New stratification (existing records; Table 57). Tasks are split into within-benchmark tertiles of uncovered-state fraction. The tertiles use frozen reference rollouts and the Table 19 criterion, and all methods share them.

Coverage tertileFull − G/W/P MC [95% CI]Full − equal-budget predictor [95% CI]
Best covered+2.27 [1.03, 3.64]+1.70 [0.51, 2.99]
Middle+2.24 [0.84, 3.83]+1.61 [0.24, 3.13]
Least covered+2.08 [0.61, 3.69]+1.55 [−0.06, 3.18]
Least covered, raw-state criterion+1.76 [0.15, 3.48]+1.24 [−0.23, 2.86]

Two existing results show how coverage gaps behave:

  • Degradation is local, not global. Withholding one intervention family degrades mainly the corresponding component: 12.1/15.7/17.2 points, against at most 3.2/4.1/2.7 off-target (App. E.8).
  • The coverage sweep marks the boundary. It favors the predictor when only 10–25% of pairs are retained (Fig. 2, right). The new Scope paragraph states this regime explicitly.

Does adaptation require new intervention pairs and adapters (E.10)? What are the development and simulator/executor costs?

Short answer. Yes. A new environment needs:

  • reset/snapshot hooks;
  • goal/world/plan intervention programs (templates are reusable);
  • canonical action classes with binding support;
  • router labels.

The recurrent update and metric LoRA are reused. With 10% target data, core pretraining beats training from scratch on all four target benchmarks (Table 25; e.g., AppWorld 32.1 vs 29.7, WA-Lite 27.6 vs 24.9). Deployment needs no simulator, reset or transport.

Native costs (adapter code, build logs and time records; one backbone; one continuation cache shared by all factorial cells; 8 CPU workers; Table 61)

Per benchmarkCore-four meanExpansion mean
Adapter LOC9561,463
Person-days3.25.0
Candidate-pair rejection19.1%22.9%
Simulator/executor CPU·h192252
CF generation / total training wall-clock (h)30.3 / 63.343.8 / 83.4

Across all eight benchmarks, the means are 4.1 person-days and 222 CPU·h, paid once per environment.

In the paper: App. F.3 (coverage strata) and App. F.5 (Tables 61–62: per-benchmark costs and component checklist).

AuthorsOpenReview ↗

Response to Reviewer m1pf (3/3): instrumented training and conditional guarantees

We agree with this characterization. Training requires resettable environments and validated interventions, and the guarantees are stated for the exact covered process. To make these conditions explicit, we have made three changes:

  • (i) removed the uniform policy-shift suprema from Proposition 3;
  • (ii) added an exact check of the bound;
  • (iii) added a Scope and limitations paragraph to the main text.

(i) Proposition 3′, occupancy form (App. D, new). We agree that the visitation maxima in Table 37 do not establish population suprema; App. D notes the same. We therefore derived a version that does not need them:

∣Vπ(ω)−Vπ(ω′)∣≤d~ϕ(ω,ω′)+ϵres+ϵest+ϵmiss1−γ+ϵˉ(ω)+ϵˉ(ω′)1−γ.|V^\pi(\omega)-V^\pi(\omega')|\le \tilde d_\phi(\omega,\omega')+\frac{\epsilon_{\rm res}+\epsilon_{\rm est}+\epsilon_{\rm miss}}{1-\gamma}+\frac{\bar\epsilon(\omega)+\bar\epsilon(\omega')}{1-\gamma}.

Here ϵˉ(ω)\bar\epsilon(\omega) is the expectation of uncovered mass, class shift and binding shift under the deployed policy's normalized discounted occupancy from ω\omega.

  • Proof. The proof applies Prop. 3's pointwise TV decomposition inside the performance-difference lemma. This removes the common forward-invariant set and the policy-shift suprema.
  • Never looser than Prop. 3, since ϵˉ≤ϵout+ϵclass+ϵbind\bar\epsilon\le\epsilon_{\rm out}+\epsilon_{\rm class}+\epsilon_{\rm bind}.
  • What remains. The approximation terms remain sup-norm quantities, and the reference kernel must be defined on the deployed occupancy support. On benchmarks, both forms serve as diagnostics rather than numerical certificates.

As you note, final-policy class-mass coverage (93.1%) exceeds raw-state coverage. This mass-based notion is the one that enters Prop. 3, through its uncovered-mass term.

(ii) Exact check (24 delivery SCMs, 36–120 states; Table 63). We used exhaustive controls, singleton bindings, and a class-policy perturbation of TV at most 0.012. The bound holds on every state pair at γ = .80/.90/.95, with median LHS/RHS of 0.52/0.39/0.24. This verifies the implementation against the theorem.

(iii) Scope and limitations (Sec. 6, new).

"DynaCAST requires resettable training environments, validated intervention pairs, and benchmark-specific adapters (native costs in Appendix F.5); deployment requires none of them. Its guarantees concern the exact covered process; on benchmarks they serve as diagnostics, and Proposition 3′ replaces uniform policy-shift suprema by occupancy-weighted terms. Exact shared-witness additivity does not by itself make the attributed register targets jointly realizable (Proposition 2′). When admissible pairs are sparse (10–25% retention, Figure 2), the matched predictor is preferable."

We see instrumented training as the cost of a state criterion with behavioral meaning. It is paid once per environment, at training time, and the deployed agent then carries only three recurrent registers. We hope the revisions make this trade-off and its boundaries transparent, and we are happy to clarify anything further.

Reviewer_3rjYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DynaCAST learns persistent goal, world, and plan registers from executable interventions using a recursive bisimulation objective. Canonical action classes and frozen argument bindings make histories comparable, while shared transport witnesses connect the global objective to register-specific targets. The paper evaluates the approach across eight benchmarks and four backbones, with matched ablations and finite-state checks.

Critical strengths
  1. The paper carefully distinguishes executable environment interventions from internal register swaps. This gives the proposed register semantics a concrete behavioral basis, rather than relying on architectural labels.

  2. The experimental design includes relevant controls for supervision, capacity, and compute. In particular, the matched Monte Carlo objective, identity-only interventions, and equal-budget predictor are useful comparisons for assessing what the recursive objective contributes.

Critical weaknesses
  1. Several central effect estimates do not agree. Table 10 reports Qwen’s all-eight recursion gains as 1.8/1.6 points, while Table 11 reports 2.2/2.0 with different confidence intervals. It is unclear whether these are separate runs. Table 44 also reports an MC-versus-recursive contrast of −1.5 despite scores of 66.7 and 68.0. Under the stated common paired units and weights, pairing does not explain this difference, and rounding is insufficient. These discrepancies need to be resolved before I can rely on the confirmatory claim, especially given its one-point effect threshold.

  2. The long-horizon analysis does not isolate the source of the gains. Figure 3 and Table 17 compare Full with ReBel, factorized state without interventions, and Static CAST, but omit the matched Monte Carlo control and equal-budget predictor. These comparisons change intervention supervision, auxiliary objectives, or compute alongside recursion. The aggregate H1 results suggest a bootstrap benefit, but do not establish that this benefit grows with horizon. This leaves the paper’s explanation of its largest gains insufficiently supported.

Questions for authors

NA

4 comments
AuthorsOpenReview ↗

Response to Reviewer 3rjY (1/2): Tables 10, 11 and 44 reconciled; the confirmatory claim holds

Thank you for checking these numbers so carefully. You are right about both discrepancies. Since you made your reliance on the confirmatory claim conditional on resolving them, we address them first; the corrected values and their consequence for H1 follow.

Summary of changes:

  1. Tables 10, 11, 44 and Fig. 4 are regenerated from one paired protocol, and Table 10's caption now states the averaging identity.
  2. Sec. 4.3 reports the one-sided bounds.
  3. Fig. 3 and App. F.4 now include the matched MC and equal-budget-predictor controls (Part 2).

Source of the discrepancies. Tables 10 and 11 report the same five-seed Qwen2.5 confirmation runs, not separate replications. Some summary rows had been carried over from an earlier analysis version and were not updated with the per-benchmark scores: the Table 10 macro row, the Qwen rows of Table 11, and the Table 44 Δ column, whose four entries all point to an earlier reference of 68.2 (65.9+2.3 = 66.7+1.5 = 67.1+1.1 = 67.4+0.8). As you note, neither pairing nor rounding can explain −1.5, and we have removed the caption sentence suggesting otherwise.

Reconciled values (from the per-benchmark scores of Table 10)

Qwen2.5, recursive − matched MCOriginalReconciled
All-eight, single / G/W/P1.8 (T10), 2.2 (T11) / 1.6, 2.01.90 [1.45, 2.34] / 1.63 [1.17, 2.07]
One-sided lower bounds1.1 / 1.01.53 / 1.25
Core-four, single / G/W/P1.8 / 1.5 (T11)1.45 [0.47, 2.45] / 1.30 [0.27, 2.33]
Tab. 44, MC − shared recursive−1.5−1.3 [−2.33, −0.27]

These can be checked from the displayed cells. The all-eight effects are 15.2/8 = 1.90 and 13.0/8 = 1.625, and the core-four effects are 5.8/4 = 1.45 and 5.2/4 = 1.30; Table 44's MC row is the core-four G/W/P contrast with the sign reversed (66.675 vs 67.975). The core-four values already matched Table 2 and Sec. 4.3. All intervals come from one protocol (5,000 crossed task–seed resamples, shared across methods and benchmarks) and agree to within 0.02 with the normal approximation from the paired standard errors (0.23/0.23 all-eight); for example, 1.625 − 1.645 × 0.228 = 1.25.

Confirmatory claim. Sec. 4.3 now reads:

"Bellman bootstrapping improves on the matched MC metric in both architectures (all-eight 1.90 [1.45, 2.34] single; 1.63 [1.17, 2.07] G/W/P). H1 clears its one-point margin in both architectures (one-sided lower bounds 1.53 and 1.25; joint p = 0.0035), and the three replication backbones give 2.0–3.1-point effects whose two-sided 95% intervals lie entirely above one point in both architectures (Appendix Table 11)."

H2 and H3 then pass under the fixed sequence with unchanged bounds. The core four, reported separately from H1, exclude zero but do not clear one point on their own (lower bounds 0.62/0.43), as App. E.3 now states.

Other aggregates. We rechecked every other summary in the paper against its per-benchmark table and corrected three of the same kind: the executable-control effect (2.1 → 1.8 [1.0, 2.7]), the shared-witness task gain (1.1 → 0.9 [0.12, 1.55]) and the six-benchmark sensitivity (2.04/1.70 → 2.47/2.20). The same five leave-one-out omissions pass Holm adjustment as before. The per-benchmark scores, the other three backbones and the headline Full-vs-predictor gain (1.7 [0.9, 2.6]) are unchanged.

AuthorsOpenReview ↗

Response to Reviewer 3rjY (2/2): source of the long-horizon gains with matched controls

Figure 3 and Table 17 … omit the matched Monte Carlo control and equal-budget predictor … do not establish that this benefit grows with horizon.

We agree that the original comparisons changed several factors at once. We have added the two controls you identified to the horizon analysis. The analysis uses:

  • the existing five-seed records;
  • the method-independent quartiles frozen before evaluation;
  • the paired task–seed resampling of Fig. 3.

Each method's four quartiles average exactly to its core-four macro.

Matched contrasts by quartile (core four, Qwen2.5; slope = −0.3ΔQ1 − 0.1ΔQ2 + 0.1ΔQ3 + 0.3ΔQ4; Table 59)

Contrast (what changes)Q1 ΔQ4 ΔSlope per quartile [95% CI]
Recursion, G/W/P (bootstrap only)+0.24+2.77+0.83 [0.18, 1.48]
Recursion, single state (bootstrap only)+0.29+2.89+0.85 [0.14, 1.57]
Executable vs identity-only controls+0.06+3.19+1.06 [0.33, 1.79]
Full vs equal-budget predictor+0.12+4.12+1.28 [0.34, 2.23]
CF data + MC objective vs Factorized+3.86+10.01+1.75 [0.54, 2.97]

The growing bootstrap effect is not a longer-lookahead artifact. Both H1 cells use the same four-step truncated target, discount weights and per-step control re-selection (Eq. 8). Recursion differs from the matched MC metric only in the EMA bootstrap.

Where the Q4 gap comes from. Full's Q4 gap over Factorized is 14.73 points from unrounded quartile means (14.8 from the rounded Table 17 entries). Along the ordered ladder Factorized → G/W/P MC → Global → Core → Full, the gap splits as follows:

  • executable counterfactual data with the matched MC objective: 10.0 (68%);
  • recursion: 2.8 (19%);
  • route/sensitivity: 1.9;
  • attribution/invariance: ≈0.

The split depends on the ladder order, and its first step changes data and objective together.

Takeaway:

  • The bootstrap effect is small at short horizons and largest in Q4, in both architectures. The aggregate H1 contrast could not show this trend.
  • Executable counterfactual learning supplies most of the long-horizon advantage.
  • Full's lead over the equal-budget predictor also grows, so the trend is not a compute artifact.

We have revised the explanation of our largest gains accordingly (Sec. 4.4, Sec. 5, Conclusion).

Robustness (App. F.4, Table 60; outside the prespecified confirmatory family, not multiplicity-adjusted):

  • With milestone-count quartiles, the recursion slopes are +0.76 [0.08, 1.47] (G/W/P) and +0.81 [0.04, 1.63] (single).
  • Slopes are positive on 3/4 benchmarks (G/W/P) and 4/4 (single).
  • The sign is the same, with wider intervals, for Llama (+0.72 [−0.04, 1.50] / +0.79 [0.02, 1.61]), for ceiling-adjusted relative gains (+1.82 / +1.70, crossing zero) and for the low-aliasing stratum.

In the paper:

  • Fig. 3 adds the matched-MC, matched-recursive and equal-budget-predictor curves, and its right panel shows the matched contrasts.
  • App. F.4 (Tables 58–60) gives all strata, contrasts and the ordered decomposition.

We hope this resolves both concerns, and we are happy to provide further breakdowns.

Thank you for the detailed response and the efforts to address my concerns. The changes and clarifications have addressed my main concerns. I will maintain my positive assessment.

Thank you for the follow-up, and for checking our numbers so carefully. Your two points, the Tables 10/11/44 summaries and the missing matched controls in the horizon analysis, made both the confirmatory claim and the long-horizon explanation more precise. Both changes are in the revised PDF (Sec. 4.3, Table 10 caption, Fig. 3, App. F.4). We appreciate your positive assessment.

Reviewer_ozAaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

DynaCAST learns compact goal, world, and plan registers for long-horizon language agents using executable interventions in resettable environments. Its contributions are an intervention-indexed behavioral distance, canonical actions with frozen argument bindings, and shared optimal-transport couplings that make component targets sum to a global Bellman target. The paper proves contraction and value preservation for the exact covered process. Validation spans eight benchmarks, four model families, matched objective/architecture ablations, intervention and deployment diagnostics, and 24 enumerated delivery models. It reports 1.6–3.1-point gains from bootstrapping and a 1.7-point Qwen gain over an equal-budget predictor.

Critical strengths
  1. A precise formulation of intervention-grounded memory. The distinction between executable environment changes and internal register swaps is particularly valuable. Canonical controls and frozen bindings make comparisons across histories well-defined, while shared couplings ensure exact target additivity. This gives the memory objective a concrete behavioral meaning beyond assigning semantic names to registers.
  2. A substantial effort to isolate the mechanisms. The factorial design matches counterfactual data, routed information, capacity, and computation, while excluding component-specific semantic losses from the primary comparison. Physical-intervention, swap-recovery, and horizon analyses supplement task scores. These controls address consequential alternative explanations, especially gains from additional supervision.
Critical weaknesses
  1. Shared-target additivity does not guarantee that the component targets are jointly learnable. Equation 26 constrains each register distance to be an L1 pseudometric. However, Equation 30 constructs its target using a maximizing control and transport coupling chosen from the entire state pair. Different pairs can therefore select different witnesses, producing component targets such as T_{G}(A,B)=T_{G}(B,C)=0, T_{G}(A,C)>0, which violate the triangle inequality required of the learned component distance. Consequently, the component objectives can have irreducible conflicts even with exact continuations and unlimited model capacity. The shared-witness sum identity remains correct, but it establishes arithmetic consistence, not that the proposed registers can simultaneously represent those targets. This is consequential because compatible component learning is a central methodological contribution.
Questions for authors
  1. Which runs and task-level records reconcile Tables 10, 11, and 44, and do the corrected H1 estimates and confidence bounds still exceed the prespecified threshold?
3 comments
AuthorsOpenReview ↗

Response to Reviewer ozAa (1/2): additivity vs. joint realizability of component targets

Thank you for this precise observation; we agree with it. We understand it to be your main concern, so we address it first; your question on Tables 10/11/44 and the presentation points follow in Part 2.

Summary of changes:

  1. Claims now state exact additive attribution, not compatibility (Abstract, Contribution 2, Fig. 1, Secs. 2.2, 3, 4.3, 6).
  2. New Proposition 2′ (App. D) bounds the triangle obstruction and gives a closure correction.
  3. New audits and a typed-term ablation (App. F.1, Tables 49–52).

Short answer. Your counterexample is correct, and the revision formalizes it as Proposition 2′. Equation 30 selects the control and coupling for each pair from the global cost. The component targets TgT_g are therefore exact per-pair summands of the global Bellman target, but they need not satisfy the triangle inequality. When they violate it, no bounded-L1 pseudometric of Eq. 26 fits them exactly, even with exact continuations and unlimited capacity. Proposition 2 thus establishes additive attribution, not joint realizability. Our phrases "compatible component targets", "cannot drift into mutual inconsistency" and "agree by construction" overstated this, and we have corrected them.

What the shared witness still provides. It removes the conflict that independently maximized typed operators would create with the global fixed point (App. C, after Eq. 26). Theorems 1–4 concern the global operator and do not assume component realizability. As you noted, the H1 factorial excludes component-specific losses, so this issue does not enter it.

(i) Proposition 2′. Let TgT_g be nonnegative, symmetric and zero-diagonal on a finite support; practical targets are symmetrized and zero-diagonalized first. Let δg=max⁡A,B,C[Tg(A,C)−Tg(A,B)−Tg(B,C)]+\delta_g=\max_{A,B,C}[T_g(A,C)-T_g(A,B)-T_g(B,C)]_+. Then every pseudometric dd satisfies ∥d−Tg∥∞≥δg/3\|d-T_g\|_\infty\ge\delta_g/3. The shortest-path closure Tˉg\bar T_g is the largest pseudometric below TgT_g. It is a one-sided, per-batch correction, not the sup-norm projection. Closure removes only the triangle obstruction: exact bounded-L1 representation would further require ℓ1-embeddability, which we do not claim and instead measure with the encoder-free fit in (iii).

(ii) Optional correction. The typed term can instead regress to Tˉg\bar T_g while the global term still regresses to YY. With r=Y−∑gβgTˉg≥0r=Y-\sum_g\beta_g\bar T_g\ge 0, we have ∣Dϕ−Y∣≤∑gβg∣dϕg−Tˉg∣+r|D_\phi-Y|\le\sum_g\beta_g|d^g_\phi-\bar T_g|+r (Eq. 58b). All five-seed results use the original TgT_g, which sums exactly to YY; the closure variant is evaluated in (iv), and we report rr.

(iii) Size of the obstruction

SettingViolating triples G / W / PUnattributed mass rr
24 exact SCMs0.03% / 0.00% / 0.88–0.95%1.35–3.56% (γ = .95 to .80)
Qwen2.5 core-four training batches0.08% / 0.03% / 1.28%1.9%
  • The global target has no triangle violations in all 72 exact settings.
  • Violations concentrate in P (single-SCM maximum 6.65%).
  • In an encoder-free fit (128 free parameters per register and state), typed targets lower P's sup error from 0.386 to 0.168 (original) and 0.176 (closure), while global RMSE moves only from 0.0264 to 0.0275 and 0.0291.

(iv) Behavioral consequence. The original five-seed ladder already shows the pattern (Table 13). Adding Bellman attribution to Global raises physical-do from 85.8 to 87.0 and selectivity from 82.0 to 83.4, at a 0.2-point task change, and Core (attribution + invariance) returns task to 68.0. New three-seed runs vary only the typed term (Table 52):

Paired Δ [95% CI]TaskG/W selectivity3-step recoveryPhysical-do
Original typed − no typed−0.3 [−0.9, 0.2]+1.6 [0.5, 2.8]+1.8 [0.6, 3.1]+1.1 [0.2, 2.1]
Closure typed − no typed−0.1 [−0.6, 0.5]+1.9 [0.7, 3.2]+1.7 [0.4, 3.0]+1.5 [0.5, 2.6]

Takeaway: the obstruction you identified is real but small, and typed attribution yields clear component-level gains with or without the closure correction, with no detectable change in task score.

AuthorsOpenReview ↗

Response to Reviewer ozAa (2/2): H1 after reconciliation, and presentation

Which runs and task-level records reconcile Tables 10, 11, and 44, and do the corrected H1 estimates and confidence bounds still exceed the prespecified threshold?

Short answer: yes. Both corrected effects exceed the prespecified one-point threshold:

All-eight, Qwen2.5Effect [95% CI]One-sided 95% lower boundp (Δ ≤ 1)
Single state1.90 [1.45, 2.34]1.53<0.0001
G/W/P1.63 [1.17, 2.07]1.250.0035

H1 is confirmed (joint p = 0.0035), and H2 and H3 pass with unchanged bounds under the fixed sequence. On all three replication backbones, both all-eight two-sided 95% intervals lie entirely above one point (Table 11).

Runs and records. Tables 10, 11 and 44 report the same five-seed Qwen2.5 confirmation runs: task and seed are crossed, environment replicates are nested within each cell, and ComplexMCP's 47 scenarios are the paired units (task counts in App. E.1). Some summary rows had been carried over from an earlier analysis version; all summaries are now regenerated from the per-benchmark scores of Table 10 with one paired protocol, and each point estimate is the mean of the displayed per-benchmark effects (General Response).

Presentation. We made targeted edits rather than restructuring:

  • Roadmap (Sec. 2): what is learned, how its targets are built (Eqs. 4–6), and what is guaranteed (Sec. 3).
  • Running example (Sec. 2.2, new paragraph): the delivery task, with two observation-identical histories that differ in a hidden lock. It shows how executable controls separate them and how Eq. 6 splits that separation across G/W/P, in contrast to an internal swap.
  • Shared-witness paragraph: now separates additive attribution from realizability, and Sec. 3 reports the measured violation rates.

Where the main findings are. To make the contribution easier to locate:

  1. Executable interventions vs. factual bisimulation. On the same recursive learner, restricting controls to identity (factual) bisimulation lowers task from 68.0 to 66.6, physical-do from 85.8 to 74.1 and selectivity from 82.0 to 73.5 (Table 2).
  2. Bootstrapping vs. matched MC. The effect is confirmed (H1) and grows with horizon (+0.83 per quartile; App. F.4).
  3. Learning object vs. equal-budget prediction. On the same continuations and budget, Full beats prediction in task (+1.7 [0.9, 2.6]), physical-do (89.1 vs 83.1) and three-step recovery (84.1 vs 74.6) (Fig. 2).

We hope these changes address your concerns, in particular the realizability of component targets. We would be glad to clarify anything further.

AuthorsOpenReview ↗

Brief summary of how the revision addresses your review

Dear Reviewer ozAa,

As the discussion period closes soon, we would like to briefly summarize how the revision addresses your review.

1. Additivity vs. joint realizability (your main concern). We agree with your counterexample. The paper now claims exact additive attribution, not compatibility, throughout (Abstract, Contribution 2, Secs. 2.2, 3, 4.3, 6).

  • Proposition 2′ (App. D) formalizes the obstruction: every pseudometric is at least δ_g/3 from T_g in sup norm. The shortest-path closure gives an optional one-sided correction.
  • The obstruction is small in practice: 0.08% / 0.03% / 1.28% of G / W / P triples on Qwen2.5 core-four batches, and none for the global target in all 72 exact settings (App. F.1).
  • With or without the closure, the typed term improves G/W selectivity (+1.6 / +1.9), 3-step recovery (+1.8 / +1.7) and physical-do (+1.1 / +1.5), with no detectable task change (Table 52).

2. Your question on Tables 10/11/44. Yes, H1 still clears the one-point threshold: 1.90 [1.45, 2.34] (single state) and 1.63 [1.17, 2.07] (G/W/P), with one-sided lower bounds of 1.53 / 1.25. Every point estimate can be recomputed from the per-benchmark cells of Table 10 (e.g., 15.2/8 = 1.90).

3. Presentation. Sec. 2 now opens with a roadmap, and Sec. 2.2 adds a running example: two observation-identical delivery histories that differ in a hidden lock, separated by executable controls but not by an internal register swap.

We would be grateful to know whether these changes address your concerns, in particular the realizability point, and we are happy to clarify anything further.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response: summary of revisions and main conclusions

We thank the reviewers for careful and constructive reviews. They found the distinction between executable environment interventions and internal register swaps to be the paper's central strength (ozAa, 3rjY, m1pf), and the matched objective, supervision and compute controls useful for isolating what recursion contributes (ozAa, 3rjY, m1pf). We have uploaded a revised PDF with all changes in blue. New analyses are collected in Appendix F (Analyses Added in Rebuttal), and all original table and equation numbers are preserved.

Main conclusions after revision

  • Bootstrapping is confirmed against its matched MC control (H1). The gain is 1.90 [1.45, 2.34] for a single state and 1.63 [1.17, 2.07] for G/W/P; both one-sided lower bounds clear the one-point margin (1.53 and 1.25; joint p = 0.0035).
  • The bootstrap effect grows with horizon under matched controls: +0.83 [0.18, 1.48] per horizon quartile (App. F.4).
  • The end-system gain is unchanged. Full exceeds the equal-budget predictor by 1.7 [0.9, 2.6] on Qwen2.5 and by 1.8/1.8/2.6 on the other backbones.
  • Component realizability is now quantified. Triangle violations are rare: on average 0.08%/0.03% of G/W triples and 0.9–1.3% of P triples (App. F.1).
  • P's contribution now has paired intervals: +0.70 [0.12, 1.29] over G+W at fixed capacity, +1.58 [0.46, 2.70] in the longest-horizon quartile, and a 3.6-point recovery loss when P is reset (App. F.2).

Changes by reviewer

  1. Additivity vs. joint realizability (ozAa): new Proposition 2′ with a closure correction (App. D), audits in F.1, and a running example in Sec. 2.2.
  2. Matched MC and equal-budget-predictor controls added to the long-horizon analysis (3rjY): Fig. 3 and App. F.4.
  3. Plan register and scope (m1pf):
    • P analyses in F.2: paired CI, horizon strata, route-failure recovery and Plan selectivity;
    • coverage strata (F.3) and native adaptation cost (F.5);
    • occupancy-form Proposition 3′ (App. D);
    • a Scope and limitations paragraph (Sec. 6).
  4. Summary rows of Tables 10/11/44 reconciled (3rjY, ozAa); see below.

Tables 10/11/44. These tables report the same five-seed Qwen2.5 runs. Some summary rows had been carried over from an earlier analysis version and were not updated with the per-benchmark scores. We regenerated all summaries and intervals from the per-benchmark scores of Table 10 with one paired protocol, and rechecked every other aggregate in the paper (details in our reply to 3rjY).

Qwen2.5, recursive − matched MCReconciled [95% CI]
All-eight, single / G/W/P1.90 [1.45, 2.34] / 1.63 [1.17, 2.07]
Core-four, single / G/W/P1.45 [0.47, 2.45] / 1.30 [0.27, 2.33]
Tab. 44, MC − shared recursive−1.3 [−2.33, −0.27]

Each point estimate is the mean of the displayed per-benchmark effects (e.g., 15.2/8 = 1.90), as Table 10's caption now states.

Unchanged: the per-benchmark scores of Tables 1, 10–13, 42 and 43; the Llama/Gemma/Mistral replications (Table 11); and the physical-do, selectivity, coverage and calibration results.

AuthorsOpenReview ↗

Summary of the discussion

Dear AC and reviewers,

As the author discussion closes, we summarize where each concern stands. All revisions are in blue in the PDF; new analyses are in Appendix F.

What the reviewers agree on. All three reviewers identify the same core contribution: state grounded in executable environment interventions rather than internal register swaps. They describe it as "particularly valuable" (ozAa), "a concrete behavioral basis, rather than relying on architectural labels" (3rjY), and "a concrete behavioral criterion" (m1pf). They also credit the matched controls: "the evaluation is carefully controlled" (m1pf); the controls "address consequential alternative explanations, especially gains from additional supervision" (ozAa).


3rjY: summary-row consistency (Tables 10/11/44) and the source of long-horizon gains. Resolved, confirmed by the reviewer.

We added the matched MC control and the equal-budget predictor to the horizon analysis. Under these controls, the bootstrap effect grows with horizon: +0.83 [0.18, 1.48] per quartile for G/W/P and +0.85 [0.14, 1.57] for a single state (App. F.4). The paper now attributes most of the long-horizon advantage to executable counterfactual learning. 3rjY: "The changes and clarifications have addressed my main concerns."

ozAa: additivity does not imply jointly realizable component targets. Accepted; the claim is narrowed.

The counterexample is correct. The paper now claims exact additive attribution, not compatibility, throughout. Proposition 2′ (App. D) shows that every pseudometric is at least δ_g/3 from T_g in sup norm, and it gives an optional closure correction. The obstruction is measurably small:

  • 0.08% / 0.03% / 1.28% of G/W/P triples violate the triangle inequality;
  • the global target has no violations in any of the 72 exact settings.

The component-level gains hold with or without the correction (Table 52).

This point concerns the wording of Contribution 2, not the main conclusions. As ozAa's review notes, the primary factorial excludes "component-specific semantic losses from the primary comparison", and Theorems 1–4 concern the global operator only.

m1pf: the independent role of P, and the scope of training requirements and guarantees. Addressed with new paired analyses and an explicit scope statement.

  • P: at fixed capacity, P adds +0.70 [0.12, 1.29] over G+W, and +1.58 [0.46, 2.70] in the longest-horizon quartile. Resetting P at an injected route failure lowers recovery by 3.6 [1.3, 5.9] points (App. F.2).
  • Under-covered states: in the least-covered tertile, Full keeps +2.08 [0.61, 3.69] over matched MC (App. F.3).
  • Adaptation cost: 4.1 person-days and 222 CPU·h per environment, paid once; deployment needs no simulator (App. F.5).
  • Theory: Proposition 3′ replaces the uniform policy-shift suprema with occupancy-weighted terms.

Main results after revision

  • H1, bootstrapping vs. matched MC:
    • 1.90 [1.45, 2.34] (single state) and 1.63 [1.17, 2.07] (G/W/P);
    • one-sided lower bounds of 1.53 and 1.25 clear the prespecified one-point margin;
    • the three replication backbones, Llama, Gemma and Mistral, are unchanged by the revision and give 2.0–3.1-point effects, with two-sided 95% intervals entirely above one point (Table 11).
  • Learning object vs. prediction: Full exceeds the equal-budget predictor by +1.7 [0.9, 2.6] on Qwen2.5 and by 1.8/1.8/2.6 on the other backbones.
  • Executable vs. identity-only controls: on the same recursive learner, identity-only controls lower physical-do from 85.8 to 74.1 and selectivity from 82.0 to 73.5 (Table 2).

What we do not claim (Sec. 6):

  • Training requires resettable environments, validated interventions and adapters; deployment requires none of them.
  • The guarantees hold for the exact covered process and serve as diagnostics on benchmarks.
  • Additivity alone does not give joint realizability.
  • With sparse admissible pairs (10–25% retention), the matched predictor is preferable.
  • P's contribution is established within validated plan support.
  • The core four alone do not clear the one-point margin (lower bounds 0.62 / 0.43).

Tables 10/11/44. These tables report the same five-seed runs. Some summary rows came from an earlier analysis version. They are now regenerated with one paired protocol from the per-benchmark scores of Table 10, which are unchanged, so every point estimate can be recomputed from the displayed cells (e.g., 15.2/8 = 1.90). 3rjY, who identified the discrepancies, confirmed the resolution.

We thank the reviewers for their careful reading. ozAa's counterexample and 3rjY's checks have made the paper's claims more precise.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.