- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DynaCAST framework for learning compact persistent states in long-horizon language agents through separate Goal (G), World (W), and Plan (P) registers. Training uses paired futures from executable goal, physical-world, and route interventions in resettable environments. Canonical action classes and frozen argument bindings make actions comparable across histories, while shared transport couplings and a common maximizing control produce compatible global and register-specific Bellman targets. For the exact operator on the covered process, the paper establishes contraction and agent-value bounds. Experiments across eight benchmarks and four model families include existing agent and memory baselines, matched Monte Carlo and future-prediction controls, and mechanism ablations. The authors report 1.6–3.1-point macro-averaged gains from recursive bootstrapping over matched Monte Carlo objectives, alongside long-horizon, deployment coverage and calibration analyses, and exact operator checks on 24 delivery SCMs. Deployment retains the recurrent state and policy without simulator resets or transport.
Critical strengths
The paper gives persistent agent states a concrete behavioral criterion. Executable goal, world, and route interventions define relevant history differences, while canonical actions and frozen bindings make comparisons possible. A shared transport witness keeps Bellman targets consistent, with contraction and covered-value guarantees for the exact operator on the covered process. This grounds G/W/P in a formal objective rather than architectural labels alone.
The evaluation is carefully controlled. A matched 2 × 2 comparison tests recursive learning under single-state and G/W/P architectures, while DAS, intervention, and coupling ablations examine separate mechanisms. Across eight benchmarks and four model families, recursion improves aggregate scores over matched Monte Carlo objectives. Long-horizon, deployment-coverage, and finite-SCM checks provide additional validation.
Critical weaknesses
The training setup limits applicability beyond well-instrumented environments. DynaCAST requires resettable environments, validated intervention pairs, benchmark-specific adapters, and frozen action-binding distributions. Table 19 reports only 67.3% raw visited-state coverage and 84.3% final-policy visited-state coverage on the core four, despite higher policy-mass coverage. The theoretical guarantees also remain conditional: the contraction result concerns the exact covered operator, while the learned-distance bounds require approximation and missed-control errors to be bounded. Proposition 3 further requires uniform policy-shift bounds on a common forward-invariant set, which the finite-sample statistics in Table 37 do not establish. The coverage and calibration audits provide useful empirical support, but the guarantees and gains remain insufficiently established outside the evaluated instrumented settings.
The independent benefit of the Plan register is not sufficiently established. H2 measures only goal/world selectivity, while Plan-specific outcomes remain secondary. Validated Plan pairs cover only 28.6% of visited states, 22.7% of states in the longest-horizon quartile, and 12.4% of states on failed reference trajectories (Table 20). Table 43 provides a capacity-matched register ablation, but adding P to G+W improves the Core macro score by just 0.7 points, with no paired uncertainty interval reported for this difference. Closed-loop route diagnostics show that the policy uses P, but this does not establish a substantial incremental benefit from maintaining a separate Plan register. Its contribution in difficult states outside validated Plan-pair support remains unclear.
Questions for authors
H2 does not test Plan selectivity, and valid Plan pairs cover only 28.6% of visited states (Table 20). Given the 0.7-point gain over G+W (Table 43), what establishes P's independent contribution to long-horizon performance and failure recovery?
Table 19 reports 67.3% raw-state and 84.3% final-policy visited-state coverage. Does DynaCAST retain its advantage over matched baselines on tasks involving under-covered states?
Does adaptation to new environments require constructing new executable intervention pairs and adapters (E.10)? What are the associated development and simulator/executor costs beyond the reported forward-equivalent compute budget (E.21)?
6 comments
Response to Reviewer m1pf (2/3): under-covered states and adaptation cost
Does DynaCAST retain its advantage over matched baselines on tasks involving under-covered states?
Short answer: largely yes. In the least-covered third of tasks, Full's advantage over the matched G/W/P MC cell is +2.08 [0.61, 3.69], nearly the +2.27 of the best-covered third. Against the equal-budget predictor, the point estimate stays positive (+1.55), but its interval just includes zero.
Existing evidence. WA-Lite and AppWorld have the lowest final-policy coverage (88.7/79.8 and 90.4/82.4; Table 31). On these two benchmarks, Full exceeds the equal-budget predictor on all 8 backbone–benchmark pairs, by +1.6 to +3.1 points (Table 1). The Qwen paired intervals are +2.0 [0.1, 3.9] and +2.4 [0.1, 5.0] (Table 7).
New stratification (existing records; Table 57). Tasks are split into within-benchmark tertiles of uncovered-state fraction. The tertiles use frozen reference rollouts and the Table 19 criterion, and all methods share them.
| Coverage tertile | Full − G/W/P MC [95% CI] | Full − equal-budget predictor [95% CI] |
|---|---|---|
| Best covered | +2.27 [1.03, 3.64] | +1.70 [0.51, 2.99] |
| Middle | +2.24 [0.84, 3.83] | +1.61 [0.24, 3.13] |
| Least covered | +2.08 [0.61, 3.69] | +1.55 [−0.06, 3.18] |
| Least covered, raw-state criterion | +1.76 [0.15, 3.48] | +1.24 [−0.23, 2.86] |
Two existing results show how coverage gaps behave:
- Degradation is local, not global. Withholding one intervention family degrades mainly the corresponding component: 12.1/15.7/17.2 points, against at most 3.2/4.1/2.7 off-target (App. E.8).
- The coverage sweep marks the boundary. It favors the predictor when only 10–25% of pairs are retained (Fig. 2, right). The new Scope paragraph states this regime explicitly.
Does adaptation require new intervention pairs and adapters (E.10)? What are the development and simulator/executor costs?
Short answer. Yes. A new environment needs:
- reset/snapshot hooks;
- goal/world/plan intervention programs (templates are reusable);
- canonical action classes with binding support;
- router labels.
The recurrent update and metric LoRA are reused. With 10% target data, core pretraining beats training from scratch on all four target benchmarks (Table 25; e.g., AppWorld 32.1 vs 29.7, WA-Lite 27.6 vs 24.9). Deployment needs no simulator, reset or transport.
Native costs (adapter code, build logs and time records; one backbone; one continuation cache shared by all factorial cells; 8 CPU workers; Table 61)
| Per benchmark | Core-four mean | Expansion mean |
|---|---|---|
| Adapter LOC | 956 | 1,463 |
| Person-days | 3.2 | 5.0 |
| Candidate-pair rejection | 19.1% | 22.9% |
| Simulator/executor CPU·h | 192 | 252 |
| CF generation / total training wall-clock (h) | 30.3 / 63.3 | 43.8 / 83.4 |
Across all eight benchmarks, the means are 4.1 person-days and 222 CPU·h, paid once per environment.
In the paper: App. F.3 (coverage strata) and App. F.5 (Tables 61–62: per-benchmark costs and component checklist).
Response to Reviewer m1pf (3/3): instrumented training and conditional guarantees
We agree with this characterization. Training requires resettable environments and validated interventions, and the guarantees are stated for the exact covered process. To make these conditions explicit, we have made three changes:
- (i) removed the uniform policy-shift suprema from Proposition 3;
- (ii) added an exact check of the bound;
- (iii) added a Scope and limitations paragraph to the main text.
(i) Proposition 3′, occupancy form (App. D, new). We agree that the visitation maxima in Table 37 do not establish population suprema; App. D notes the same. We therefore derived a version that does not need them:
Here is the expectation of uncovered mass, class shift and binding shift under the deployed policy's normalized discounted occupancy from .
- Proof. The proof applies Prop. 3's pointwise TV decomposition inside the performance-difference lemma. This removes the common forward-invariant set and the policy-shift suprema.
- Never looser than Prop. 3, since .
- What remains. The approximation terms remain sup-norm quantities, and the reference kernel must be defined on the deployed occupancy support. On benchmarks, both forms serve as diagnostics rather than numerical certificates.
As you note, final-policy class-mass coverage (93.1%) exceeds raw-state coverage. This mass-based notion is the one that enters Prop. 3, through its uncovered-mass term.
(ii) Exact check (24 delivery SCMs, 36–120 states; Table 63). We used exhaustive controls, singleton bindings, and a class-policy perturbation of TV at most 0.012. The bound holds on every state pair at γ = .80/.90/.95, with median LHS/RHS of 0.52/0.39/0.24. This verifies the implementation against the theorem.
(iii) Scope and limitations (Sec. 6, new).
"DynaCAST requires resettable training environments, validated intervention pairs, and benchmark-specific adapters (native costs in Appendix F.5); deployment requires none of them. Its guarantees concern the exact covered process; on benchmarks they serve as diagnostics, and Proposition 3′ replaces uniform policy-shift suprema by occupancy-weighted terms. Exact shared-witness additivity does not by itself make the attributed register targets jointly realizable (Proposition 2′). When admissible pairs are sparse (10–25% retention, Figure 2), the matched predictor is preferable."
We see instrumented training as the cost of a state criterion with behavioral meaning. It is paid once per environment, at training time, and the deployed agent then carries only three recurrent registers. We hope the revisions make this trade-off and its boundaries transparent, and we are happy to clarify anything further.
Response to Reviewer m1pf (1/3): independent contribution of the Plan register
Thank you for the thoughtful review. We understand your main concerns to be P's independent contribution, and the scope of the training requirements and guarantees. We address P first.
Summary of changes:
Short answer. We agree that P's aggregate increment is small. Its contribution concentrates where your question points: the longest-horizon quartile and recovery after route failures. Resetting P at the failure point removes more than half of Full's recovery advantage over G+W.
Paired CI for Table 43. Total capacity is fixed at 12 tokens (6+6 for G+W, 4+4+4 for G/W/P). P receives the typed attribution term. As a descendant of every intervention, it receives no invariance term (Eq. 31), and Core has no route head or route loss. The increment is therefore not explained by register capacity.
Point estimates are positive on all four benchmarks, largest on ComplexMCP, which has the longest dependency chains.
Long horizons (frozen quartiles). P's gain grows from +0.09 [−0.72, 0.91] in Q1 to +1.58 [0.46, 2.70] in Q4, an interaction of +0.50 per quartile [0.06, 0.93]. Full − G+W in Q4 is +3.51 [2.00, 5.02].
Failure recovery (new evaluation of existing checkpoints). At 50% of the intrinsic horizon, we inject a route failure with the App. E.15 physical-world programs: a moved object, an out-of-stock item, a changed order state or a failed tool call. Every injection point admits an alternative route to the same terminal effect. Goals, injection points and prefixes are paired across methods (5 seeds).
Core − G+W is +2.9 [1.0, 4.9], and resetting P lowers Full by 3.6 [1.3, 5.9], so recovery relies on the route information held in P. A reset also perturbs the state distribution, so we do not read this as P alone causing recovery.
Plan selectivity (secondary; outside H1→H2→H3). On validated plan pairs, G/W/P exceeds the capacity-matched DAS subspace in path response by +3.4 [0.9, 6.0] under MC and by +4.6 [1.7, 7.6] under recursion. Off-target G/W changes stay at 3–5%.
Outside plan support, the point estimates remain positive with wider intervals:
We therefore state P's established contribution as long-horizon performance and route-failure recovery within validated support (Sec. 4.3).