acceptodds
Withdrawn from ICLR 2027

Do Small Errors Snowball?Understanding and Controlling Error Propagation in Language Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Small observation changes can propagate through a language agent’s feedback loop, changing its actions, environment state, and later inputs. Terminal success conflates initial sensitivity with subsequent amplification, while one-step agreement cannot measure recovery. We introduce an affine Wasserstein profile in a policy- independent task-semantic geometry that separates leakage on equivalent states from amplification on separated states, with composition and first-exit bounds linking local dynamics to outcomes. Coupled-Rollout Contraction Optimization (CRCO) then trains marginal-preserving coupled continuations, penalizing suc- cessor separation only beyond a state-conditioned allowance. We prove that this correspondence distinguishes pair laws that absolute and constant-threshold penal- ties cannot, and that the optimized hinge bounds task-score sensitivity with a finite-sample guarantee. Across nine backbones and eight interactive settings, CRCO has the highest perturbed-score point estimate among six matched objec- tives on all nine backbones. It exceeds matched absolute propagation by 2.6/3.3 points on two instrumented anchors, retaining a 2.6-point mean gain with an observation-trained metric without latent predicates. Separately analyzed reference- prefix and reset-to-terminal protocols yield equal-cell mean gains of 3.2 and 2.5 points. Threshold reassignment, recovery interventions, and five-step checkpoint selection corroborate the mechanism; deployment remains one ordinary rollout.

Primary area:
foundation or frontier models, including LLMs
Keywords:
language agents, large language models, error propagation, reinforcement learning, learning theory, robustness, dynamical systems, Wasserstein distance, long-horizon planning, coupled rollouts

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LzenOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how small errors propagate through agent interactions and proposes a framework for measuring and controlling the resulting deviations. Theoretically, it shows that a state-conditioned hinge captures current–successor distance correspondence that absolute and constant-threshold penalties cannot distinguish, and derives a finite-sample bound relating hinge violations to task-score sensitivity under stated assumptions. In practice, it introduces Coupled-Rollout Contraction Optimization (CRCO), which adds an initial-distance penalty and a state-conditioned propagation hinge to paired task return. Training uses tokenwise maximal coupling and a joint-likelihood policy update, while deployment requires only a single ordinary rollout. Experiments across nine backbones and eight interactive settings evaluate robustness and error propagation against matched training objectives.

Critical strengths
  1. The paper offers a novel perspective on agent reliability by separating initial sensitivity from subsequent error amplification and recovery. This framework is particularly relevant to long-horizon agent tasks, where small deviations can accumulate through repeated interactions. Its focus on task semantics and the evolution of deviations provides a useful basis for analyzing reliability beyond local accuracy or terminal success alone.

  2. The experiments cover nine backbones and eight interactive settings, with meaningful improvements across a broad range of model–task combinations. CRCO improves perturbed scores by 2.6 and 3.3 points over matched absolute propagation on the two anchor backbones. Matched controls and recovery analyses further support the value of state conditioning, while deployment requires only a single ordinary rollout.

Critical weaknesses
  1. The connection between the theoretical analysis and the practical CRCO algorithm is kind of week. Although the theory motivates controlling error propagation, it is difficult to trace how the theoretical quantities and assumptions translate into the measurements and optimization choices used in training. This makes it challenging to assess how directly the theoretical results explain the observed improvements in task performance.
  2. I think there is still room for improvement in the presentation. The paper places substantial emphasis on equations and theoretical results, while the actual training procedure is less clearly explained in the main text, making it difficult to reconstruct a complete data collection and training iteration. Figure 1 would also benefit from a more explicit walkthrough connecting its components to the algorithm and a concrete task example.
Questions for authors
  1. Could you clarify how the theoretical analysis informs the practical design of CRCO? A concrete task example tracing the initial perturbation, subsequent state distances, and propagation penalties would help illustrate this connection. It would also be useful to explain which intermediate measurements in the existing experiments support the proposed link between controlled propagation and improved task outcomes.
  2. Can you provide a detailed description of the training pipeline, including the training data, reference-prefix construction, semantic extractor configuration, and optimization settings? A walkthrough of one training iteration, from generating paired rollouts to computing rewards and updating the policy, with references to the relevant appendix details, would make the implementation easier to understand.
Reviewer_cGxQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper models error propagation in LLM agents as an affine Wasserstein recurrence (leakage q, amplification rho) and trains CRCO, a GRPO variant whose penalty fires only when a successor's paired distance exceeds rho times the current distance plus q (Eq. 16, p.6). On Qwen2.5-7B, CRCO reaches a perturbed score of 53.3 against 50.7 for the matched absolute penalty (Table 1, p.8). The matched controls are careful, but the headline claims outrun them; I lean toward rejection.

Critical strengths
  • The constant-threshold and shuffled-current hinges (Table 35, p.54) match CRCO's activation rate and mean penalty and still score 2.1 and 2.4 P points lower; the state-conditioned threshold is the working part.
  • The randomized slope-target sweep (Table 21, p.43) assigns targets within blocks; horizon-24 success falls from 54.4 at target 0.65 to 36.8 at 1.10, with the clean score within 1.1 points.
  • Six to eight seeds per cell with seed-level intervals, and the negative controls on p.51.
Critical weaknesses
  • The abstract's "all nine backbones" rests on Gemma-4-26B at +0.5 (Table 1, p.8), from cells +0.5, -0.5, +1.1, +0.8 with seed SDs 0.9 to 2.3 (Table 2). Give an interval per backbone and say which exclude zero.
  • HCAPO beats CRCO on perturbed ALFWorld (68.0 vs 65.8) and WebShop (63.6 vs 62.7) and every clean score (by 12.7 on ALFWorld; Table 32, p.53), yet Table 1 omits it. Add the row and, for the rebuttal, train the CRCO hinge on top of HCAPO.
  • Section 5.4 reports 23 vs 21 oracle matches and "40.7% fewer robustness calls"; Appendix I.3 (p.41) gives a regret difference of -0.14 [-0.52, 0.21] and a 20.2% saving once clean validation is counted. State both.
  • CRCO costs 2x tokens, 2x simulator calls, and 1.91x GPU time against unpaired GRPO (Table 36, p.55); the main text says only "compute-matched". Put the ratio in Section 5.1.
  • The eight-seed retention intervals against matched absolute include zero on WebShop [-0.2, 2.7] and tau-bench [0.0, 3.6] (Table 51, p.64). Say so in Section 5.2.
  • Theorem 2's finite-sample bound (Lg = 1/mg for binary success, p.5) is never evaluated; only the distance recurrence is (Table 22, p.46). Instantiate it on one anchor or drop "guarantee" from the abstract.
  • The artifact described on p.10 is a scalar reference implementation; the paper does not say whether training code, replay wrappers and transformation manifests are released. Say what is.
Questions for authors

Theorem 1 has the geometric-sum form of Theorem 3.1 in Rudolf and Schweizer (Bernoulli, 2018), which the paper cites, applied to one kernel with a quotient-leakage term; the first-exit accounting and the affine hinge of Eq. 16 are new. The one-step consistency baseline is the agent analog of the SA-MDP regularizer of Zhang et al. (NeurIPS 2020), which is not cited. Neither are Ross et al. (AISTATS 2011) on compounding errors nor Zhang et al. (2023), who named hallucination snowballing. Direct-state training (Table 12, p.35) reaches 53.1 against CRCO's 53.3, so the look-ahead mixing in the distance adds nothing measurable; the contribution is the hinge. The 3.2 and 2.5 equal-cell means, the 2.8 and 1.4 reassignment drops, and the 2.6-point observation-metric gain match Tables 2, 16, and 3.

The paper says LLMs drafted its text (p.10), and it shows: 66 pages, 52 tables, 16 propositions, notation that Appendix A admits is overloaded, and verdict sentences such as "This pattern supports the recovery hypothesis without requiring a uniform per-task effect" (p.62). Cut the main text to the hinge, Tables 1, 2, and 35, and the sweep; move the measurement theory to its own paper.

Reviewer_AEiwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes two contributions: first, a measure that separates only states which are not semantically equivalent; and second, CRCO, an optimization method that rewards recovery from error, despite recovering trajectories appearing different textually. CRCO is then compared against other optimization methods on a variety of agentic benchmarks.

Critical strengths

The broad motivation of the paper (distinguishing recovery from expansion of errors) seems worthwhile to pursue, and a clearer explanation of the specific method proposed may represent a useful step forward on this problem.

Critical weaknesses

The motivation for the specific methods would benefit from further clarification. I found it difficult to see how the different parts of the theoretical framework fit together, particularly where key notation is introduced without enough explanation of its meaning or role. For example, for pˉ\bar{p} and qˉ\bar{q} in Proposition 1, the clearest description I could find was in Appendix B, where they are called the “training tolerance and target slope, selected on development data”. I could not find a more precise definition or explanation of how they were selected, despite their prominent role in the subsequent results.

Some experimental choices would also benefit from a clearer rationale. For RQ1, I could not find an explanation of why the perturbed-score point estimate was chosen over alternative measurements, or what advantages it offers. Similarly, for RQ3, I would appreciate clarification of why five-step selection was used rather than another selection method.

The organization of the paper also made parts of the argument difficult to follow. The proof of Theorem 1 is said to appear in Appendix D.1, but that section appears empty, while the proof appears in the middle of Appendix D.2. I was unsure whether this proof concerns Theorem 1 in the main text or Corollary 1 in D.2. The appendices do not appear to follow the order of their references in the main text, and I found the connections between adjacent appendices difficult to identify. More accessible definitions of technical terms would also help readers who are less familiar with the relevant subfields.

Given these clarity concerns, I recommend rejection in its current form, although a revision addressing the points above may be suitable for acceptance.

Questions for authors
  1. Concretely, what is the motivation for the specific method developed?
  2. In each experiment, what context is preserved between each step?
  3. What prompts are used for each experiment? What transformations are used?
  4. In Appendix F, how are the weights assigned to each feature?
  5. Is the proof of Theorem 1 in Appendix D.2 meant to be in Appendix D.1?
  6. What are pˉ\bar{p} and qˉ\bar{q}, and how are they selected?
  7. Why is a perturbed-score point estimate used as the key measurement for RQ1?
  8. Why is five-step selection in particular used in RQ3?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.