- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
ECHO-Graph extends ECHO, which rebuilds agent context from selected past turns, rewards only the final reconstruction trace, and masks failures. It adds (i) a dependency DAG over successful trajectories with discounted credit propagated backward from the terminal turn, and (ii) attenuated dense GRPO-style negative advantages on failures. Experiments on curated subsets of BrowseComp-Plus and SWE-Bench Pro report gains over ECHO and ablations.
Critical strengths
S1. Credit assignment across repeated reconstructions is a real gap, and the observation that final-trace masking leaves early prerequisite turns unreinforced is sound.
S2. Modeling cross-reconstruction dependencies explicitly as a DAG and propagating credit over it without additional rollouts, critics, or model evaluations is a reasonable direction.
Critical weaknesses
W1. [Motivation] The motivation does not hold. Entropy collapse is not a credit-assignment phenomenon. ECHO masks every failed trajectory, so all advantages are positive and each update only sharpens the policy. The paper's own results confirm this (entropy goes to zero at λneg=0 and rises with λneg, Figure 3(c). Negative credit alone raises entropy while graph credit alone accelerates its decline, in Section 5.3. Entropy coefficient 0 and KL disabled Appendix). The paper presents this artifact as a research problem and restoring negative samples as a contribution. The second limitation, limited horizon gains, rests on one or two questions out of fifty in Figure 1(a). The need for a dependency graph is not clear.
W2. [Method] The edge construction is unjustified and self-contradictory. Edges are the policy's own selection records and adjacency. No argument is given to justify that selected turns contribute, or that unselected turns do not, yet the latter receive only geometrically decaying chain credit. The paper acknowledges provenance is "a proxy for contribution rather than a guarantee of causal relevance" (Section 4.3) and "does not reliably identify" credit-worthy decisions on failures (Section 4.2), but contradictorily relies on it for successes. The paper should state the principle under which selection implies contribution, its conditions, and a formal relation between Eq. (8) credit and counterfactual contribution, instead of only providing edge definitions and the elementary bound in Eq. (11). Absent this, any favourable result may be a coincidence specific to the selected datasets.
W3. [Method] The reported gains are attributable to λneg rather than to the graph. Positive advantages are scaled by credits below one while negative advantages are scaled by a fixed coefficient. Token-level advantages are therefore no longer centered within a group, and every update carries a net sign bias determined jointly by γ and λneg. The entropy dynamics follow this bias alone. Entropy vanishes at λneg=0, remains low at 0.5, and rises sharply at 1.0 in Figure 3(c). The accuracy differences across these settings follow the same pattern. The paper does not separate the effect of this bias from the effect of graph structure, so the improvements it reports can be explained by the negative-feedback weight without reference to the dependency graph.
W4. [Experiments] The evaluation does not support the claims. First, the paper lacks direct evidence that graph-assigned credit reflects actual contribution. This requires validation against an independently defined and reproducible ground truth, demonstrating that credited turns are necessary for the outcome while uncredited turns are not. Section 4.4 only examines credit magnitudes and does not establish this correspondence. Second, the reliability of the reported gains remains unclear. The evaluation sets also serve to select the coefficient, aggregation rule, and checkpoint (Sections 4.4 and 5), which introduces potential selection bias. Select Agent + GRPO, the baseline implied by the paper's own motivation, is absent from Table 1. Results are confined to author-curated subsets of 50 instances per level, evaluated with a single seed, with difficulty defined by rollouts of an unspecified model. The gains should be established on the full benchmarks, on data uninvolved in any design or checkpoint choice, against Select Agent + GRPO, and with improvements exceeding run-to-run variance. Finally, the method is presented as a general framework for long-horizon agentic RL, yet no evidence is given that it works beyond the setting in which its design choices were tuned.
Questions for authors
Q1. Are the validation sets used to select the coefficient, aggregation rule, and checkpoint identical to the Table 1 evaluation sets?
Q2. Does Select Agent + GRPO with λneg=0.8 match ECHO-Graph? This determines whether the gains come from the graph or from the negative-feedback weight.
Q3. What fraction of training rollouts involve at least one reconstruction? Without reconstruction, the graph is a chain, and the method reduces to discounted dense credit.
5 comments
Additional Baselines: Validating the Effectiveness of the Graph Structure
We understand that the reviewer would like to examine the effect of the credit-assignment structure while keeping positive and negative training signals comparable.
The balance between positive and negative gradients also depends on how often the current policy succeeds. Even if we fix and match the token-weighted total positive advantage per successful rollout, different credit-assignment methods may learn at different rates and produce different proportions of successful rollouts. These two constraints therefore cannot hold the overall gradient balance fixed throughout training. We use policy entropy to compare how concentrated the policies become as training progresses, as Reviewer F12S also suggested. For the original ECHO-Graph run, we retained evaluation checkpoints, but not the per-rollout selection records needed to recover its training-time credit totals.
To address your concern, we designed the following comparisons:
- Random-connect. We replace the selection edges with edges to random earlier turns and keep the rest of the graph unchanged. We use four random connections per selection, based on the average number of turns selected per event in evaluations of the initial and final ECHO-Graph checkpoints. We keep .
- Sequential-only. Our earlier sequential-only control used the same as ECHO-Graph. Here we keep the same sequential discount and set to target ECHO-Graph’s entropy trajectory.
BCP results are reported as mean ± SD over three training seeds (7, 14, and 42), as in our previous experiments. Except for the settings specified above, the experiments below use split-half evaluation, with results averaged over three splits.
| Method | BCP Easy | BCP Medium | BCP Hard | SWE Easy | SWE Hard |
|---|---|---|---|---|---|
| Random-connect (4 per selection) | 0.80 ± 0.04 | 0.38 ± 0.05 | 0.10 ± 0.03 | 0.67 ± 0.05 | 0.19 ± 0.04 |
| Sequential-only, | 0.80 ± 0.03 | 0.32 ± 0.04 | 0.06 ± 0.03 | 0.63 ± 0.02 | 0.13 ± 0.02 |
Sequential-only shows limited gains on BCP Medium and Hard and SWE-Bench Pro Hard. Random-connect improves over sequential-only on these levels, reaching performance close to ECHO. Its BCP results also vary somewhat more across seeds, possibly due to the additional randomness in edge selection. These results suggest that the specific dependency structure matters for ECHO-Graph’s gains.
A Formal Interpretation of Graph Credit and Eq. (8)
We believe the random-connect and sequential-only results above support the role of the graph. We also welcome the opportunity to explain the design of Eq. (8) in more detail.
Our gold-doc recall analysis provides evidence that the selector retains relevant information, supporting the emphasis on selected turns. The graph is designed to address two questions: how to distinguish the importance of selected turns, and how to assign credit to unselected turns.
Credit for selected turns. A turn selected multiple times is reused across multiple stages of reasoning and receives credit from multiple paths. We take this repeated selection as evidence that the turn has an important role in the successful trajectory. This is the main reason for using sum aggregation.
Max aggregation can propagate credit to selected turns, but it only keeps the largest incoming credit and cannot emphasize repeated selection. Moreover, credit may continue to decay as it propagates backward, favoring later turns in a way similar to sequential-only discounting.
Unclipped sum aggregation has a different difficulty. With weak discounting, credit can accumulate excessively at nodes where paths converge, which we found harmful in practice. Stronger discounting prevents this inflation, but it also reduces the credit reaching earlier convergence points, making them harder to distinguish from other turns. The appropriate discount can therefore depend on trajectory length and task difficulty. Clipping credit at 1 is a practical way to balance these effects: it allows multiple paths to highlight reused turns while preventing their credit from growing without bound.
Credit for unselected turns. Unselected turns lack selection records that indicate their relevance. To assign credit without additional evaluations, we use temporal proximity as a signal of local support.
For example, when reconstructing context, agent harnesses often retain recent turns alongside selected or compressed history. Recent turns provide details about the agent’s current progress; removing them can force the agent to reconstruct that progress and repeat tool calls. Similarly, the preceding turn will generally provide more direct support for the current step than a turn from dozens of steps earlier. This motivates discounting credit as it propagates backward.
How discounting and aggregation work together. Discounting spreads credit to earlier turns, while aggregation identifies turns supported by multiple paths and increases their credit before it is propagated further backward. Without aggregation, continued discounting increasingly favors turns near the end of the trajectory, regardless of whether earlier turns were repeatedly selected. Both the selection behavior and our sequential-only results suggest that temporal distance alone is insufficient.
Under this interpretation, Eq. (8) gives each turn a bounded measure of its structural support for the successful reasoning trajectory. This interpretation relies on the selector tending to retain relevant information, repeated selection indicating reuse, and nearby turns providing stronger local support.
Thank you for setting out the value you see in this work and for the new horizon runs. The three-seed BCP gains over ECHO exceed the seed standard deviations at all levels, and the new horizon runs show ECHO-Graph ahead of dense positive credit at every level and horizon. My main concerns, however, are not fully resolved.
I read the method as a joint scheme of graph credit and dense negative feedback, and I accept that dense negative feedback at the chosen λneg does not achieve the gains on its own, which answers my Q2. The reported reconstruction fractions also address my Q3. What remains open from W3 is whether the dependency graph structure itself matters. The new horizon runs show that the margin over dense positive credit is not specific to one horizon. Like the sequential-only run, however, each is a single run that differs from ECHO-Graph both in how positive credit is propagated across turns and in the total positive credit each successful rollout receives, so none of them isolates the effect of the graph structure. The λneg and checkpoint selection rule used in these runs are not stated. Two controls would separate the propagation of positive credit from its total amount, both at the same λneg and each matched to ECHO-Graph in token-weighted total positive advantage per rollout. The first is a sequential discount whose rate is set per rollout to reach that total. The second keeps the graph but connects each selection edge to a random earlier turn, with the rate set per rollout in the same way. If ECHO-Graph clearly outperforms both over the same seeds under the protocol in point 2 below, the dependency graph structure itself matters on BCP. On SWE-Bench Pro, where the gain over ECHO currently rests on a single run, the same comparison even with a single shared seed would show whether this holds beyond BCP.
Following up on Q1, the reported gains are measured on the same questions that are used to select checkpoints and, on BCP, to compare λneg values. This applies to the trained results in Table 1, Figure 6, and the three-seed runs, and each Figure 6 horizon is a single run. The larger last-checkpoint advantage mentioned to Reviewer F12S comes without specific numbers. Selecting the checkpoint on one half of the questions in each level and reporting on the other half, averaged over many random splits, for the existing runs including SWE-Bench Pro, the three-seed runs, each Figure 6 horizon and the new horizon runs, would show whether the gains hold without checkpoint-selection bias. The same split applied to the Figure 3(d) runs would do the same for λneg, and neither requires new training.
Regarding W2, my earlier wording on per-turn necessity may have been unclear. I did not mean that training should verify every credited turn, which would be computationally impractical, and I agree that such a check would not by itself establish that the overall scheme works. My point is that ECHO-Graph, unlike GRPO, is designed to credit turns based on their role in the dependency graph, so it is fair to ask whether its credit tracks contribution better than a sequential discount. The argument that unselected turns are not directly necessary explains why they should receive less credit than selected turns, but not why the amounts should follow the clipped discounted sums of Eq. (8), and Section 4.3 treats provenance as a proxy for contribution without defining what contribution means. The paper should formally state what quantity Eq. (8) approximates and under which conditions, or show on a few successful rollouts with at least one memory selection that removing turns ranked higher by graph credit than by sequential credit reduces the success rate more than removing turns ranked lower. Specifically, each turn would be removed from the memory and all later contexts, and the drop in success rate would be measured over several samples of the rest of the rollout from the next memory selection, relative to the same sampling without the removal. It would also help to state how graph credit would apply when context is managed without explicit selection records, as in the Summary Agent of Table 1.