- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Two issues remain. A generic history-check reminder reaches 48.60% versus 48.25% for outcome labeling, so the gains may come from more careful history checking rather than action–outcome binding. The calibrator also adds a review call after each step, with labels from GPT judgments rather than executed outcomes, and no cost-matched comparison or independent label-quality check is provided.
Critical strengths
- Well-motivated experiments. The history interventions, fixed-history repetition analysis, and full rollouts connect the initial diagnosis to the proposed interventions. This makes the paper more informative than a method evaluated solely by its final success rate.
- Broad empirical validation. The calibrator improves over the raw-history baseline in every reported model-level and environment-level aggregate. The overall gain of 5.06 percentage points suggests that the approach is useful beyond a single actor or benchmark.
Critical weaknesses
- The proposed mechanism is not fully isolated. A generic history-check reminder reaches 48.60% success, comparable to the 48.25% achieved by explicitly labeling action outcomes (Appendix D.1). It remains unclear how much of the latter’s benefit comes from action–outcome attribution rather than prompting the actor to reconsider its history. This limits the strength of the paper’s central mechanistic claim.
- The calibrator’s benefit is not established under a matched inference budget. Unlike the raw-history and outcome-labeling conditions, the calibrator adds a review call after each completed nonterminal interaction. Its training targets also depend on model-generated candidates and model-based judgments. Without cost-matched results and an independent check of label quality, it is difficult to assess the practical significance of its improvement.
Questions for authors
- Under matched prompt length and inference budget, how do outcome labeling, the history-check reminder, and their combination compare? The answer would help distinguish action–outcome attribution from a more general history-rechecking effect.
- What are the average additional calls, tokens, and costs of the calibrator, and does its advantage persist under a matched budget? How well do independent or human assessments agree with the model-based judgments used to construct its training targets?
4 comments
Response to Reviewer b3Zs (part 2)
Q1: Under matched prompt length and inference budget, how do outcome labeling, the history-check reminder, and their combination compare? The answer would help distinguish action–outcome attribution from a more general history-rechecking effect.
Response: The prompts and inference budgets for Causal, Check, and their combination are already similar, with the prompts differing by only a limited number of tokens. We pool actor calls across models and environments. The per-call means below are total input or output tokens divided by total calls. The differences are small.
| Condition | Average input tokens per call | Average output tokens per call |
|---|---|---|
| Causal | 3,137.05 | 177.57 |
| Check | 3,281.84 | 178.10 |
| Causal + Check | 3,447.44 | 190.12 |
For performance results, please see our response to W1 above. Overall, combining the two methods brings no additional advantage, further indicating that they operate through similar mechanisms.
Q2: What are the average additional calls, tokens, and costs of the calibrator, and does its advantage persist under a matched budget? How well do independent or human assessments agree with the model-based judgments used to construct its training targets?
Response: Output token statistics for each actor are provided in W2 above. To assess agreement with human judgments, we randomly selected 100 experiences for human assessment. The assessment was conducted without access to GPT’s judgments. Human and GPT judgments agree in 83% of cases, indicating that GPT-based data selection is reliable. The calibrator trained on these data also achieves clear performance improvements. These results show that the judgments are reliable.
Thank you for the additional experiments. The combined Causal and Check result is consistent with the two interventions having overlapping effects. The Experience baseline, token breakdown, and blind human check also help address my evaluation concerns. These additions strengthen the paper. I will update my score to weak accept.
Response to Reviewer b3Zs (Please update the score)
Thank you for your positive assessment of the additional experiments and for your constructive feedback throughout the discussion. We noticed that the displayed rating may not yet reflect your intended update to weak accept. Could you please update the rating in your original review accordingly?
Response to Reviewer b3Zs (part 1)
Thank you for recognizing the connection between the diagnostic experiments and the interventions. Below, we report the added combined condition and measured review overhead.
Response: Similar performance from Causal and Check is precisely our intended finding. Causal adds no review instruction: it labels observations as outcomes of preceding actions, implicitly prompting reassessment. Check requests reassessment explicitly, so similar performance is expected.
We further combine Causal and Check and compare the following four conditions on the same 2,000 tasks (100 per model–environment cell). Actor rows average four environments; Overall averages the five actors:
Overall, both methods improve over Raw. Their combination does not outperform Causal alone. This is consistent with overlapping roles in history checking.
Response: Calibrator is an optimization of Experience. Both generate experiences from history, then let the original actor choose actions using that history and the retained experiences. Calibrator trains a model to generate better experiences. The original method-evaluation figure did not display Experience alongside Calibrator. Figure 4 had already shown that Experience performed worse than Raw. This motivated us to train a better experience generator. Experience therefore provides a direct baseline for comparing review overhead and benefits. Each condition uses 2,000 tasks, with 100 per model–environment cell. Rates average the five actors and four environments:
We also compare their output-token use. For each actor, output tokens are averaged over four environments, with 100 tasks per environment. Cells report actor output + review-call output in k tokens per task. For example, 1.45 + 1.40 means 1.45k actor output tokens and 1.4k review-call output tokens per task.
Calibrator’s review-call output per task is substantially smaller than the actor’s output. Calibrator adds little output-token overhead. Its total output is usually similar to Raw and lower than Experience. Its 8B experience generator further reduces resource cost.
GPT-5.6-Sol’s larger review-call output may be related to internal reasoning tokens. These account for about 71.4% of its review-call output tokens pooled across the four environments. This explains its approximately 1.40k review-call output tokens per task. The other models generate both actions and experiences with thinking mode disabled.