- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes ThesisAgent, which maintains a persistent set of assumptions about an ongoing task and revises them as new evidence arrives. It includes three mechanism to achieve persistent decision theses: Evidence-Local Revision, Decision–Execution Separation, and Intervention-Based Utility Audit. The paper evaluates these designs in ThesisWorld, ALFWorld, and a financial proxy, reporting improvements in belief and decision metrics and task success.
Critical strengths
In long-horizon tasks, new observations can invalidate some earlier assumptions while leaving others intact. How an agent maintains and revises its beliefs as conditions change is an important question.
Critical weaknesses
W1. The motivation and terminology are difficult to follow. Here are some examples:
- The introduction presents three newly named challenges, but Figure 1 mainly contrasts step-wise rationale regeneration with persistent selective thesis revision; it does not illustrate the execution or decision-quality issues.
- In the discussion of Unscoped Thesis Revision, the text moves from information omissions and incorrect updates (line 49), to prior memory-editing methods (lines 50–51), and then to evidence relevance and update magnitude (lines 52–70), without clearly distinguishing these failure modes or explaining how the cited methods motivate the specific scope problem.
- Terms such as hypothesis, belief, thesis, desired decision, and executed action appear in the motivation before their relationships are made clear. A single worked example connecting each challenge to its mechanism and evaluation would improve readability.
W2. The ALFWorld baseline results require stronger validation. In Table 2, all six adapted baselines achieve only 17–25 successes out of 134 tasks (12.7–18.7%), while the paper’s own no-persistent-thesis variant achieves 103/134 (76.9%). The gap is particularly striking for ReAct, which scores 17/134 (12.7%) here. On 134 unseen ALFWorld tasks, the original ReAct paper [1] reports 57% averaged over six prompts, 71% with its best prompt, and 78.4% using GPT-3 in the appendix. A later study[2] reports 75.4% for ReAct using gpt-4.1-mini, 134 tasks, and a 50-step budget. Reflexion also scores only 17/134 in Table 2, whereas an independent reimplementation[3] reports 59.8% with GPT-3.5 Turbo and 85.9% with GPT-4 Turbo. These cross-paper numbers are not controlled comparisons: models, prompts, action interfaces, and—in Reflexion’s case—retry protocols differ. These unusually low baselines need explanation. The authors should disclose their prompts and protocols and rerun ReAct under the same setup. Until then, Table 2 cannot establish the size of ThesisAgent’s gain over prior methods.
[1]https://arxiv.org/html/2210.03629
Questions for authors
As shown in Weakness.
3 comments
Response to the authors
Dear Authors,
Thank you for clarifying Figure 1. I accept that it illustrates only the first challenge; that is a minor point. My broader presentation concern remains. Lines 49–70 are one example of a difficult argument, and the paper does not consistently distinguish thesis, belief, hypothesis, desired action, and executed action. Your rebuttal helps, but these connections need to be clearer in the manuscript before I can raise my Presentation score.
The ReAct rerun strengthens my concern about Table 2. Its success rises from 17/134 to 56/134 after you both complete the implementation and relax the history restriction. Because both changed, the rebuttal does not explain the original result. The manuscript does not disclose the 1,800-character limit. Appendix A.1 describes different recent-history windows but does not specify what each literature baseline receives. A common limit also does not ensure comparable access to information when ThesisAgent retains a thesis and PhaseGuard retains a separate ledger. What does “Full-History” mean under this limit?
The within-paper contrast is striking: all six adapted literature baselines solve 17–25 tasks, while your no-persistent-thesis variant without PhaseGuard solves 103. The matched 124-to-132 result supports thesis persistence, but it does not explain this much larger gap, and the evaluation includes development-touched tasks. Published ReAct scores are also substantially higher, though their protocols differ. Please provide the baseline prompts, exact history rules, paired task outcomes, and separate tests of the implementation and history changes. Until then, Table 2 does not reliably establish the size of the improvement over prior agents.
### Final Response to Reviewer wqnp
Dear Reviewer,
We sincerely thank the reviewer for the thoughtful follow-up and detailed clarification. Your latest response has helped us better understand the limitations of our current manuscript and, more importantly, provided concrete guidance on how to improve our work.
After carefully considering your feedback, we recognize that the manuscript requires further refinement, particularly in terms of conceptual clarity, baseline implementation transparency, and experimental validation.
We have therefore decided to withdraw our submission from ICLR 2027 to undertake a more thorough revision. We will clarify the relationships among the core concepts, systematically re-examine the experimental protocols and baseline implementations, and conduct additional controlled experiments to ensure that our conclusions are adequately supported by the evidence.
We sincerely appreciate the time and effort you have devoted to reviewing our work. Your detailed comments and follow-up discussion have provided valuable direction for improving the rigor, clarity, and reproducibility of our research.
Thank you again for your careful evaluation and constructive feedback.
Response to W1 and W2: Clarifications on Motivation, ALFWorld Baselines, and Evaluation Scores
Response to Reviewer wqnp
We thank the reviewer for the feedback and provide the following clarifications and new experimental evidence.
W1. Motivation and Figure 1
We respectfully disagree that Figure 1 is inadequate because it does not illustrate all three challenges.
The Introduction explicitly states: "Figure 1 illustrates how maintaining this thesis differs from regenerating a rationale at each step."
Figure 1 specifically motivates Unscoped Thesis Revision, not all three challenges. We never claim otherwise. The remaining challenges, Belief–Execution Entanglement and Sensitivity–Utility Ambiguity, are separately explained in the Introduction and illustrated through their corresponding mechanisms in Figure 2.
A focused motivation figure should not be considered inconsistent merely because it does not illustrate every technical challenge. We therefore disagree with this particular criticism.
Regarding terminology, a Thesis contains persistent, identifiable hypotheses. Desired Action precedes feasibility enforcement, while Executed Action follows it. Unscoped Thesis Revision distinguishes which hypotheses may change from how much they change. We agree that these definitions and their connections to prior work can be introduced more clearly and will revise the presentation accordingly.
W2. ALFWorld Baseline Reliability
First, the 1,800-character reasoning-retention limit was applied to ALL methods, not specifically to ReAct. Thus, the original 12.7% result reflects ReAct under a common restricted-context protocol, rather than its full-history implementation. Cross-paper success rates under different models, prompts, and execution protocols cannot alone establish unfair treatment.
Nevertheless, we acknowledge that common restrictions do not guarantee faithful reproduction of every original method.
To directly address this concern, we completed the ReAct implementation, relaxed history restrictions, and reran all 134 tasks.
Success improved by 29.1 percentage points, while token consumption increased 2.37×. Since implementation details were also completed, we do not attribute the entire gain solely to history retention.
We acknowledge that 41.8% remains below some published results and will clarify the prompting and execution protocols. The two configurations should not be conflated.
Crucially, ThesisAgent's contribution does not depend solely on ReAct's performance. Our subsequent mechanism-matched ALFWorld evaluation, using the same PhaseGuard controller, yields:
This +6.0 percentage-point gain directly supports the incremental contribution of thesis persistence. We acknowledge that this evaluation is post-hoc, not an untouched test. Independent ThesisWorld experiments also evaluate the revision mechanism without relying on ReAct.
We therefore distinguish baseline reproduction fidelity, comparisons under common restrictions, and the independent contribution of persistent thesis state. Limitations in one comparison do not automatically invalidate separate controlled evidence.
Clarification on the Lowest Scores
We respectfully request clarification regarding the minimum scores assigned to all three dimensions.
Presentation (1): Figure 1 fulfills its explicitly stated purpose. What additional fundamental presentation issues justify the lowest score?
Informativeness (1): The paper introduces a persistent thesis-transition contract, scoped revision, and intervention-based utility auditing. Which prior work already establishes these contributions sufficiently to justify the lowest informativeness score?
Soundness (1): Beyond the ALFWorld baseline concerns, are there fundamental issues with the independent ThesisWorld experiments, analytical results, or matched ablations?
We acknowledge genuine limitations but believe that concerns about specific comparisons should not automatically be extended to all methodological contributions. We respectfully ask the reviewer to reconsider the scores and overall recommendation in light of the new evidence.