- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces the Execution Information Requirement (EIR), a lower bound on the information an agent must retain to guarantee correct continuation under a specified workload. It also presents LACUNA, a controlled framework for studying state loss and recovery, and VESTIGE, a retrospective estimator of relevant information in completed agent trajectories. The experiments show that restoring a missing result can repair errors and that retention and recovery behavior affect long-running execution.
Critical strengths
- The paper brings information requirements, state retention, and recovery into a coherent framework. Its comparison between restoring the exact missing result and supplying equally long irrelevant information supports the claim that unavailable required state can cause execution errors.
Critical weaknesses
-
The connection between the theoretical definition and the practical estimate is insufficiently established. EIR measures the minimum information needed to guarantee correct continuation for every allowed future, whereas VESTIGE measures file states used later in one completed trajectory. The paper establishes neither a bound between these quantities nor another precise relationship.
-
The experiments do not sufficiently validate VESTIGE as an estimator of required information. In the state-restoration experiments, the framework identifies the required result in advance and supplies it directly. These experiments demonstrate the effect of missing state, but they do not show that VESTIGE can accurately identify or quantify the information an agent needs.
-
The presentation of key experimental results in the main text is insufficiently consolidated. Although the paper includes figures and tables in the appendix, results on recovery cost, retention policies, and architecture comparisons are mostly reported as numbers in prose. A summary figure or table comparing experimental settings, controls, and outcomes would make it easier to assess how each method is validated. The main text also exceeds nine pages by two lines: the conclusion continues onto page 10.
Questions for authors
-
Is there a precise, testable relationship, such as an upper or lower bound, between VESTIGE's retrospective estimate and theoretical EIR?
-
Can the authors evaluate VESTIGE's accuracy in identifying required state and estimating information demand on tasks for which the required information can be determined independently?