- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes CoPE, a persistent task-state framework for long-horizon vision-language-action (VLA) agents, designed to support runtime replanning and failure responsibility attribution in dynamic environments. CoPE maintains structured task commitments with stable occurrence identities, allowing local state updates while preserving unaffected commitments. By leveraging occurrence-level information, the framework further aims to identify the modules responsible for execution failures and guide policy adaptation through occurrence-gated deployment. The authors evaluate CoPE on controlled state-scaling tasks, a failure attribution benchmark, LIBERO simulations, and OpenVLA post-training experiments. The results demonstrate improvements in task-state consistency, update efficiency, and failure attribution accuracy over the evaluated baselines.
Critical strengths
Well-motivated problem formulation and unified framework. This paper addresses an important challenge in long-horizon VLA agents: distinguishing legitimate task changes from genuine execution failures. By introducing persistent task states with occurrence-level identities, CoPE provides a unified framework for runtime replanning, failure attribution, and policy adaptation. This is a meaningful contribution, as it offers a systematic approach to maintaining task consistency and supporting targeted adaptation in dynamic environments.
Critical weaknesses
Strong dependence on accurate task-state estimation and limitations in failure attribution.
CoPE relies heavily on the accurate interpretation of task and environmental events to maintain its persistent task state. However, in realistic environments, noisy observations, ambiguous instructions, or missing events may lead to incorrect state updates, potentially propagating errors to subsequent replanning and failure attribution. Furthermore, identifying a violated commitment does not necessarily establish which module is responsible for the failure, particularly when failures arise from interactions among multiple components. The paper provides limited discussion or empirical evidence regarding how CoPE handles such uncertainties and complex failure scenarios. This raises concerns about the framework's robustness and reliability in real-world deployments.
Limited downstream performance gains and insufficient experimental validation.
Although CoPE demonstrates improvements in task-state consistency and failure attribution, the benefits for downstream task success remain less convincing. In the LIBERO experiments, CoPE achieves the same success rates as FSR-PC (17/20 in Stage B and 9/15 in Stage C). Similarly, the OpenVLA post-training experiments show only a marginal improvement over the All-Failure baseline (9/18 vs. 8/18). These results make it difficult to assess whether the proposed framework offers meaningful advantages in end-to-end robotic performance.
Furthermore, the experimental comparisons are insufficient to establish CoPE's advantages over strong alternative approaches. The evaluation also relies substantially on custom benchmarks, particularly the 1,200-record mixed-event failure attribution benchmark, for which the data construction process, event distribution, and annotation procedures are not described in sufficient detail. These limitations raise concerns about the strength of the empirical evidence, the fairness of the comparisons, and the generalizability of the reported findings.
Questions for authors
- End-to-end performance: Given that CoPE achieves the same task success rates as FSR-PC in LIBERO and only marginally outperforms the All-Failure baseline in OpenVLA post-training, could the authors provide more extensive experiments to demonstrate whether the proposed framework yields meaningful improvements in end-to-end task performance?
- Benchmark construction and transparency: Could the authors provide additional details about the construction of the 1,200-record mixed-event benchmark, including the data generation process, annotation procedures, and distribution of different event types and failure categories? Such information would help assess the reliability and representativeness of the benchmark.
- Robustness to imperfect task states: How does CoPE perform when task states are inaccurate or incomplete due to noisy observations, ambiguous instructions, or missing events? It would be valuable to evaluate the sensitivity of both replanning and failure attribution to these sources of uncertainty.
- Component contributions and complex failures: Could the authors provide more detailed ablation studies to disentangle the individual contributions of occurrence identity, lineage, grounding, and provenance to failure attribution performance? Additionally, how does CoPE handle failures arising from interactions among multiple modules, where responsibility cannot be attributed to a single component?
- Clarity and organization: The presentation could be improved in terms of clarity and conciseness. Several important methodological and experimental details are deferred to the appendix, making it difficult to fully understand and evaluate the proposed framework based on the main text alone. I encourage the authors to streamline the subsection structure, simplify unnecessarily technical descriptions, and incorporate essential methodological and experimental details into the main text. These revisions would help readers better understand the core contributions and assess the significance of the empirical findings.
3 comments
Thank you for your detailed and thoughtful responses. I appreciate the authors' efforts to address my concerns and clarify the motivation, benchmark construction, and experimental results.
After carefully considering the rebuttal, I am sorry that I will maintain my original rating, as my major concerns regarding the empirical validation and overall presentation of the paper remain. Nevertheless, I believe the research problem is meaningful and the proposed framework has potential.
Beyond the technical aspects discussed above, I would encourage the authors to substantially revise the paper's presentation. In particular, I would recommend studying Best Paper Award-winning papers at leading conferences in related research areas, paying attention to:
- How to establish the research context and clearly articulate the key motivation.
- How to organize and present the proposed methodology in a coherent and intuitive manner.
- How to design and present experiments that convincingly demonstrate the necessity and advantages of the proposed approach.
- How to improve the quality, clarity, and informativeness of figures, which are also important factors in reviewers' assessment of a paper's overall quality.
I sincerely hope the authors will take these suggestions as constructive feedback. I believe that with further experimental validation and substantial improvements to the presentation, this work could become a much stronger contribution.
I wish the authors all the best in revising this work and sincerely hope that an improved version of the paper will find a good home at a leading conference in the future.
Thank you so much for your thoughtful and encouraging response. I really appreciate the time you took to read our work and offer such detailed advice!
Thank you for recognizing CoPE's motivation. Our work addresses two costs in long-horizon robot tasks: repeatedly figuring out what remains to be done, and collecting training data without knowing what went wrong.
1. Why maintain task state when task success is similar?
Suppose a user asks for an object in basket A, temporarily changes the destination to B, then cancels B. The robot needs to recover A and its remaining work. Repeatedly reading the full history takes time and can lead to mistakes.
CoPE keeps the current requests, their relationships, and their changes in a persistent state. It updates affected parts while preserving the rest.
Table 2 measures update cost. At 32 commitments, CoPE generates 6,891 output tokens across 36 calls, compared with 191,787 for full-state regeneration. Other local-update methods achieve similar efficiency.
Table 3 measures what earlier information remains available. In LIBERO Stage C, CoPE and FSR-PC both succeed in 9/15 cases. However, CoPE retains occurrence IDs in 15/15 cases and achieves historical audit coverage of 1.000, compared with 0/15 and 0.344 for FSR-PC. Adding provenance to FSR-PC raises coverage to 0.933. The audit measures whether historical information can be retrieved, which matters when restoring requests or examining past failures.
2. How was the 1,200-record benchmark built?
We used controlled fault injection across four held-out-task folds of 300 records. Events include 504 object movements, 333 temporary unavailability cases, 142 new preferences, 103 reissued goals, 72 cancellations, and 46 no-go restrictions.
Reference labels cover 320 policy, 160 editor, 160 planner, 120 verifier, and 80 multi-module failures, plus 240 no-update and 120 abstain cases.
For each record, we compared normal execution with an injected-fault continuation from the same state. A deterministic program checked the component outputs and available counterfactual evidence. Records were accepted only when the measured cause matched the predefined assignment and all required checks passed.
Labels were hidden from the evaluated models. They describe verified controlled interventions, not human annotations of natural failures.
3. What if the stored state is wrong?
CoPE checks proposed changes before saving them, including task references, legal status changes, and relationships between requests. Invalid changes leave the previous state unchanged.
Our validation pipeline rejected all 88 targeted injected faults. This addresses the tested state-update errors. Misunderstood instructions or missed observations may still produce plausible but incorrect updates, which require further evaluation.
4. How do we assign responsibility?
Our second goal is to use expensive robot training data more carefully. A changed goal, a planning mistake, and a failed grasp may require different corrections.
CoPE retrieves the task state at the time of failure to see which request was valid, what was expected, and what actually happened. It can assign several modules or leave the decision unresolved.
On the frozen benchmark, CoPE's F1 for editor/planner/policy/verifier is 0.875/0.923/0.536/0.505, compared with 0.842/0.943/0.528/0.468 for Generic Persistent. Macro F1 is 0.710 versus 0.695. The results are close overall.
Each state component has a role. Occurrence identity separates repeated requests; lineage tracks replacements and restorations; grounding links requests to objects; provenance records accepted changes and evidence. Table 3 examines some of these properties. Their individual effects on responsibility assignment, including complex multi-module failures, need further testing.
5. Does this improve robot learning?
We used responsibility decisions to select correction examples and train actual OpenVLA adapters with matched budgets.
In the full frozen dynamic evaluation, CoPE succeeds in 9/24 runs versus 8/24 for All-Failure. The paper's 9/18 and 8/18 figures came from a later six-state subset. The success difference is small, while the training data selection shows how responsibility information can focus corrections on the relevant task.
CoPE also records which task occurrence an adapter belongs to. In the clean-retention test, all 9/9 cases stayed on the original policy and succeeded because the adapter was inactive.
The same task information supports both goals: helping the robot avoid repeatedly reconstructing earlier requests, and helping us decide what needs correction after a failure. We would make these points and the key experimental details clearer in the main text.