- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents DynaHarness, a hierarchical robot-agent framework combining a slow semantic planner with a faster execution supervisor. The system selects among analytic skills, recovery skills, and a frozen VLA, monitors execution, and switches capabilities or requests replanning when needed. Execution records support offline failure diagnosis and capability revisions, which undergo paired validation before admission. My main concerns are the unclear methodological novelty beyond system integration and the presentation, which makes the concrete mechanisms and experimental conclusions difficult to follow.
Critical strengths
The paper includes a broad range of robot-policy and harness baselines, covering several upper-level models and a subset of matched-model comparisons. This breadth provides useful context for evaluating the proposed system rather than comparing it only against a standalone policy.
Critical weaknesses
-
The methodological novelty beyond system integration is unclear. The framework combines capability selection, execution monitoring, recovery, replanning, logging, and validation-gated revisions, but does not clearly identify a new algorithm or decision mechanism underlying this combination. Its own related-work discussion attributes runtime scheduling and verification to PhyAgentOS, and hierarchical failure diagnosis and validation-gated updates to Zetta. The proposed “physical execution contract” does not sufficiently explain what is technically different. The experiments somewhat demonstrate system utility, though it does not win Zetta in table 27.
-
The terminology-heavy presentation obscures the method and its evaluation. Terms such as “physical execution contract,” “lease,” “evidence store,” and “frozen snapshots” are introduced without sufficiently concrete explanations at the point of use. Several equations name interfaces or operations without explaining how they are implemented. Although success rates are reported, the narrative repeatedly shifts among development rounds, snapshots, execution statistics, and differently configured comparisons. This makes it unnecessarily difficult to determine what the algorithm actually does and which results substantiate each contribution, limiting my ability to assess the work.
Questions for authors
-
Step budget and lease, line 184: What exactly does each quantity constrain, and how are they initialized, consumed, and expired? Simply naming them does not explain their operational meaning. In particular, what does “spend the same budget under the same lease” mean when switching between executors?
-
Capability execution, lines 210–214: How does the analytic-skill mapping convert physical parameters and observations into actions? Please specify its implementation and required inputs, rather than only expressing it as a function. The observation is introduced earlier, but its use by each executor remains insufficiently explained.
-
Evidence store and diagnostic layers, lines 220–226: line 220 “evidence store” you mean a dictionary? “N ordered diagnostic layers” You mean the dictionary has N items?
-
Library revision, line 233: What precisely does the following operation mean? Does it add a capability, replace an implementation, or modify parameters?
-
Matched controlled comparisons, line 310: Which aspects are held fixed, and which remain system-specific? Please explain concretely what changes under the task and swap perturbations and how the perturbed instances are matched across methods.
-
Upper-model dependence: Beyond the small Sonnet replacement study, can the authors establish whether DynaHarness’s advantage persists in a full-benchmark comparison using the same stronger upper model across harnesses? Zetta’s higher reported Goal-T and Goal-S scores in Table 27 make this important.