- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Contact Code, a typed program interface for contact-rich robotic manipulation. Within predefined task templates and fixed primitive graphs, the method associates controller parameters, measured execution evidence, and permitted updates with individual contact stages. A VLM proposes a TaskSpec, which is validated and completed with task-specific defaults before being instantiated into an executable program. Controllers can perform bounded recovery within a rollout. Following an unsuccessful rollout, the system selects a repair base and an unresolved stage, obtains a constrained VLM-proposed patch, validates it, and resets the task before executing the revised program. The contribution is an interface connecting physical feedback to checkable, stage-local parameter repair within existing control skills, rather than the generation of new skills or arbitrary program structures.
The evaluation covers eight tasks and 800 held-out instances across ManiSkill, Isaac Lab, and UniVTAC-based simulation. The paper reports an increase in success from 60.4% on the first rollout to 73.6% after repair, and an average advantage of 8.3 percentage points over free-form parameter repair under matched execution feedback and rollout budgets. On a Flexiv robot, success increases from 20/25 to 24/25 trials. Since these physical trials use the same nominal initial configuration without deliberately varying the initial object poses, they primarily support interface portability rather than robustness across a broad distribution of initial conditions.
Critical strengths
-
A useful interface design with clearly stated boundaries. The integration of controller parameters, stage-aligned measurements, and permitted repair fields provides a concrete connection between physical execution feedback and program interventions. Explicit field ownership, stage-local patches, and atomic validation make the scope of changes inspectable, while the distinction between within-rollout recovery and between-rollout repair is clear.
-
Controlled repair experiments provide evidence beyond simple retries. Section 4.4 compares repair methods on 287 shared initial failures using the same frozen initial TaskSpec, controller, task contract, and execution budget. This comparison supports the value of parameter updates beyond merely repeating execution.
Critical weaknesses
-
The independent contribution of the central interface mechanism has not been sufficiently isolated. The comparison in Appendix F is useful because the two conditions share the same feedback, underlying controller, and rollout budget. However, the free-form condition rewrites a complete Python program that outputs parameters, while the contact code uses typed TaskSpec and stage-local patches. This changes both the generated representation and the editable scope. Furthermore, it is not clear whether these two conditions start from the same frozen initial TaskSpec.
-
The quantitative results of the audit using the corrected success metric are not reported. Appendix B clearly states that the upstream UV HDMI success expression uses p[1:2] while omitting other horizontal coordinates. The author recorded a revised dual-coordinate audit metric, but did not report its quantitative results. This is important because HDMI showed the greatest improvement from the first stage to the final stage, reaching 22 percentage points (see Section 4.2 and Table 1). Using the same upstream predicate allows for consistent comparison, but it is uncertain whether it fully measures physical completion.
Overall, I recognize the value of the system design, but the attribution of core benefits and the audit of unreported success metrics make the evidence insufficient to support my recommendation for acceptance. Therefore, I tend to lean towards a weak rejection. My assessment may change if the information boundaries are clarified, the existing audit results are reported, and a more direct comparison of cross-stage versus stage-local repair is provided.
Questions for authors
-
Clarification and targeted additional validation: Do the two conditions underlying the 8.3-percentage-point advantage in Appendix F start from exactly the same initial TaskSpec? Please report their respective first-rollout success rates and clarify whether default completion and validation behavior are matched. Could you compare typed cross-stage repair against typed single-stage repair with frozen initial TaskSpecs and all other factors held constant?
-
Clarification and analysis of existing results: In the failure-flag and uniform-search conditions, can the scheduler, repair-base selection, and repair history still access the actual measurement evidence? Please specify which components have their information restricted. Please also provide paired per-instance outcomes or an uncertainty interval for the difference between the measured-evidence and failure-flag conditions, to clarify the 78-versus-72 recovery result.
-
Reporting existing audit data: Please report a retrospective audit of the existing HDMI rollouts using the corrected two-lateral-coordinate predicate, including Contact Code's first and final outcomes and the same-feedback free-form repair condition. How many rollouts satisfy the original predicate but not the corrected one? How does this affect the method comparison on HDMI and the average result across the eight tasks?
7 comments
HDMI Success Metric Audit
W2 & Q3: How many HDMI success labels change under the corrected two-coordinate predicate, and how does this affect the HDMI results and the eight-task comparison?
Thank you for your feedback. Adding the missing x-coordinate check changes zero success labels in the audited Contact Code and free-form parameter repair rollouts.
Our experiments used UniVTAC’s then-current official evaluator. The audit adds the missing x-coordinate check while retaining the other conditions, matching the corrected official success criterion. Both predicates yield the same results:
| HDMI outcome | Contact Code | Free-form parameter repair |
|---|---|---|
| First-rollout successes | 52/100 | 25/100 |
| Final successes | 74/100 | 56/100 |
Contact Code’s first-to-final improvement remains 22 percentage points, its final-success advantage over free-form remains 18 percentage points on HDMI, and the eight-task average advantage remains 8.3 percentage points. These quantitative audit results are reported in Appendix B of the revised paper.
Measurement Access and Paired Recovery Analysis
Q2: Which components retain measurement access in the failure-flag and uniform-search conditions, and what is the uncertainty in the 78-versus-72 recovery difference?
Thank you for requesting this clarification. Both conditions retain measurement-based repair-base selection, stage scheduling, and internal execution history. The restrictions concern how patches are proposed.
1. Information access and paired analysis of the original comparison.
| Component or information | Measured-evidence VLM repair | Failure-flag VLM repair | Uniform search |
|---|---|---|---|
| Controller, repair-base selection, and stage scheduling | Use measurements | Use measurements | Use measurements |
| Internal execution history | Retains measurements | Retains measurements | Retains measurements |
| Explicit measurement and diagnostic guidance in the repair request | Provided | Omitted | No VLM |
| Attempt history supplied to the VLM | Parameters, outcomes, and measurements | Parameters, iteration indices, and failure flags | No VLM |
| Patch proposal | VLM-generated | VLM-generated | Random parameter updates within the permitted repair scope |
| Patch validation | Retained | Retained | Retained |
| Patch rejection feedback | Retained | Retained, potentially including measurement-derived guidance | No VLM |
Thus, the original failure-flag condition masks explicit request and history fields without fully excluding measurement-derived guidance. Uniform search replaces VLM-generated patches with random legal parameter updates while retaining the system’s measurement access.
Among the 287 shared first-rollout failures, recovery is 78 versus 72, with a paired difference of 2.1 percentage points, 95% CI [−2.8, +7.3], and exact two-sided McNemar p = 0.504.
2. Additional paired comparison with controlled repair-VLM inputs.
We added a comparison covering all 317 first-rollout failures in the main evaluation, using identical frozen initial TaskSpecs and first-rollout records. Both conditions share the controller, typed patch validation, repair-permission rules, repair-base selection rules, and stage scheduling rules, with up to four program iterations and three physical rollouts. Diagnosis-dependent patch-rejection gates are disabled in both to prevent rejection feedback from returning withheld guidance.
Providing measurement summaries and diagnostic guidance yields 88/317 recoveries (27.8%), compared with 72/317 (22.7%) without these inputs. The paired difference is 5.05 percentage points, 95% CI [−0.3, +10.4], with exact two-sided McNemar p = 0.093.
The updated Fig. 4(b) reports this new comparison. The original comparison is retained in Fig. 13 in Appendix F.1, with its information boundaries and paired analysis clarified. Both experiments concern explicit inputs to the repair VLM, while measurement-based mechanisms remain active in the system.
Initialization and Controlled Repair-Scope Comparison
Thank you for raising these concerns. We clarify the original comparison and report the requested controlled experiment.
1. Initialization and validation in the original comparison — W1 and Q1.
The original conditions use the same 800 task instances but independently generate their initial programs, rather than sharing frozen initial TaskSpecs. Both use the same controller, fixed primitive sequence, and measured execution feedback, with up to four program iterations and three physical rollouts.
Both apply missing-parameter completion, type/range checks, and static prechecks, but their handling differs. Contact Code completes missing or invalid TaskSpec fields using fallback values and uses system-calibrated values for certain fields. Free-form completes missing parameters while retaining valid model-generated values. Invalid free-form programs are rejected and count toward the program-iteration budget.
The paired final-success difference is 8.3 percentage points, 95% CI [5.1, 11.4]. We have clarified that this compares complete pipelines, including initialization and repair, rather than isolating repair representation or editable scope.
2. Controlled comparison of typed repair scope — W1 and the additional validation requested in Q1.
We appreciate your suggestion to examine editable scope directly. Typed cross-stage repair and free-form parameter repair are distinct conditions. The former retains TaskSpecs, typed patches, and validation while broadening the editable stage scope. The latter generates and rewrites Python programs to output execution parameters. Our reported 8.3-point advantage compares the complete Contact Code and free-form parameter repair pipelines. It does not represent a claim that single-stage typed repair is superior to typed cross-stage repair.
To address this question directly, we compared two controlled variants across eight tasks and 800 paired instances. They share frozen initial TaskSpecs, first-rollout records, typed representations, basic validation, and the same four-program-iteration and three-physical-rollout budget. Both retain measurements but omit stage-pointing diagnostic guidance and disable the same diagnosis-dependent patch restrictions. Only the editable stage scope varies.
The paired difference is Cross-stage − Single-stage = −0.88 percentage points, 95% CI [−3.0, +1.2], indicating similar observed success rates in this controlled comparison.