acceptodds
Withdrawn from ICLR 2027

Contact Code: A Typed Program Interface for Measurable and Repairable Contact-Rich Manipulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The ability to adapt to physical feedback is essential for robots performing contact-rich manipulation. Vision-language models (VLMs) provide a promising way to interpret task instructions and configure robot behavior from visual observations. However, adapting these decisions after physical interaction requires connecting execution feedback to the parameters that can be revised. We introduce Contact Code, a typed program interface that supports this connection by organizing controller parameters, measured execution evidence, and permitted updates around individual contact stages. Within predefined task templates, a VLM proposes a task configuration that is validated and instantiated into an executable program. During execution, robot motion and contact measurements provide feedback for controller-level recovery and evidence for subsequent program revision. If execution still fails, the system uses this evidence to select a contact stage and guides the VLM to propose a constrained update before the next attempt. We evaluate Contact Code on eight tasks and 800 held-out instances across ManiSkill, Isaac Lab, and UniVTAC-based simulation, spanning wrist force and fingertip tactile sensing as well as single-arm and bimanual settings. Contact Code increases success from 60.4% on the first rollout to 73.6% after repair and outperforms free-form parameter repair by 8.3% under the same feedback and rollout budget. On a Flexiv arm, the same planning and repair pipeline, with only the platform-facing adapters changed, improves peg-insertion success from 80% on the first trial to 96% after repair, providing evidence of portability across robot embodiments and control APIs. This interface provides a foundation for studying how physical feedback can guide program revision, offering a path toward more adaptable language-guided robotic systems.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Code-based Robot Control, Contact-Rich Manipulation, Robotics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NKvdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Contact Code, a typed program interface for contact-rich robotic manipulation. Within predefined task templates and fixed primitive graphs, the method associates controller parameters, measured execution evidence, and permitted updates with individual contact stages. A VLM proposes a TaskSpec, which is validated and completed with task-specific defaults before being instantiated into an executable program. Controllers can perform bounded recovery within a rollout. Following an unsuccessful rollout, the system selects a repair base and an unresolved stage, obtains a constrained VLM-proposed patch, validates it, and resets the task before executing the revised program. The contribution is an interface connecting physical feedback to checkable, stage-local parameter repair within existing control skills, rather than the generation of new skills or arbitrary program structures.

The evaluation covers eight tasks and 800 held-out instances across ManiSkill, Isaac Lab, and UniVTAC-based simulation. The paper reports an increase in success from 60.4% on the first rollout to 73.6% after repair, and an average advantage of 8.3 percentage points over free-form parameter repair under matched execution feedback and rollout budgets. On a Flexiv robot, success increases from 20/25 to 24/25 trials. Since these physical trials use the same nominal initial configuration without deliberately varying the initial object poses, they primarily support interface portability rather than robustness across a broad distribution of initial conditions.

Critical strengths
  • A useful interface design with clearly stated boundaries. The integration of controller parameters, stage-aligned measurements, and permitted repair fields provides a concrete connection between physical execution feedback and program interventions. Explicit field ownership, stage-local patches, and atomic validation make the scope of changes inspectable, while the distinction between within-rollout recovery and between-rollout repair is clear.

  • Controlled repair experiments provide evidence beyond simple retries. Section 4.4 compares repair methods on 287 shared initial failures using the same frozen initial TaskSpec, controller, task contract, and execution budget. This comparison supports the value of parameter updates beyond merely repeating execution.

Critical weaknesses
  • The independent contribution of the central interface mechanism has not been sufficiently isolated. The comparison in Appendix F is useful because the two conditions share the same feedback, underlying controller, and rollout budget. However, the free-form condition rewrites a complete Python program that outputs parameters, while the contact code uses typed TaskSpec and stage-local patches. This changes both the generated representation and the editable scope. Furthermore, it is not clear whether these two conditions start from the same frozen initial TaskSpec.

  • The quantitative results of the audit using the corrected success metric are not reported. Appendix B clearly states that the upstream UV HDMI success expression uses p[1:2] while omitting other horizontal coordinates. The author recorded a revised dual-coordinate audit metric, but did not report its quantitative results. This is important because HDMI showed the greatest improvement from the first stage to the final stage, reaching 22 percentage points (see Section 4.2 and Table 1). Using the same upstream predicate allows for consistent comparison, but it is uncertain whether it fully measures physical completion.

Overall, I recognize the value of the system design, but the attribution of core benefits and the audit of unreported success metrics make the evidence insufficient to support my recommendation for acceptance. Therefore, I tend to lean towards a weak rejection. My assessment may change if the information boundaries are clarified, the existing audit results are reported, and a more direct comparison of cross-stage versus stage-local repair is provided.

Questions for authors
  1. Clarification and targeted additional validation: Do the two conditions underlying the 8.3-percentage-point advantage in Appendix F start from exactly the same initial TaskSpec? Please report their respective first-rollout success rates and clarify whether default completion and validation behavior are matched. Could you compare typed cross-stage repair against typed single-stage repair with frozen initial TaskSpecs and all other factors held constant?

  2. Clarification and analysis of existing results: In the failure-flag and uniform-search conditions, can the scheduler, repair-base selection, and repair history still access the actual measurement evidence? Please specify which components have their information restricted. Please also provide paired per-instance outcomes or an uncertainty interval for the difference between the measured-evidence and failure-flag conditions, to clarify the 78-versus-72 recovery result.

  3. Reporting existing audit data: Please report a retrospective audit of the existing HDMI rollouts using the corrected two-lateral-coordinate predicate, including Contact Code's first and final outcomes and the same-feedback free-form repair condition. How many rollouts satisfy the original predicate but not the corrected one? How does this affect the method comparison on HDMI and the average result across the eight tasks?

7 comments
AuthorsOpenReview ↗

Initialization and Controlled Repair-Scope Comparison

W1 & Q1: Are initial TaskSpecs, default completion, and validation matched? How does single-stage typed repair compare with cross-stage typed repair under otherwise matched conditions?

Thank you for raising these concerns. We clarify the original comparison and report the requested controlled experiment.

1. Initialization and validation in the original comparison — W1 and Q1.

The original conditions use the same 800 task instances but independently generate their initial programs, rather than sharing frozen initial TaskSpecs. Both use the same controller, fixed primitive sequence, and measured execution feedback, with up to four program iterations and three physical rollouts.

Both apply missing-parameter completion, type/range checks, and static prechecks, but their handling differs. Contact Code completes missing or invalid TaskSpec fields using fallback values and uses system-calibrated values for certain fields. Free-form completes missing parameters while retaining valid model-generated values. Invalid free-form programs are rejected and count toward the program-iteration budget.

OutcomeContact CodeFree-form parameter repair
First-rollout success483/800 (60.4%)416/800 (52.0%)
Final success589/800 (73.6%)523/800 (65.4%)

The paired final-success difference is 8.3 percentage points, 95% CI [5.1, 11.4]. We have clarified that this compares complete pipelines, including initialization and repair, rather than isolating repair representation or editable scope.

2. Controlled comparison of typed repair scope — W1 and the additional validation requested in Q1.

We appreciate your suggestion to examine editable scope directly. Typed cross-stage repair and free-form parameter repair are distinct conditions. The former retains TaskSpecs, typed patches, and validation while broadening the editable stage scope. The latter generates and rewrites Python programs to output execution parameters. Our reported 8.3-point advantage compares the complete Contact Code and free-form parameter repair pipelines. It does not represent a claim that single-stage typed repair is superior to typed cross-stage repair.

To address this question directly, we compared two controlled variants across eight tasks and 800 paired instances. They share frozen initial TaskSpecs, first-rollout records, typed representations, basic validation, and the same four-program-iteration and three-physical-rollout budget. Both retain measurements but omit stage-pointing diagnostic guidance and disable the same diagnosis-dependent patch restrictions. Only the editable stage scope varies.

ConditionEditable scopeFinal success
Single-stage typed repairThe scheduler-selected stage582/800 (72.8%)
Cross-stage typed repairOne or more repairable stages575/800 (71.9%)

The paired difference is Cross-stage − Single-stage = −0.88 percentage points, 95% CI [−3.0, +1.2], indicating similar observed success rates in this controlled comparison.

AuthorsOpenReview ↗

HDMI Success Metric Audit

W2 & Q3: How many HDMI success labels change under the corrected two-coordinate predicate, and how does this affect the HDMI results and the eight-task comparison?

Thank you for your feedback. Adding the missing x-coordinate check changes zero success labels in the audited Contact Code and free-form parameter repair rollouts.

Our experiments used UniVTAC’s then-current official evaluator. The audit adds the missing x-coordinate check while retaining the other conditions, matching the corrected official success criterion. Both predicates yield the same results:

HDMI outcomeContact CodeFree-form parameter repair
First-rollout successes52/10025/100
Final successes74/10056/100

Contact Code’s first-to-final improvement remains 22 percentage points, its final-success advantage over free-form remains 18 percentage points on HDMI, and the eight-task average advantage remains 8.3 percentage points. These quantitative audit results are reported in Appendix B of the revised paper.

AuthorsOpenReview ↗

Measurement Access and Paired Recovery Analysis

Q2: Which components retain measurement access in the failure-flag and uniform-search conditions, and what is the uncertainty in the 78-versus-72 recovery difference?

Thank you for requesting this clarification. Both conditions retain measurement-based repair-base selection, stage scheduling, and internal execution history. The restrictions concern how patches are proposed.

1. Information access and paired analysis of the original comparison.

Component or informationMeasured-evidence VLM repairFailure-flag VLM repairUniform search
Controller, repair-base selection, and stage schedulingUse measurementsUse measurementsUse measurements
Internal execution historyRetains measurementsRetains measurementsRetains measurements
Explicit measurement and diagnostic guidance in the repair requestProvidedOmittedNo VLM
Attempt history supplied to the VLMParameters, outcomes, and measurementsParameters, iteration indices, and failure flagsNo VLM
Patch proposalVLM-generatedVLM-generatedRandom parameter updates within the permitted repair scope
Patch validationRetainedRetainedRetained
Patch rejection feedbackRetainedRetained, potentially including measurement-derived guidanceNo VLM

Thus, the original failure-flag condition masks explicit request and history fields without fully excluding measurement-derived guidance. Uniform search replaces VLM-generated patches with random legal parameter updates while retaining the system’s measurement access.

Among the 287 shared first-rollout failures, recovery is 78 versus 72, with a paired difference of 2.1 percentage points, 95% CI [−2.8, +7.3], and exact two-sided McNemar p = 0.504.

2. Additional paired comparison with controlled repair-VLM inputs.

We added a comparison covering all 317 first-rollout failures in the main evaluation, using identical frozen initial TaskSpecs and first-rollout records. Both conditions share the controller, typed patch validation, repair-permission rules, repair-base selection rules, and stage scheduling rules, with up to four program iterations and three physical rollouts. Diagnosis-dependent patch-rejection gates are disabled in both to prevent rejection feedback from returning withheld guidance.

Providing measurement summaries and diagnostic guidance yields 88/317 recoveries (27.8%), compared with 72/317 (22.7%) without these inputs. The paired difference is 5.05 percentage points, 95% CI [−0.3, +10.4], with exact two-sided McNemar p = 0.093.

The updated Fig. 4(b) reports this new comparison. The original comparison is retained in Fig. 13 in Appendix F.1, with its information boundaries and paired analysis clarified. Both experiments concern explicit inputs to the repair VLM, while measurement-based mechanisms remain active in the system.

Reviewer_odh2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents Contact Code, a typed interface for using force and tactile feedback to repair VLM-generated programs for contact-rich manipulation. Each task is represented by a fixed primitive graph, where every contact stage is associated with controller parameters, execution evidence, and permitted repair fields. After a failed rollout, the system uses the measured motion and contact signals to select a stage and asks the VLM to modify only the corresponding parameters. The method is evaluated on eight simulated tasks, covering force/torque and tactile feedback, as well as single-arm and bimanual manipulation. A real-robot experiment is also conducted for the task of peg insertion.

Critical strengths
  1. The idea of adapting the VLM decisions after physical interaction is an interesting idea. The typed task contract provides a clear way to constrain VLM-based program repair.

  2. The simulation study covers a reasonably broad set of tasks and sensing modalities.

  3. The overall method and experiments are simple and described well in the paper, which is easy to follow.

Critical weaknesses
  1. The main contribution appears to be a well-organized system built from predefined components rather than a new planning method. Primitive graphs, contact stages, evidence adapters, controllers, and repair fields are manually specified for each task, while the VLM mainly changes parameters within this fixed structure. The paper should clarify what is genuinely new beyond combining structured program generation, contact feedback, and language-model-based repair.

  2. The evaluation is somewhat questionable. In Table 1, the first-rollout success of Contact Code is much higher than the reported GenCHiP and CaP-X scores, which is surprising and not sufficiently explained. The baselines are evaluated under different protocols and do tehy receive the same stage-aligned force/tactile feedback or contact controller?

  3. The real-robot validation is very limited. The physical evaluation contains only 25 trials of one peg-insertion task on one robot arm, and does not cover tactile manipulation, bimanual tasks, or other contact modes used in simulation. Therefore, the current results provide limited support for the claims about transfer across embodiments and tasks.

Questions for authors
  1. Which parts of the primitive graph, evidence adapter, repair scope, and parameter ownership are learned or automatically inferred, and which are manually designed for each task?

  2. To what extent are the baselines evaluated under the same setup, including the task inputs and observations, prompt information, feedback, and reset protocol? Please clarify any unavoidable differences and how they may affect the comparison.

  3. In the simulated experiments, how are the object and fixture poses obtained? Does the method assume access to reset-time geometric information or accurate robot-object calibration? If so, how would perception and calibration errors affect the proposed repair procedure?

7 comments
AuthorsOpenReview ↗

Contribution Beyond Predefined Components

W1: The contribution appears to combine predefined components rather than introduce a new planning method. What is genuinely new beyond structured program generation, contact feedback, and language-model-based repair?

We agree that structured program generation, contact feedback, and language-model-based repair are established ideas. However, our central contribution is to make physical execution evidence actionable for contact execution, recovery, and program revision.

Contact Code provides an explicit, checkable connection through three functions:

  1. Stage-aligned execution and evidence. Each contact stage associates controller parameters with actual TCP motion and F/T or tactile measurements. During a rollout, these measurements guide contact behavior and controller-level recovery, while the system records stage-aligned execution evidence.
  2. Evidence-guided program revision. After a failed rollout, measured evidence and stage dependencies guide repair-base selection and identify the stage and permitted fields available for revision. The VLM proposes a typed patch within this scope.
  3. Validated parameter updates. Each patch is checked against field ownership, types, and numerical constraints before re-execution, while unmodified fields retain their values.

The manually specified components and their relationship to automatic program construction are detailed in our response to Q1 below.

AuthorsOpenReview ↗

Baseline Setup and Comparison

W2 & Q2: Why is Contact Code’s first-rollout success much higher than the baselines? How do task inputs, observations, prompts, controllers, feedback, and reset protocols differ, and how do these differences affect the comparison?

Thank you for raising these concerns. We address W2 and Q2 together, clarifying the shared evaluation setup, method-specific interfaces and protocols, and their implications for the comparison.

1. Shared evaluation setup, inputs, and prompts — Q2.

On each method’s supported tasks, we use the same held-out seeds (100–199), initial scene exports, task objectives, and simulator success evaluator. All Table 1 conditions use Qwen3-VL-Plus. We reimplement each method on our task environments while retaining its method-specific interfaces and protocols.

MethodInputs and observationsPrompt information
Contact CodeInitial sensor images and simulator-exported reset-time geometry.Task instruction, planning-visible scene information, task contract, grounding vocabulary, permitted fields, and constraints.
CaP-X M4-style reimplementationVDM scene description from the initial camera image and pose estimates queried through the perception API.Task instruction, pose-query/position-motion/gripper API documentation, and worked examples.
GenCHiP reimplementationInitial RGB image and a fixed reset-time scene estimate queried through the API.Task instruction, compliant-motion/gripper API documentation, and usage examples.

None receives evaluator-only poses, hidden oracle state, or intermediate success-evaluation metrics. Temperatures are 0.1 for Contact Code, 1.0 for CaP-X, and 0.0 for GenCHiP. The latter two follow the public launcher’s default and the original paper’s reported experimental setting, respectively.

2. Execution interfaces, feedback, and reset protocols — W2 and the protocol differences in Q2.

The baselines do not share Contact Code’s contact controller or stage-aligned contact measurements. Their interfaces and protocols are summarized below.

DimensionContact CodeCaP-X M4-style reimplementationGenCHiP reimplementation
Execution interfaceTaskSpec with task-specific contact controllerPython with Cartesian position-motion and gripper APIsPython with compliant Cartesian-motion and gripper APIs
Repair feedbackStage-aligned TCP motion and F/T or tactile evidencePrevious code, stdout/stderr, binary outcome, and visual-difference descriptionsNo failure feedback passed between candidates
Reset protocolReset to the same seed after failureContinue within one persistent episodeReset to the same seed for each candidate
Generation and execution budgetUp to four program iterations and three rollouts, with up to five response attempts per repairUp to four code versions and three executions, with one candidate per turnFive completions with execution-based selection and no iterative repair

3. First-rollout success and interpretation of the comparison — W2 and the impact of these differences in Q2.

Contact Code’s first rollout already includes measurement-conditioned contact control and controller-level recovery. Its initial success therefore reflects the combined TaskSpec, primitive graph, and controller. Differences in geometric estimation, available control abstractions, feedback, and reset protocols also affect the comparison. Table 1 compares complete pipelines, and we do not attribute the entire gap to the repair interface.

The separate comparison in Appendix F matches the controller, measured feedback, fixed primitive sequence, reset protocol, and rollout limits between Contact Code and free-form parameter repair.

We additionally evaluated the released CaP-Agent0 agent loop under adapted task interfaces, obtaining 0/50 success on MS Peg and 1/50 on UV HDMI. These results further indicate that directly applying the evaluated CaP-X pipeline to these two contact-rich tasks remains challenging. These supplementary pipeline-level results are reported separately in Appendix D.3. We have clarified the scope of these comparisons in Table 1, the accompanying discussion, and Appendix D.

AuthorsOpenReview ↗

Additional Real-Robot Validation

W3: Real-robot validation covers only 25 peg-insertion trials on one robot arm, without tactile or bimanual manipulation, providing limited support for transfer across embodiments and tasks.

Thank you for pointing this out. We agree that the original real-robot evaluation was limited in task diversity and trial count. We selected peg insertion as a representative setting for studying program revision under alignment errors, unsuccessful entry search, and insertion jams.

To broaden the physical task and contact-mode coverage, we have added 25 real-robot nut-threading trials on the same Flexiv platform. Nut threading introduces thread engagement and coupled rotational–axial motion beyond peg insertion. Success increased from 8/25 (32%) on the first rollout to 15/25 (60%) after repair, recovering seven initial failures. Together with the original peg-insertion experiments, the physical evaluation now comprises 50 trials across two tasks.

In one representative nut-threading trial, execution completed the prescribed half-turn and axial-feed sequence but failed the terminal TCP-height criterion. The VLM proposed a typed patch increasing the angular step from 10° to 15°. After validation and re-execution, the next rollout satisfied the joint criteria for rotation, axial feed, and terminal TCP position.

These results broaden the empirical support for the interface and repair procedure across two physical contact modes. However, the evaluation still uses one robot platform and does not cover real-world tactile or bimanual manipulation. We therefore interpret these results as initial evidence of interface portability and applicability to the tested physical tasks, rather than broad generalization across embodiments and tasks.

Reviewer_69kROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Contact Code, a typed program interface for contact-rich manipulation. It connects execution feedback to stage-local program repair, and uses predefined task templates, contact measurements, and contract-constrained VLM updates to improve task success after failed attempts. Experiments are conducted on eight simulated tasks and a real-robot peg-insertion task to demonstrate its effectiveness.

Critical strengths
  • Practical problem. The paper addresses an important and practical challenge: how to use physical execution feedback to improve robot behavior after failed contact-rich manipulation.

  • Structured design. The typed interface provides a clear and verifiable process for associating execution evidence with repairable parameters.

  • The experiments cover eight simulation tasks with additional real-robot validation and useful ablations.

Critical weaknesses
  • Limited novelty and task-specific design. The main contribution appears to be the typed-stage interface, which organizes execution evidence and permitted parameter updates around individual contact stages. However, the approach relies on predefined task templates, primitive graphs, task-specific feedback adapters, and repairable parameters. The paper does not sufficiently demonstrate how much of this task-specific engineering can be reused across different tasks, limiting the evidence for its generality.

  • Limited real-world validation. The real-robot experiments focus on a single peg-insertion task with 25 trials. While the results are promising, broader real-world evaluations are needed to establish the method’s robustness across different manipulation tasks and failure conditions.

  • In Figure 4(b), repair using measured evidence recovers 78 out of 287 initial failures, compared with 72 cases when only failure feedback is provided. This relatively small difference raises questions about the necessity and contribution of the measured-evidence module.

Questions for authors
  • How does Contact Code differ fundamentally from existing structured robot program generation and feedback-based repair methods, beyond introducing a typed, stage-local interface?

  • How much manual engineering is required to apply Contact Code to a new task? Which components can be reused across tasks?

  • Are there failure modes that cannot be resolved by modifying the predefined parameters of existing contact stages? If so, how does the framework detect and handle such failures?

6 comments
AuthorsOpenReview ↗

Novelty, manual engineering, and cross-task reuse

W1 & Q1 & Q2: What distinguishes Contact Code from existing program-generation and feedback-based repair methods, how much manual engineering is required for a new task, and which components can be reused across tasks?

Thank you for your constructive feedback. We address W1, Q1, and Q2 together in this part, distinguishing the contribution of Contact Code from the task-specific components needed to deploy it.

1. Contribution beyond the interface representation — Q1 and the novelty concern in W1.

We agree that structured robot programs and feedback-based repair are established ideas. Our central contribution is to make physical execution evidence actionable for contact execution, recovery, and program revision. After a failed rollout, Contact Code uses this evidence to select which previous configuration to retain, which contact stage to revise, and which parameters to modify. The interface makes this connection explicit and checkable by validating each proposed patch against stage ownership, types, and numerical constraints before re-execution.

MethodExecution feedback for control or recoveryStage/subgoal-aligned execution evidenceProgram structure/type checksEvidence-constrained repair fields and patch validation
ProgPrompt✓———
GenCHiP✓———
REFLECT✓✓——
CaP-X✓———
GaP✓✓✓—
Contact Code✓✓✓✓

✓ indicates an explicitly described mechanism; — indicates that the mechanism is not described.

2. Reuse and manual engineering in the original system — Q2 and the task-specific-design concern in W1.

In the original system, all eight tasks share the TaskSpec representation, typed patches, validation, and execution/repair procedures. However, applying the system to a task requires specifying its graph template, evidence adapter, and repairable parameter domains, together with the necessary controller bindings. Six graph-building modules serve the eight tasks, with shared builders for MS Peg/Plug and FG Peg/Gear. Task-specific configuration is therefore partly reused, while the remaining graph, evidence, and repair-domain definitions are supplied manually. We acknowledge that the original evaluation demonstrates applicability across multiple contact settings, rather than automatic construction of programs for new tasks.

3. Newly added automatic construction and reuse evaluation — addressing the remaining concerns in W1 and Q2.

To reduce this task-specific configuration burden, we have additionally developed an automatic program-construction extension and evaluated it on the same eight tasks. This new extension retains Contact Code’s shared representation and execution/repair procedures, while automating the following configuration steps:

ComponentNewly automated stepStill human-provided
Program structureThe VLM composes shared primitives; the system validates and constructs the graphPrimitive implementations, task instructions, physical facts, and grounding
Evidence bindingsThe system derives stage-aligned bindings from primitive result fields and backend capabilitiesMeasurement-processing capabilities and sensor/backend adapters
Repair schemasThe system derives parameter ownership, permitted repair fields, and patch schemasParameter declarations, repair permissions, and numerical bounds

The constructed programs require no post-generation manual edits. For each backbone, six of the ten primitive types used across its eight automatically constructed programs are reused across tasks.

We further evaluated Insert Tube from UniVTAC, a task beyond the original eight, using the same primitive library and interface. Integration required adapting the HDMI runner and grounding, without modifying existing primitives or shared algorithms. The automatically constructed five-stage program achieved 99/100 first-rollout success and 100/100 final success.

These additional results provide evidence that the new extension can reduce per-task program specification and reuse existing capabilities for a related task. New contact skills or sensing capabilities still require additional primitive or backend implementations.

AuthorsOpenReview ↗

Limited real-world validation

W2: The real-robot evaluation covers only peg insertion with 25 trials, providing limited evidence of robustness across manipulation tasks and failure conditions.

Thank you for pointing this out. We agree that the original real-robot evaluation was limited in task diversity and trial count. We select peg insertion because it provides a representative setting for studying how physical execution evidence guides program revision under alignment errors, unsuccessful entry search, and insertion jams.

To broaden the evaluation across manipulation tasks, we have added real-robot nut-threading experiments. This task introduces thread engagement and coupled rotational-axial motion, extending the evaluation beyond peg insertion. Across 25 trials on the same Flexiv platform, success rate increases from 8/25 (32%) on the first rollout to 15/25 (60%) after repair.

For example, in one nut-threading trial, the first rollout completed the prescribed half-turn and axial feed but did not meet the terminal TCP-height criterion. The VLM proposed a typed patch that increased the angular step size from 10° to 15°. After the system validated the patch and re-executed the program, the next rollout satisfied the joint success criteria for rotation, axial feed, and terminal TCP position.

These additional results broaden the empirical support for the interface and repair procedure.

AuthorsOpenReview ↗

Necessity and contribution of measured evidence

W3: Measured-evidence repair recovers 78 of 287 initial failures, compared with 72 for failure-only repair. This small difference raises questions about the necessity and contribution of measured evidence.

We respectfully clarify that Fig. 3(b) is our primary ablation of physical feedback. These ablations show that physical measurements contribute to task success by guiding contact behavior, informing execution verdicts, and supporting controller-level recovery.

The original Fig. 4(b) (now Fig. 13 in Appendix F.1) instead evaluates the recovery outcomes of VLM-generated patches under different input conditions, while measurement-based control, repair-base selection, and stage scheduling remain active. The condition previously labeled “failure-only” omits explicit measurement summaries and diagnostic guidance from the repair request and removes measurement summaries from its attempt history. Thus, this comparison examines explicit evidence inputs to patch generation, rather than the overall contribution of physical feedback to Contact Code. We acknowledge that the paired recovery difference between 78/287 and 72/287 is 2.1 percentage points (95% CI [−2.8, +7.3]) and is not statistically significant. The revised “failure-only” label clarifies that this condition restricts direct VLM inputs while measurement-based mechanisms remain active.

To examine these inputs under tighter controls, we added a paired comparison using identical frozen initial TaskSpecs and first-rollout records for all 317 initial failures in the main evaluation. Both conditions share the controller, typed repair permissions, legality checks, and repair-base selection and stage scheduling rules, with up to four program iterations and three physical rollouts. Diagnosis-dependent patch-rejection gates are disabled in both to prevent rejection feedback from returning withheld guidance. Providing measurement summaries and diagnostic guidance yields 88 recoveries versus 72 without these inputs, with a paired difference of 5.05 percentage points (95% CI [−0.3, +10.4]). The updated Fig. 4(b) reports this comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.