acceptodds
Under review as a conference paper at ICLR 2027

AffordCode: Learning Generalizable Affordances as Executable Programs

Abstract

Robotic affordance learning treats affordances as prediction outputs, including interaction regions, poses, or trajectories. Advances in language models and programmatic object modeling enable inspectable, editable, and composable affordance programs, whose potential for feedback-driven refinement and cross-instance reuse remains underexplored.To bridge this gap, we present AffordCode, an agentic framework that learns reusable affordance programs over parameterized object templates exposing part geometry and composition. Following inter-part structural relationships, AffordCode constructs functional chains bottom-up, composing part-level functions into object-level programs that compute instance-specific affordances. A closed-loop procedure verifies candidate programs, traces structural, geometric, and interaction feedback to faulty assumptions and dependencies, and refines the corresponding code. This method offers three advantages: (1) Precise Feedback-Driven Refinement where editable code exposes faulty interaction logic; (2) Coherent Object-Level Behavior since bottom-up composition preserves part dependencies and aligns local functions with the task; and (3) Synthesis-Free Reuse: validated part functions can be recomposed and object programs instantiated across compatible objects. Extensive experiments demonstrate the effectiveness of the generated affordances, their generalization across object instances, and their utility in downstream tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robotic affordance learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_L7FvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AffordCode is an agentic framework that acquires reusable affordance programs over parameterized object representations. Using ConceptFactory/Digital-Gene interfaces, a language-model agent retrieves or synthesizes part-level routines and composes them into object-level interaction logic. Executing a program with new object parameters produces regions, contact poses, motion quantities, or trajectories. Structural, geometric, and interaction checks provide feedback for revising implicated code, and validated routines are retained in a hierarchical memory. Under the default protocol, each template-task program is acquired using eight sampled instances and at most three refinement rounds; programs and memory are frozen before downstream evaluation.

The contribution is the organization, refinement, and reuse of executable interaction knowledge. Evaluation covers 479 simulated interactions with representation-specific ground-truth inputs, 42 RGB-D RoboTwin tasks, demonstration generation for four policy-learning tasks, and affordance guidance for a frozen policy on two articulated-manipulation tasks. The paper reports improvements across these settings, supplemented by ten qualitative real-world object-task examples and a real-robot scissors-to-cabinet demonstration.

Critical strengths
  1. The representation provides a concrete mechanism for inspecting and reusing interaction rules. The division between local part computations and object-level composition makes the dependencies among contact geometry, coordinate frames, motion axes, and task outputs explicit. Listing 2 demonstrates this through handle contact computation, hinge geometry, and a door-opening trajectory. Re-evaluating these functions on new parameters explains how a stored program adapts to compatible instances. Appendix A.2 also specifies the compatibility requirements, while Appendix A.5 distinguishes direct instantiation from further acquisition for new tasks or compositions. This is a useful design for editable and transferable affordance knowledge, and a meaningful application of program synthesis to structured object interaction.

  2. The experimental results provide substantial evidence of downstream utility, including an informative controlled ablation. AffordCode reaches 42.80% interaction success versus 26.93% for GAPartNet-GT in Table 1, and 36.67% average RoboTwin success versus 19.76% for ReKep in Table 2, with nonzero success on 35 of 42 tasks. In the matched 20-mesh × 50-demonstration setting, policies trained on its demonstrations achieve 65.2% average success on unseen meshes versus 51.0% for AffordGen. Particularly useful is Table 4: all affordance-guided variants share a frozen downstream policy, splits, and execution settings. The full method improves over the unverified variant by 6-10 percentage points and over direct object-level synthesis by 5-7 points across the six task-split combinations. These controls support the usefulness of verification and part-based composition, and the explicit separation of acquisition from evaluation strengthens the evidence.

Critical weaknesses
  1. The implementation and independence of the verification signals are insufficiently specified for a central contribution. Equations (4)-(5) define a rollout and task predicates, but Appendix A.4.1 allows geometric propagation, kinematic propagation, or an execution backend without identifying which is used in each experiment. I could not determine the concrete implementations, tolerances, and information sources for the relevant task predicates, contact/collision checks, and motion constraints. This matters particularly for hypotheses such as the hinge side and axis in Listing 2, which the authors explicitly describe as assumptions to be checked. If a rollout and its success predicate both inherit an incorrect candidate hinge hypothesis, passing them could establish internal consistency without independently validating that hypothesis. The manuscript does not provide enough implementation detail to assess how this possibility is excluded. The authors appropriately acknowledge that geometric or kinematic verification does not guarantee contact-dynamic feasibility; my concern is the evidence supporting correction of the underlying interaction assumptions. An executable example of the checker and an actual failure/revision trace would materially strengthen this central claim.

  2. The amount of newly acquired knowledge and the scope of program reuse are not sufficiently disentangled from the supplied priors. Appendix A.5 initializes part memory with template routines and previously generated programs, but does not provide a benchmark-specific inventory of their provenance or existing affordance content. The geometry-only example in Listing 1 is helpful, yet does not establish the contents of the actual experimental memory. Likewise, Table 4 defines unseen objects/categories in the downstream-policy setting without establishing whether those categories or part compositions were excluded from AffordCode's initial memory and acquisition. The stated validation/evaluation instance separation and frozen memory are valuable, but do not resolve that additional distinction. Consequently, it is difficult to quantify how much success comes from newly learned rules, direct reuse within a compatible template, or acquisition using familiar part routines.

Questions for authors
  1. Clarification of the implemented verifier. For the settings in Tables 1-4, which validation backend and concrete task predicates are used during acquisition, and where do their geometry, kinematics, and success conditions come from? Which checks are deterministic and which depend on VLM judgment? Please provide a representative executable checker, including its tolerances, and one actual trace showing how an incorrect hinge-axis or motion-direction hypothesis is rejected using evidence independent of that hypothesis, revised, and re-validated. This would address Weakness 1 without requiring a contact-dynamics simulator for every output type.

  2. Clarification of knowledge provenance, splits, and coverage. Please report the initial part/object memory contents and provenance, the number of acquired template-task programs, and how acquisition instances and parameter ranges relate to the evaluated objects. In particular, are Table 4's unseen categories also unseen to AffordCode's initial memory and program acquisition, or only to the frozen policy? Please distinguish reuse by direct instantiation from acquisition of programs for new compositions, and explain how unsuccessful acquisitions or incompatible instances enter the reported success-rate denominators. These details would determine the extent of the supported learning and reuse claims.

  3. Control clarification and targeted additional evidence. Do AffordCode-Direct and AffordCode-Verified receive the same initial routines, validation instances, tool access, and model-call/refinement budgets? Beyond this existing ablation, a quantitative comparison on a representative subset of the current tasks against a general coding agent or direct use of existing affordance routines, with these factors matched, could resolve Weakness 2. Please report any available variation across independently acquired programs or paired episode outcomes, and state explicitly if acquisition was run only once. The control details are clarification requests; the matched comparison is a targeted request for additional evidence about the present contribution.

3 comments
AuthorsOpenReview ↗

Response to Reviewer L7Fv (Part I)

W1&Q1. Verification during program acquisition

Thank you for the detailed questions. We added Algorithm 1 in Appendix A.4.1, specifying the verification stages, implemented predicates, numerical tolerances, and acceptance rule.

For the verified configurations in Tables 1-4, kinematic propagation is the acquisition backend for outputs requiring a rollout in Eq. (4). The instantiated validation template supplies the initial meshes and part poses; candidate tool poses drive propagation, and the fixed task specification defines the intended state change.

The VLM participates only in geometric feasibility and interaction consistency. Structural validity checks program execution and output contracts. Geometric feasibility combines applicable numerical predicates with VLM review of rendered outputs against the object geometry and task's spatial requirements. For trajectories, interaction consistency checks motion properties such as fixed-axis consistency, while the VLM reviews hinge placement, maintained attachment, and requested motion direction. All applicable required checks must pass; a visual pass does not override a numerical failure.

The reviewer is right that a trajectory can satisfy a fixed-axis check while rotating about the wrong edge. Fitting an axis from candidate poses establishes internal consistency, not whether that axis matches the object's connection. One could use the handle position to infer the free edge and place the hinge on the opposite side, but this cue depends on the handle layout and joint design. Encoding such cues as hard predicates can require experts to specify mechanism-specific rules and maintain their exceptions. We therefore use VLM review to consider the initial door geometry, cabinet connection, and handle layout together with the original task instruction. These references remain fixed while the candidate axis and motion change, providing evidence beyond the candidate's internal consistency.

We have also uploaded a supplementary microwave verification case with complete VLM inputs and responses and multi-view candidate renderings, documenting rejection of an incorrect hinge-axis motion and revalidation of the corrected motion.

Q2. Knowledge provenance, splits, and coverage

Initial memory. AffordCode uses the same three initial few-shot examples across benchmarks, including the real-world experiments; the w/o Few-Shot ablation explicitly omits them. Their object categories do not overlap with the evaluated categories. Their contents are now listed in Appendix A.5, Table 8.

Object categoryPart-Level Affordance PrimitivesObject-Level Affordance Knowledge
WindowHandle grasp pose; window sliding axisSlide window open
Washing MachineDoor hinge pivot; door hinge axis direction; door grasp poseOpen washing machine door
RefrigeratorDoor hinge pivot; door hinge axis direction; handle grasp poseOpen refrigerator door

Acquisition instances and evaluation objects. Acquisition instances are sampled from each template's valid parameter ranges, separately from evaluation objects. The parameter domain may overlap, but evaluation instances and their outcomes are not used for acquisition or refinement. Programs and memory are frozen before evaluation.

Unseen categories in Table 4. For Unseen Instance, an existing program is instantiated on new objects from a known category. For Unseen Category, target categories are unseen in both policy training and AffordCode's initial memory. Before evaluation, AffordCode acquires the corresponding programs on separate acquisition instances, freezes them, and instantiates them on the evaluation objects.

Program reuse. The initial few-shot object-level programs cannot be directly instantiated to solve the evaluated tasks. Direct reuse applies an acquired, frozen program unchanged to compatible evaluation objects, whose parameters determine the outputs. For new part compositions or tasks, the agent acquires and verifies a new object-level program, reusing compatible part routines and related object-level programs as context. Appendix A.5 clarifies these two forms of reuse.

How acquisition failures and incompatible instances are counted. Except for interruptions caused by external factors such as network outages, every scheduled evaluation case remains in the denominator. An incompatible instance counts as a failure. If acquisition fails, every evaluation case requiring that program also counts as a failure; none is removed from the evaluation set.

AuthorsOpenReview ↗

Response to Reviewer L7Fv (Part II)

W2&Q3. Acquisition controls and comparison with a general coding agent

In Table 4, AffordCode-Direct and AffordCode-Verified use the same initial routines, validation instances, tool access, and limits on model calls and refinement rounds. Their verification procedure and downstream evaluation setup are also identical. Direct synthesizes the object-level program directly, while Verified composes part-level routines.

We added a matched comparison with GPT-5.6 Sol via Codex CLI, providing the same initial code memory, validation instances, tools, feedback, and acquisition budget as Verified. The coding agent organizes synthesis and revision itself. For this comparison, acquisition was run once for each object template and task, including verification and refinement within that run. The programs were then frozen and evaluated using the same downstream policy on the same 100 episodes for each task and split. The controls and results are included in Appendix A.6.1, Table 9.

TaskMethodSeen InstanceUnseen InstanceUnseen Category
PullDrawerGPT-5.6 Sol (Codex CLI)88%52%60%
PullDrawerAffordCode-Verified92%60%72%
OpenDoorGPT-5.6 Sol (Codex CLI)71%49%23%
OpenDoorAffordCode-Verified80%55%33%

Verified improves success in all six settings, by 8.2 percentage points on average, with gains of 12 and 10 points on Unseen Category for PullDrawer and OpenDoor. With prior code and validation resources matched, these results support the contribution of AffordCode's structured acquisition procedure.

Variation across independently acquired programs. We also report five independent acquisition runs of AffordCode-Verified for each task under Unseen Category. Each run starts from the same initial code memory and follows the same acquisition protocol, including its own verification and refinement. The resulting programs are frozen and evaluated with the same downstream policy on the same 100 episodes. Appendix A.6.2, Table 10 reports the individual success rates and their mean ± sample standard deviation.

TaskSuccess rates across five acquisitions (%)Mean SR ± SD (%)
PullDrawer72, 73, 69, 73, 7672.6 ± 2.5
OpenDoor33, 35, 31, 33, 3633.6 ± 1.9

These repeated acquisitions reproduce the original performance levels, with standard deviations of 2.5 and 1.9 percentage points for PullDrawer and OpenDoor, respectively.

Thanks for the additional experiments to clarify my concerns. I would like to maintain my score.

Reviewer_pooUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an agentic approach for articulated object manipulation. Building on existing line of works that use visual affordance for articulated object manipulation, the main challenge being tackled by the work is to come up with parametrizations for known motion primitives (e.g., grasp and pull) given an articulated object at test time — which is sometimes referred to as affordances. Specifically, this work uses an agentic approach to tackle this problem by letting a coding agent to write programmatic representations of such “affordances” on top of given programmatic object templates. Then these “affordance programs” are verified and refined autonomously by the agent before being used for object manipulation. Evaluations are conducted on a set of articulated objects and compared against learning-based and VLM-based baselines with focus on task success rates. Evaluations are primarily conducted in simulation with very limited real world demonstrations.

Critical strengths
  • It’s timely to how agentic approaches enable better generalization in articulated object manipulation
  • It is interesting to see how well articulated objects can be represented programmatically, which might be a compelling interface for agentic-based approaches to manipulation
Critical weaknesses
  • Very limited real-world demonstrations: The paper primarily conducts experiments in simulation. While there is a section that shows application to real-world scenarios, only affordance prediction is shown and only qualitative results are shown. It also appears that some objects being studied are also taken from the web rather than real in-the-wild captures. The supplementary also includes a real-robot qualitative demonstration. However, there is extremely limited information about the demonstration (e.g., what primitives are being used, what the affordance predictions are, what object templates are being used, agent execution traces, etc). Furthermore, it seems that there is only one demonstration for one initialization of the task, which is extremely limited and uninformative to understand the limits of the system.
  • Uninformative metrics: The paper relies primarily for task success metrics when compared to baselines that presumably operate on very different assumptions, e.g., some might require demonstrations and some don’t, and some might assume programmatic object representations to begin with and some don’t. While the proposed method is shown to perform the best, using only the success rate is also quite uninformative to help understand the system.
  • Very limited qualitative visualizations: It is hard to understand the specific setting of the conducted evaluations (which are primarily in simulation) because there is almost no qualitative visualization at all throughout the paper draft and the supplementary. These visualizations can help understand the behaviors of the system and possibly why the system performs better than the baselines.
  • Motion primitives: Based on previous works in affordance learning, it is often the case that motion primitives are required to couple with “affordance prediction” to perform the manipulation tasks, but it seems that information regarding this is missing. It is also often the case that these primitives are primarily the bottlenecks of these works when generalizing to more challenging manipulation tasks beyond a short motion for manipulating a known category of articulated object. More discussions on this can greatly shed light on the contributions of the paper.
Questions for authors

See weaknesses above

2 comments
AuthorsOpenReview ↗

Response to Reviewer pooU (Part I)

Q1. Additional real-world experiments and execution details

Thank you for raising these concerns. Some photographs in Figure 5 are web-sourced examples used for affordance visualization. To evaluate physical execution quantitatively, we conducted five tasks with ten trials per task, using Flexiv Rizon 4 and RealMan RM65 arms, with initial configurations varied between trials. We supplied the corresponding parameterized templates and GT geometric parameters, so these experiments assess execution with known object geometry.

Real-world taskSuccessful trials
(a) Place scissors in a drawer8/10 (80%)
(b) Open a suitcase10/10 (100%)
(c) Place scissors in a drawer and close it7/10 (70%)
(d) Place a block in a bowl10/10 (100%)
(e) Pull a drawer open10/10 (100%)
Overall45/50 (90%)

These results quantify repeated physical execution across the tested tasks and initial configurations. We added the results and initial, intermediate, and final camera frames in Appendix A.10, Table 14 and Figure 6.

To explain the demonstration, Appendix A.10.1, Figure 7 connects the cabinet template to program generation and execution. The template comprises Cuboid, Single_Drawer_Body, Single_Drawer_Front, and Round_Drawer_Handle parts. Given this template and the instruction "Open the top drawer," AffordCode generates reusable part-level primitives and object-level interaction logic offline. Instantiating the program computes the interaction trajectory in the object frame; the cabinet pose estimated from the point cloud is used to transform it into the robot's execution frame.

Q2. Evaluation metrics and comparison settings

We added normalized distance to Section 4.1, Table 1, following the definition used in FlowBot3D. Success rate provides a common measure of task completion across different affordance representations; normalized distance additionally captures partial progress. Both metrics use the same 479 evaluation cases:

Egoal=∥qend−qgoal∥∥qinit−qgoal∥.E_{\mathrm{goal}} = \frac{\lVert q_{\mathrm{end}}-q_{\mathrm{goal}}\rVert}{\lVert q_{\mathrm{init}}-q_{\mathrm{goal}}\rVert}.

Here, qinitq_{init}, qgoalq_{goal}, and qendq_{end} denote the initial, target, and final object states used by the task evaluator. The metric measures the fraction of initial goal error remaining: 0 means the target is reached, and 1 means unchanged goal error. Lower is better.

MethodDoorDrawerScreenCoverLift lidHandleKnobOverall
GAPartNet-GT0.78890.48420.90620.63840.95210.57440.92970.6953
Where2Act-GT0.82580.48460.88110.71320.94120.48130.93840.7105
FlowBot3D-GT0.83510.73120.93820.76390.98360.67830.92140.8075
AffordCode0.62280.35100.21390.65890.91480.49590.73490.5288

AffordCode achieves the lowest overall distance, 0.5288, versus 0.6953 for the best baseline. Its interactions therefore also finish closer to the task goals on average, complementing the success-rate results.

To further clarify the comparison settings, we address the two examples raised by the reviewer.

Demonstrations. AffordCode generates demonstrations without a source expert trajectory. In Section 4.3, Table 3, every method supplies 1,000 demonstrations per task, and we use the same downstream policy architecture and training/evaluation protocol. This keeps policy learning fixed when comparing how well each method's demonstrations support task performance and generalization.

GT inputs. We also evaluated GPT-5.6 Sol via Codex CLI on the same 100 interaction cases selected from Table 1's original benchmark, changing only the GT input. Each GT configuration follows Section 4.1 exactly. The protocol and results are included in Appendix A.7, Table 11.

GT inputSuccess rate
AffordCode-GT14%
GAPartNet-GT13%
Where2Act-GT21%
FlowBot3D-GT23%

With the agent fixed, AffordCode-GT does not show a consistent task-performance advantage over the other GT settings. Thus, the more explicit structural description does not automatically make task completion easier for the agent.

AuthorsOpenReview ↗

Response to Reviewer pooU (Part II)

Q3. Qualitative visualization

We added an OpenDoor comparison in Section 4.4, Figure 4, showing the affordance guidance and subsequent execution under the same frozen AffordDP policy.

AffordCode's guidance follows an arc with a fitted XY radius of 0.43 m, close to the 0.41 m simulator reference. Original-AffordDP's transferred guidance follows a flatter arc with a fitted radius of 0.71 m (Figure 4a). The execution frames show that the baseline opens the door slightly but fails to reach the required angle, whereas the policy completes the task with AffordCode's guidance (Figure 4b).

In this episode, the guidance mismatch is accompanied by incomplete opening, while guidance closer to the door's rotational geometry supports successful execution. This provides a concrete illustration of how the affordance guidance affects the behavior of the same downstream policy.

Q4. Motion primitives and execution

In our direct trajectory-execution experiments, the robot follows the end-effector poses generated by the affordance program through its controller. The only predefined action primitive is gripper opening and closing, used for grasping and release. The part-level affordance primitives in Figure 2 are computational functions that provide quantities such as grasp poses and motion axes to the object-level program.

Some prior methods connect affordance predictions to predefined execution routines, such as the pre-programmed push/pull trajectories in Where2Act and the part-pose-based manipulation heuristics demonstrated in GAPartNet. AffordCode instead synthesizes the motion logic as executable code from the task and object template, computing the interaction trajectory without invoking predefined arm-motion primitives.

Our real-robot evaluation also includes a multi-stage task: placing scissors in a drawer and then closing it (Figure 6c). However, we have not conducted a controlled comparison with a primitive-based system to quantify the potential benefits of this design for motion adaptation and task composition. We have explicitly acknowledged this limitation in Appendix A.11 (Limitations) of the revised manuscript.

Reviewer_UQCROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AffordCode, an agentic framework that represents robotic affordances as executable programs over parameterized object templates. It constructs object-level interaction programs by composing reusable part-level functions and iteratively refines them through structural, geometric, and interaction feedback. Experiments on GAPartNet, RoboTwin, ManiSkill, and real-world objects demonstrate improvements in manipulation success, affordance transfer, and downstream policy learning.

Critical strengths
  1. The paper introduces an interesting approach to representing robotic affordances as executable programs rather than fixed spatial predictions. This representation makes interaction rules more interpretable, editable, and reusable.
  2. The bottom-up composition of part-level functions provides a structured way to construct object-level interactions. The integration of geometric and interaction feedback also enables targeted refinement of incorrect programs.
Critical weaknesses
  1. In Table 1, AffordCode receives ground-truth programmatic object representations containing geometry and structural relationships, while the baselines receive different forms of ground-truth information. This makes it difficult to isolate the benefits of affordance-program learning from those of the richer object representation.
  2. Although Table 4 demonstrates the benefits of verification and bottom-up composition, no direct ablation isolates the effect of memory retrieval and reuse on performance.
Questions for authors
  1. Table 1 compares methods using different forms of ground-truth object information. How would the results change if all methods received the same geometric and structural information?
  2. The RoboTwin evaluation uses a template-conditioned perception pipeline. How much of the performance improvement comes from this additional frontend rather than the proposed affordance programs?
  3. Hierarchical code memory is presented as a key component of AffordCode. Can the authors quantify its contribution through a memory-free ablation or a direct analysis of program reuse?
  4. Program verification relies partly on geometric and kinematic constraints. How well do verification results correlate with actual manipulation success or failure?
  5. The real-world evaluation contains only one physical manipulation demonstration. Can the authors provide quantitative success rates across more real-world objects and repeated trials?
2 comments
AuthorsOpenReview ↗

Response to Reviewer UQCR (Part I)

W1&Q1. Comparison fairness and representation benefits

Thank you for raising this point. The methods use different GT interfaces: GAPartNet-GT receives part categories, segmentation, target-part boxes, poses, and meshes; Where2Act-GT receives interaction points, target meshes, and joint kinematics; FlowBot3D-GT receives point clouds, target-part masks, target links, and joint information. Supplying identical inputs would require changing their original interfaces. We therefore added a complementary comparison that fixes the coding agent and varies only the GT input.

We evaluated GPT-5.6 Sol via Codex CLI on the same 100 interaction cases selected from the original 479-case benchmark under all four GT settings. Inputs follow Section 4.1 exactly; all other settings are fixed. The protocol and results are now included in Appendix A.7, Table 11.

GT inputSuccess rate
AffordCode-GT14%
GAPartNet-GT13%
Where2Act-GT21%
FlowBot3D-GT23%

With the coding agent held fixed, AffordCode-GT does not show a consistent task-completion advantage over the other GT settings. Together with the full-system results in Table 1, this supports the contribution of AffordCode's program-acquisition pipeline beyond the structured input alone.

Q2. Contribution of the perception frontend on RoboTwin

To assess how much the perception frontend contributes to the RoboTwin improvement, we compared (1) GPT-5.6 Sol via Codex CLI, (2) the same agent with AffordCode's template-conditioned perception outputs, and (3) full AffordCode. All three use the same 20 evaluation episodes per task across the 42 RoboTwin tasks in Table 2. The two Codex CLI configurations differ only in access to the frontend outputs; their executor, tools, and downstream model-call budget are identical. Configuration (2) receives the same perception outputs as full AffordCode.

MethodMean success rateTasks with nonzero success
GPT-5.6 Sol (Codex CLI)16.67%20/42
GPT-5.6 Sol (Codex CLI) + AffordCode frontend20.24%25/42
Full AffordCode36.67%35/42

The frontend improves the coding-agent baseline by 3.57 percentage points, while full AffordCode achieves a further 16.43-point gain with the same perception inputs. The frontend therefore contributes useful information, but the larger gain under shared perception supports the affordance programs as the main source of AffordCode's advantage in this comparison. We added this comparison in Appendix A.8.1, Table 13, with per-task results in Appendix A.12, Table 15.

W2&Q3. Contribution of the affordance code memory

We added AffordCode w/o Few-Shot to Section 4.4, Table 4. This ablation removes the initial part-level primitives and object-level affordance programs from the acquisition context, while retaining Verified's object templates, synthesis and verification procedure, and frozen downstream policy. Both configurations acquire their own programs and freeze them before evaluation.

TaskMethodSeen InstanceUnseen InstanceUnseen Category
PullDrawerAffordCode w/o Few-Shot90%57%64%
PullDrawerAffordCode-Verified92%60%72%
OpenDoorAffordCode w/o Few-Shot77%52%24%
OpenDoorAffordCode-Verified80%55%33%

With synthesis and verification held fixed, prior code improves success by 4.7 percentage points on average. The gains are 2-3 points on the instance splits and 8 and 9 points on Unseen Category for PullDrawer and OpenDoor, respectively. This quantifies the contribution of the stored affordance code to program acquisition; the analysis is also included in Section 4.4.

AuthorsOpenReview ↗

Response to Reviewer UQCR (Part II)

Q4. How well does verification predict manipulation success?

We added this evaluation in Appendix A.4.3, Table 7, using the physics-simulation pipeline and success criterion of Section 4.1: the target part must reach at least 90% of the required motion range within 1,000 simulation steps. We ran ten independent acquisitions for each of three tasks: opening a door, pulling a drawer, and turning a knob. Each run used two randomly parameterized validation instances and at most two verification rounds.

A candidate passes verification only if it satisfies the required checks on both validation instances. We froze the first passing candidate or, if none passed within the budget, the last candidate evaluated. We recorded the verification decision before execution and tested each frozen program on three separate evaluation instances, matched between the passing and non-passing groups. A run counts as successful only if all three executions succeed. The table therefore reports 30 acquisition runs, comprising 90 executions.

Verification outcomeSuccessful runsFailed runsRun success rate
Passed11761.1%
Did not pass0120.0%
Total111936.7%

The passing and non-passing groups differ by 61.1 percentage points, with a pooled binary correlation of φ = 0.62 across the 30 runs. These results show a positive association between passing verification and succeeding in independent downstream execution.

Q5. Quantitative real-world evaluation

We extended our real-world evaluation to five tasks with ten trials per task, using Flexiv Rizon 4 and RealMan RM65 robot arms. The initial configuration was varied between trials. To assess physical execution with known object geometry, we supplied the corresponding parameterized template and GT geometric parameters for each physical object.

Real-world taskSuccessful trials
(a) Place scissors in a drawer8/10 (80%)
(b) Open a suitcase10/10 (100%)
(c) Place scissors in a drawer and close it7/10 (70%)
(d) Place a block in a bowl10/10 (100%)
(e) Pull a drawer open10/10 (100%)
Overall45/50 (90%)

Across the 50 trials, these results show that, given known object geometry, AffordCode's programs can guide repeated physical manipulation across the tested objects and task types. We added the quantitative results and representative initial, intermediate, and final execution frames in Appendix A.10, Table 14 and Figure 6.

We use the drawer-opening example in Appendix A.10.1, Figure 7 to illustrate how AffordCode guides physical execution. The figure visualizes offline program generation from the template and task instruction, followed by online trajectory instantiation, pose estimation, and coordinate transformation for execution on the robot.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions

We thank the reviewers for their constructive feedback. We have revised the manuscript to incorporate additional experiments and clarify the implementation and evaluation protocols. The main updates are summarized below, with references to the revised PDF.

  1. Additional evaluation metric (Section 4.1, Table 1). We added normalized distance on the same 479 interaction cases to capture partial progress alongside success rate. AffordCode achieves an overall normalized distance of 0.5288, compared with 0.6953 for the best baseline on this metric.

  2. Additional policy-guidance analysis (Section 4.4, Table 4 and Figure 4). We added an ablation without the initial few-shot code examples; AffordCode-Verified improves success by 4.7 percentage points on average across the six task–split combinations. We also added an OpenDoor comparison illustrating the geometry of the affordance guidance and the resulting execution under the same frozen policy.

  3. Verification details and execution analysis (Appendices A.4.1 and A.4.3). Algorithm 1 specifies the verification checks, predicates, tolerances, and kinematic rollout implementation. We also added a 30-run analysis of verification and held-out execution success (Table 7): success rates are 61.1% for runs passing verification and 0.0% for those not passing.

  4. Initial code and program acquisition (Appendices A.5, A.6.1, and A.6.2). We now list the three initial few-shot code examples and their object categories (Table 8). We also added a comparison with GPT-5.6 Sol via Codex CLI, matching the initial code, validation instances, tools, and acquisition budgets. AffordCode-Verified achieves an average gain of 8.2 percentage points across the six task–split combinations (Table 9). We also report five independent acquisition runs per task on unseen categories (Table 10).

  5. GT input comparison (Appendix A.7, Table 11). We added a complementary comparison on 100 interaction cases, holding the coding agent fixed while varying the four ground-truth input representations.

  6. Perception frontend ablation (Appendix A.8.1, Table 13). Across all 42 RoboTwin tasks, the Codex CLI baseline, the baseline with AffordCode's frontend outputs, and full AffordCode achieve mean success rates of 16.67%, 20.24%, and 36.67%, respectively. Their per-task results are included in the expanded table in Appendix A.12 (Table 15).

  7. Expanded physical evaluation (Section 4.5 and Appendix A.10). We expanded the physical experiments to five tasks with ten trials per task, achieving 90% overall success using ground-truth instance parameters (Table 14). Figures 6–7 show representative executions and the drawer-opening program's construction and execution process.

We welcome further questions and discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.