- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AffordCode is an agentic framework that acquires reusable affordance programs over parameterized object representations. Using ConceptFactory/Digital-Gene interfaces, a language-model agent retrieves or synthesizes part-level routines and composes them into object-level interaction logic. Executing a program with new object parameters produces regions, contact poses, motion quantities, or trajectories. Structural, geometric, and interaction checks provide feedback for revising implicated code, and validated routines are retained in a hierarchical memory. Under the default protocol, each template-task program is acquired using eight sampled instances and at most three refinement rounds; programs and memory are frozen before downstream evaluation.
The contribution is the organization, refinement, and reuse of executable interaction knowledge. Evaluation covers 479 simulated interactions with representation-specific ground-truth inputs, 42 RGB-D RoboTwin tasks, demonstration generation for four policy-learning tasks, and affordance guidance for a frozen policy on two articulated-manipulation tasks. The paper reports improvements across these settings, supplemented by ten qualitative real-world object-task examples and a real-robot scissors-to-cabinet demonstration.
Critical strengths
-
The representation provides a concrete mechanism for inspecting and reusing interaction rules. The division between local part computations and object-level composition makes the dependencies among contact geometry, coordinate frames, motion axes, and task outputs explicit. Listing 2 demonstrates this through handle contact computation, hinge geometry, and a door-opening trajectory. Re-evaluating these functions on new parameters explains how a stored program adapts to compatible instances. Appendix A.2 also specifies the compatibility requirements, while Appendix A.5 distinguishes direct instantiation from further acquisition for new tasks or compositions. This is a useful design for editable and transferable affordance knowledge, and a meaningful application of program synthesis to structured object interaction.
-
The experimental results provide substantial evidence of downstream utility, including an informative controlled ablation. AffordCode reaches 42.80% interaction success versus 26.93% for GAPartNet-GT in Table 1, and 36.67% average RoboTwin success versus 19.76% for ReKep in Table 2, with nonzero success on 35 of 42 tasks. In the matched 20-mesh × 50-demonstration setting, policies trained on its demonstrations achieve 65.2% average success on unseen meshes versus 51.0% for AffordGen. Particularly useful is Table 4: all affordance-guided variants share a frozen downstream policy, splits, and execution settings. The full method improves over the unverified variant by 6-10 percentage points and over direct object-level synthesis by 5-7 points across the six task-split combinations. These controls support the usefulness of verification and part-based composition, and the explicit separation of acquisition from evaluation strengthens the evidence.
Critical weaknesses
-
The implementation and independence of the verification signals are insufficiently specified for a central contribution. Equations (4)-(5) define a rollout and task predicates, but Appendix A.4.1 allows geometric propagation, kinematic propagation, or an execution backend without identifying which is used in each experiment. I could not determine the concrete implementations, tolerances, and information sources for the relevant task predicates, contact/collision checks, and motion constraints. This matters particularly for hypotheses such as the hinge side and axis in Listing 2, which the authors explicitly describe as assumptions to be checked. If a rollout and its success predicate both inherit an incorrect candidate hinge hypothesis, passing them could establish internal consistency without independently validating that hypothesis. The manuscript does not provide enough implementation detail to assess how this possibility is excluded. The authors appropriately acknowledge that geometric or kinematic verification does not guarantee contact-dynamic feasibility; my concern is the evidence supporting correction of the underlying interaction assumptions. An executable example of the checker and an actual failure/revision trace would materially strengthen this central claim.
-
The amount of newly acquired knowledge and the scope of program reuse are not sufficiently disentangled from the supplied priors. Appendix A.5 initializes part memory with template routines and previously generated programs, but does not provide a benchmark-specific inventory of their provenance or existing affordance content. The geometry-only example in Listing 1 is helpful, yet does not establish the contents of the actual experimental memory. Likewise, Table 4 defines unseen objects/categories in the downstream-policy setting without establishing whether those categories or part compositions were excluded from AffordCode's initial memory and acquisition. The stated validation/evaluation instance separation and frozen memory are valuable, but do not resolve that additional distinction. Consequently, it is difficult to quantify how much success comes from newly learned rules, direct reuse within a compatible template, or acquisition using familiar part routines.
Questions for authors
-
Clarification of the implemented verifier. For the settings in Tables 1-4, which validation backend and concrete task predicates are used during acquisition, and where do their geometry, kinematics, and success conditions come from? Which checks are deterministic and which depend on VLM judgment? Please provide a representative executable checker, including its tolerances, and one actual trace showing how an incorrect hinge-axis or motion-direction hypothesis is rejected using evidence independent of that hypothesis, revised, and re-validated. This would address Weakness 1 without requiring a contact-dynamics simulator for every output type.
-
Clarification of knowledge provenance, splits, and coverage. Please report the initial part/object memory contents and provenance, the number of acquired template-task programs, and how acquisition instances and parameter ranges relate to the evaluated objects. In particular, are Table 4's unseen categories also unseen to AffordCode's initial memory and program acquisition, or only to the frozen policy? Please distinguish reuse by direct instantiation from acquisition of programs for new compositions, and explain how unsuccessful acquisitions or incompatible instances enter the reported success-rate denominators. These details would determine the extent of the supported learning and reuse claims.
-
Control clarification and targeted additional evidence. Do AffordCode-Direct and AffordCode-Verified receive the same initial routines, validation instances, tool access, and model-call/refinement budgets? Beyond this existing ablation, a quantitative comparison on a representative subset of the current tasks against a general coding agent or direct use of existing affordance routines, with these factors matched, could resolve Weakness 2. Please report any available variation across independently acquired programs or paired episode outcomes, and state explicitly if acquisition was run only once. The control details are clarification requests; the matched comparison is a targeted request for additional evidence about the present contribution.
3 comments
Response to Reviewer L7Fv (Part II)
W2&Q3. Acquisition controls and comparison with a general coding agent
In Table 4, AffordCode-Direct and AffordCode-Verified use the same initial routines, validation instances, tool access, and limits on model calls and refinement rounds. Their verification procedure and downstream evaluation setup are also identical. Direct synthesizes the object-level program directly, while Verified composes part-level routines.
We added a matched comparison with GPT-5.6 Sol via Codex CLI, providing the same initial code memory, validation instances, tools, feedback, and acquisition budget as Verified. The coding agent organizes synthesis and revision itself. For this comparison, acquisition was run once for each object template and task, including verification and refinement within that run. The programs were then frozen and evaluated using the same downstream policy on the same 100 episodes for each task and split. The controls and results are included in Appendix A.6.1, Table 9.
| Task | Method | Seen Instance | Unseen Instance | Unseen Category |
|---|---|---|---|---|
| PullDrawer | GPT-5.6 Sol (Codex CLI) | 88% | 52% | 60% |
| PullDrawer | AffordCode-Verified | 92% | 60% | 72% |
| OpenDoor | GPT-5.6 Sol (Codex CLI) | 71% | 49% | 23% |
| OpenDoor | AffordCode-Verified | 80% | 55% | 33% |
Verified improves success in all six settings, by 8.2 percentage points on average, with gains of 12 and 10 points on Unseen Category for PullDrawer and OpenDoor. With prior code and validation resources matched, these results support the contribution of AffordCode's structured acquisition procedure.
Variation across independently acquired programs. We also report five independent acquisition runs of AffordCode-Verified for each task under Unseen Category. Each run starts from the same initial code memory and follows the same acquisition protocol, including its own verification and refinement. The resulting programs are frozen and evaluated with the same downstream policy on the same 100 episodes. Appendix A.6.2, Table 10 reports the individual success rates and their mean ± sample standard deviation.
| Task | Success rates across five acquisitions (%) | Mean SR ± SD (%) |
|---|---|---|
| PullDrawer | 72, 73, 69, 73, 76 | 72.6 ± 2.5 |
| OpenDoor | 33, 35, 31, 33, 36 | 33.6 ± 1.9 |
These repeated acquisitions reproduce the original performance levels, with standard deviations of 2.5 and 1.9 percentage points for PullDrawer and OpenDoor, respectively.
Thanks for the additional experiments to clarify my concerns. I would like to maintain my score.
Response to Reviewer L7Fv (Part I)
Thank you for the detailed questions. We added Algorithm 1 in Appendix A.4.1, specifying the verification stages, implemented predicates, numerical tolerances, and acceptance rule.
For the verified configurations in Tables 1-4, kinematic propagation is the acquisition backend for outputs requiring a rollout in Eq. (4). The instantiated validation template supplies the initial meshes and part poses; candidate tool poses drive propagation, and the fixed task specification defines the intended state change.
The VLM participates only in geometric feasibility and interaction consistency. Structural validity checks program execution and output contracts. Geometric feasibility combines applicable numerical predicates with VLM review of rendered outputs against the object geometry and task's spatial requirements. For trajectories, interaction consistency checks motion properties such as fixed-axis consistency, while the VLM reviews hinge placement, maintained attachment, and requested motion direction. All applicable required checks must pass; a visual pass does not override a numerical failure.
The reviewer is right that a trajectory can satisfy a fixed-axis check while rotating about the wrong edge. Fitting an axis from candidate poses establishes internal consistency, not whether that axis matches the object's connection. One could use the handle position to infer the free edge and place the hinge on the opposite side, but this cue depends on the handle layout and joint design. Encoding such cues as hard predicates can require experts to specify mechanism-specific rules and maintain their exceptions. We therefore use VLM review to consider the initial door geometry, cabinet connection, and handle layout together with the original task instruction. These references remain fixed while the candidate axis and motion change, providing evidence beyond the candidate's internal consistency.
We have also uploaded a supplementary microwave verification case with complete VLM inputs and responses and multi-view candidate renderings, documenting rejection of an incorrect hinge-axis motion and revalidation of the corrected motion.
Initial memory. AffordCode uses the same three initial few-shot examples across benchmarks, including the real-world experiments; the w/o Few-Shot ablation explicitly omits them. Their object categories do not overlap with the evaluated categories. Their contents are now listed in Appendix A.5, Table 8.
Acquisition instances and evaluation objects. Acquisition instances are sampled from each template's valid parameter ranges, separately from evaluation objects. The parameter domain may overlap, but evaluation instances and their outcomes are not used for acquisition or refinement. Programs and memory are frozen before evaluation.
Unseen categories in Table 4. For Unseen Instance, an existing program is instantiated on new objects from a known category. For Unseen Category, target categories are unseen in both policy training and AffordCode's initial memory. Before evaluation, AffordCode acquires the corresponding programs on separate acquisition instances, freezes them, and instantiates them on the evaluation objects.
Program reuse. The initial few-shot object-level programs cannot be directly instantiated to solve the evaluated tasks. Direct reuse applies an acquired, frozen program unchanged to compatible evaluation objects, whose parameters determine the outputs. For new part compositions or tasks, the agent acquires and verifies a new object-level program, reusing compatible part routines and related object-level programs as context. Appendix A.5 clarifies these two forms of reuse.
How acquisition failures and incompatible instances are counted. Except for interruptions caused by external factors such as network outages, every scheduled evaluation case remains in the denominator. An incompatible instance counts as a failure. If acquisition fails, every evaluation case requiring that program also counts as a failure; none is removed from the evaluation set.