acceptodds
Under review as a conference paper at ICLR 2027

Object Count Generalization Is an Interface Problem: Warping a Frozen Skill Instead of Retraining It

13 views

Abstract

Reinforcement learning policies for robot manipulation degrade when the number of objects in the scene grows beyond the training range. This is a covariate shift that also changes the dimension of the state. We propose WAVE (Warping Actions and Views of an Expert), which extends a pretrained single-object skill to many objects without updating its weights. WAVE has three parts. First, a selection policy picks the one object the robot works on next. Second, a warping policy reads a local view of the scene around that object, whose size does not depend on . It emits a multiplier and an offset for every quantity the skill reads and every action it outputs. Third, the skill acts under this affine warp. It is shown a scene that holds a single object, while the warp steers it around the objects it is not shown and edits the actions that are sent to the robot. Both the selection and warping policies are trained with PPO on the multi-object reward and use the skill as a black box, so WAVE is task-general and skill-agnostic. It requires only that the skill was trained on the same embodiment and that every object has a goal pose. We also analyze the probability that the composition places all objects within a budget of skill invocations that grows linearly in . We evaluate WAVE on cube arrangement tasks with a gantry robot and with a Franka Panda arm. Trained with at most nine cubes, it generalizes zero-shot to several times its training count, on the gantry to scenes of up to thirty cubes. Published methods that learn multi-object manipulation show count generalization up to six objects. Fine-tuning the skill, learning from the same local view without it, and an entity transformer, even when distilled from the pretrained skill, all collapse within their training counts.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, robot manipulation, object count generalization, zero-shot generalization, physical AI, residual policy learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_X8mDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper tackles an issue of cardinality - how well can a manipulation task generalize to arbitrary numbers of objects, even when it hasn't seen such a large set in training. The approach taken is to not try and retrain a basic skill that works well for a small number of objects, but simply to crop/warp the input/output such that it is only aware of a smaller extent. In this way, the original skill does not become distracted by this domain shift. The paper shows extensive experiments on simulated (gantry and panda) and real (panda) block manipulation. This shows that basic skills can be extended greatly in cardinality e.g. up to 30 blocks for gantry.

Critical strengths
  • It tackles a clear and well motivated problem: manipulation of large numbers of objects becomes challenging. The solution is also clean: rather than training a specific multi-object network, it effectively just tricks the original policy into only considering a limited view of the world. By warping its view of and actions in the world, it can now scale more gracefully to larger numbers of objects.
  • Real world experiments are always useful! And hard to do, so well done on this.
Critical weaknesses
  • Generalization: Whilst I admire the approach of simply cropping out bits of the world to warp the policy (and I think it is a good "divide and conquer" approach in general), I do have concerns about generalization: generalization to different sized blocks even, generalization to different objects, generalization to heterogenous collections of objects, generalization to more complex/cluttered scenes. I am not fully convinced that just warping the interface is sufficient for robust control policies that generalize well.
  • Complex planning: Related to the above point is that of more complex planning and manipulation. The warp policy effectively has to take care of collision avoidance for the naive policy which now doesn't know about stacks. And it also has to learn when to take over. This seems quite challenging to get right in all situations, as you are blending the original skill with the warped policy.
  • Dynamics: Cropping is good to attend to only a limited view of the world, but it then cannot handle moving objects/changes that are outside its warped view.
  • Foundational models: I would be very interested to see how your approach fares against something like GPT Astra 6 or Fable. These models can do a relatively good job about reasoning about a scene to then decompose the planning task - I suspect the warping is doing something similar - it is balancing a global (warp) versus a local (crop) view of the world.
Questions for authors

I would be interested in more complex scenarios and failures - the starting position always seems like a nice collection of spread out blocks. How about a transformation challenge - turn a pyramid into a square? How well could it do this? Or simpler: can it take apart a pyramid without knocking it over?

3 comments

Thank you for the review and for the experiments you suggested. We ran them and report them in a new Appendix J of the revised paper, together with videos. All of these experiments use the gantry and the frozen single-cube skill of the paper without changes, and only the warping policy and the selection policy are trained, with 1 to 10 objects.

W1 other sizes, other objects, heterogeneous collections, clutter The skill, the warping and the selection policy all read the four corner points of the currently selected cube, so they are able to distinguish different object shapes. In the table, N = 20 is twice the largest training count. All generalization experiments train on line and square tasks.

ObjectsN = 10N = 20
Boxes 16 × 24 × 16 cm1.00 / 0.981.00 / 0.98
Cubes and such boxes in one scene1.00 / 0.981.00 / 0.97
Cubes of three sizes in one scene0.98 / 0.980.98 / 0.95
Half of the cubes have to stay at their start pose1.00 / 0.980.98 / 0.97
Cubes with 1.2 times the edge (five tasks)0.91 / 0.940.77 / 0.77
Cubes with 0.8 times the edge (five tasks)0.48 / 0.580.16 / 0.16

Boxes, mixed sizes and scenes with cubes that have to stay in place reach 0.95 or more at N = 20. Larger cubes reach 0.77. Smaller cubes fail beyond five cubes. This decrease begins within the training counts, so it is not a failure to generalize over the count. We attribute it to the unchanged gripper, which has to work between goals that lie closer together, but we did not isolate the cause. Objects of another class, for example cylinders, would need their own single-object skill. We also trained one warping policy and one selection policy on five tasks at once without a task identifier. This pair reaches at least 0.90 on every task at N = 10.

W3 changes outside the crop The crop restricts only the warping policy. The selection policy reads every object that is not at its goal, and the crop and the warp are recomputed at every control step. Section 3 now states this. As a test, a disturbance resets one resting cube to a free position on the floor every 10 s, using the policies of the paper without retraining. All episodes in this test have twice the usual time limit, because each disturbance adds work. At N = 20, success decreases by at most 0.10 (from 0.98 to 0.88 on the line), so our method adapts to this kind of disturbance zero shot. The method does not anticipate objects that move on their own, and we added this to the limitations.

Q: turning a pyramid into a square, and taking a pyramid apart without knocking it over. We trained both tasks. Starting from a built pyramid, the square is completed in 0.84 / 0.89 of the episodes with six cubes. Starting from stacks of height three, the line is completed in 0.88 / 0.92 with ten cubes. In these runs the policies first knock the structure down. We therefore trained a second time with an added condition under which an episode ends as a failure when a cube falls before the gripper has held it. Under this condition, a pyramid is taken apart and rebuilt as a square in 0.86 / 0.98 of the episodes with three cubes, in 0.34 / 0.50 with six, and in 0.02 / 0.17 with ten. These experiments are in Appendix J as well.

W2 collision avoidance and take-over We agree that the take-over is learned and has no guarantee from the frozen skill, and the revision now states this. Appendix G examines what is learned by switching off each side of the warp at every step. The output side is needed mainly for the release, and the input side is needed in every phase. The input side also has to change during the episode. A new baseline in which the input edit is fixed per episode learns its training counts but extrapolates far less than WAVE (Section 4.3). The experiment above on taking a structure apart shows a case in which the learned take-over is not sufficient.

W4 foundation models A language or vision-language model would likely only replace the selection policy, since it would decide which object to move next. The baseline "skill alone" in the paper corresponds to this setting with a scripted planner. A rule selects a cube whose goal can currently be fulfilled, and the unchanged skill executes the step. This baseline completes no episode with three or more cubes. When the same scripted rule is combined with the warp, it reaches 0.85 on cube stacking, compared to 0.82 with the learned selection policy, which is better on the pyramid (0.47 against 0.19). These results suggest that the execution next to other objects limits performance more than the order of the steps. A foundation model could therefore serve as the planner together with WAVE. We did not run an experiment with a foundation model.

Changes in the revision New Appendix J, a summary of its results in Section 4.1, revised limitations, a clarification about the crop in Section 3, and a paragraph on foundation-model planners in the related work.

Reviewer_X8mDOpenReview ↗

Good feedback

Thank you for your clear and honest feedback - I particularly think limitations like poor generalization to smaller blocks are significant and important to report as limitations in your paper, as it shows that whilst warping might help with cardinality it certainly does not help with scale variations. This is in itself an interesting finding as humans can adapt quite rapidly e.g. to using a microscope or loupe to have finer feedback motor control. Overall, I think this remains an interesting wrapper to add extra capability to an existing network, but I do wonder whether it is just a data problem i.e. will next month's foundation models solve this simply by throwing more and more data at the problem without needing your adaptor?

Thanks for the response. Regarding the reduced performance on the smaller cubes we would like to add that we did not change any other parameter in our simulation, training scheme or on the robot, meaning the robot itself is compared to the cube size much larger and needs to act with a lot more precision than is required in the other tests. The smaller cube task thus represents a significantly harder task than the other generalization experiments we ran. So maybe some smaller tunings like longer training, smaller LRs, better LR scheduling or simply also changing the robot dynamics could provide a different result. In the human analogy, many people have no problem stacking together lego blocks but struggle eg when it comes to gluing together minature skulptures even though they might require similar skills just on a different scale.

Regarding the question if next months foundational model could solve this challenge as well, we would concede that maybe in the limited setting that we are testing in our paper there is a chance that a huge monolith could handle these inputs as well, but in the long run, adding more and more objects to the scene (think of 100s or 1000s), the input will always drift ood for these models unless they have a special read on the scene that cannot drift. Our crop design would be one way to input a scene state into a model in a way that cannot drift regardless of the number of objects put into it.

Reviewer_DuahOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose WAVE (Warping Actions and Views of an Expert) to allow for robot policies that were trained on one object to handle many objects without retraining, and it is task general and skill agnostic. The method has three parts: selection policy, warping policy, and running the skill under the warp (which overall applies an affine edit to each input and output channel of the skill, then is trained with PPO). The authors evaluate the method on cube arrangement tasks on both a gantry robot and a Franka Panda arm. WAVE is able to generalize zero shot to up to 30 cubes when trained on at most 9 cubes, which outperforms SOTA methods that particularly look at multi-object manipulation (such as finetuning the skill, learning from local view, entity transformer).

Critical strengths

The problem statement and the motivation presented by the authors in this paper is clear. 3.1 (Intuition) section explains that warping skill is necessary such that if things are in the way such that the skill can avoid it, and warping the output is needed to let the system produce behavior that the skill never originally learned (ie release cube on stack). This is a nice explanation to understand why both input and output warp is required, and also expands on why this should generalize well (because of a good enough prior skill to be able to place an object, and the difference of input of a 30 cube vs a 9 cube scenario shouldn’t be too different). This method is also requires only the same embodiment and a goal pose per object, avoiding retraining the skill itself, and really only training for a warping and selection policy. Evaluations/experiments are thorough, compared in the same setting against 9 different common/SOTA baseline including entity transformers, finetuning the skill, subgoals, etc. Ablations are also performed compared with a random choice/nearest-first choice and also letting the warp only edit either of only the input and only the output.

Critical weaknesses

The authors note that failures for high counts come from knockdowns of placed cubes. I would be interested in a more in depth failure analysis: missed placements vs topples vs other issues (such as missed grasps). It would also be interesting if additional experiments could be done with a few other objects.

Questions for authors

The authors note that failures for high counts come from knockdowns of placed cubes. I would be interested in a more in depth failure analysis: missed placements vs topples vs other issues (such as missed grasps). It would also be interesting if additional experiments could be done with a few other objects.

2 comments

We thank the reviewer for the positive assessment and the helpful suggestions.

Following the reviewer’s suggestion, we re-evaluated the paper’s checkpoints and broke down the failures (three seeds, 20 s time limit per cube):

TaskNSuccessTime limitCube left the workspace
Cube stacking (h = 3)50.980.010.01
Cube stacking (h = 3)100.940.050.01
Cube stacking (h = 3)200.810.140.06
Cube stacking (h = 3)300.450.490.06
Pyramid50.990.010.00
Pyramid100.910.090.00
Pyramid200.430.530.04
Pyramid300.060.900.04

A knockdown counts any placed cube that becomes unplaced, including small nudges, so it does not separate a full topple from a nudge, and one episode can have many. Most failures come from reaching the time limit, not from losing cubes: the policy usually recovers from knockdowns by re-placing the cubes, but this costs time. With 60 s per cube, success at N = 30 rises to 0.67 for stacking and 0.27 for pyramid. We add this table to the appendix and replace the general knockdown statement in the paper with this more precise analysis.

Other objects We added generalization experiments in a new appendix section J that covers experiments with different sizes of cubes and bar shaped objects, as well as trainings where the object dimensions are mixed in every episode. We report these results in the revised version of the paper. Thanks for the suggestion!

Thanks for answering my questions.

Reviewer_sWjiOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a new model for goal-conditioned reinforcement learning in robotics. The essential idea is to learn a single-object policy which depends only on the “local” properties of the object and goal (i.e. its cropped view and its goal). To generalize to multi-object control, a “warp” of this single-object policy is learned, conditioned on the scene, which affinely edits the per-object policy’s inputs and outputs. The result is a policy which generalizes better to larger object counts.

Critical strengths
  • The paper is extremely well-written and clear. I tend to believe that a broad audience, even outside robotics and RL, would be interested in the methods and results.
  • The method is powerful. It outperforms several recent baselines, all of which seem fairly tuned. It works in a real-world environment, where there might be error from e.g. object pose estimates used as input.
Critical weaknesses
  • I have one question about the relationship with FIOC-WM (NeurIPS 2025; [1]). They show that a two-step training procedure (which also leverages object-centricity in its state and goal representations) can achieve fairly impressive count generalization performance, albeit up to only 8 objects. Their tasks seem substantially different, which means that it may not be suitable as a direct baseline.

None of the following weaknesses are significant; I include them for feedback:

  • The heritage of the architecture choice for the “warp” network is a bit unclear to me. Section 3.5 seems to suggest that it is motivated by a kind of “recurrence relation” involving the true underlying dynamics (the “absorption” equality (3)). While interesting, it is not clear to me that we should expect “affine” to be the correct function class in both state and action space to achieve absorption, except maybe for the tasks considered here where the states and actions have a kind of geometric specification. Clearly the architecture works empirically, but some discussion might be warranted (ablations only include more restrictive function classes, which don’t perform much worse). For example, Hyper-GoalNet [2] leverages hypernetworks to modify policy weights per robotics task, using a more general function class, but demands behavior cloning training data.
  • The theory is very much put to the side, in the appendix. It is not clear to me how I should think about Prop 1 & 2 (which seem to relate to my above question about the suitability of affine-ness); they are formatted somewhat unconventionally and are a bit opaque. Theorem 1 is intuitive and discussed in the main text sufficiently imo, although if there is room a more fulsome informal theorem statement might be warranted.
  • In 3.2: Define every element of the tuple M1M_1 in the text.
  • In 3.4: If there’s room, I personally would have found it helpful to have the PPO learning objective in the main text.
  • The appendices are detailed and seem to contain a lot of useful information that clearly would not fit in the main paper. Leading off the appendices with an index would be very helpful (a 1 or 2 sentence description of each appendix section, perhaps).

[1] Learning Interactive World Model for Object-Centric Reinforcement Learning. Feng et al. (NeurIPS 2025)

[2] Hyper-GoalNet: Hyper-GoalNet: Goal-Conditioned Manipulation Policy Learning with HyperNetworks. Zhou et al. (NeurIPS 2025)

Questions for authors
  1. Regarding FIOC-WM: is their setting substantially different from yours? Their goals seem different, for example; but it is intriguing that they explicitly leverage the same object-centric structure.
  2. Is there something special about an affine warp? If, e.g. the states and actions were learned, would something more flexible be warranted? Is there a connection to the prior use of hypernetworks for robotics such as e.g. [1]?
  3. What is the real theoretical content of Props 1 and 2? Can you format them more conventionally (and rephrase the propositions to be self-contained as how Theorem 1 is stated)? Should they be referenced in the main text, space permitting?
2 comments

We thank the reviewer for the positive and careful review. We answer your questions below:

FIOC-WM The work is related, and we will add it to Related Work. However, the settings differ. It learns a world model from pixels and addresses the generalization over object attributes and skill compositions. The count generalization test (Appendix D.3), training on 3 and testing on 6 and 8 in GYM Fetch environment, doesn’t address the task type of difficulty, and the Fetch tasks it describes push cubes or toggle switches, with no stacking or any other task that needs high level of precision. Our results show that task difficulty plays an important role in object count generalization: At 30 cubes, WAVE achieves 0.90 full success on the line arrangement task but no success on tower stacking of height 6 (Table 2). We therefore see FIOC-WM as related work and not as a direct baseline.

Affine warps Although affine transformations are not the only possible class, we adopt them because of their simplicity and suitability for our setting. The frozen policy is unaware of the other objects, so observation and action warping let the warping policy steer it around nearby objects and previously built structures, e.g., by slowing down or redirecting the motion. The warp must edit individual channels and support an identity mode (m=1, o=0) and a full takeover mode (m=0, o = desired value). Channel-wise affine maps are the minimal form that meets these requirements. Because the warping policy runs at high frequency, per-step corrections are sufficient. Per-channel warping works because channels are physical quantities. With learned latent states, cross-channel or nonlinear maps might be needed, and Sec. 3.5 names a full matrix as future work.

Hyper-GoalNet The work addresses editing the full weight space of the policy, and it needs demonstrations. WAVE edits only the interface, which is a much smaller parameter space to learn from the task reward alone. We will add this discussion to Sec. 3.5.

Propositions 1 & 2 We rewrote this part (Appendix H). The two propositions are now a single Proposition 1, stated self-contained with numbered assumptions and a proof, as Theorem 1 is, and Sec. 3.5 refers to it. It states for which warps an attempt in the multi-object scene is exactly an episode of the single-object policy (Eq. 4, Eq. 3 in the submission), so that the attempt inherits the success rate of that policy at any object count, as long as no other object touches the selected object or the gripper:

  • retargeting the goal, translating the scene and per-channel recalibration satisfy this exactly
  • a small constant bias gives only an error bound, under a Lipschitz assumption
  • takeover (m=0) adds behaviour but inherits no guarantee
  • rotations need channel mixing and are not covered The proposition describes the class of edits and motivates the per-channel affine form. It makes no statement about what the learned warping policy does, which we measure in the ablations (Table 9).

Presentation We now define every element of the MDP tuple in Sec. 3.2, add the PPO objective to Sec. 3.4, and open the appendix with a short index.

I appreciate the authors adding the references to the additional related work and clarifying the theory appendix. All of my concerns are addressed and I maintain my recommendation for acceptance.

Reviewer_UajMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how robotic manipulation policies can generalize to a number of objects beyond the range encountered during training. WAVE freezes a goal-conditioned single-object manipulation skill, learns an object-selection policy, and progressively applies channel-wise affine transformations to the skill's inputs and action outputs based on a fixed-width local scene representation. The key idea is to preserve existing manipulation capabilities while learning only the adaptations required for multi-object interactions. The paper evaluates generalization to larger numbers of objects on block-arrangement tasks using a gantry robot and a Franka Panda robot. It also presents component ablation studies, real-world demonstrations, and a conditional bound on the probability of task completion based on the net progress achieved in each attempt.

Critical strengths
  1. The paper demonstrates valuable object-count extrapolation under clearly specified experimental conditions. Under the paper's success criterion, the gantry robot achieves a success rate of 0.95 with 20 blocks on the stacking task, despite being trained on at most 7 blocks. Similarly, on the line-arrangement task, it achieves a success rate of 0.90 with 30 blocks after training on at most 6 blocks (Table 2). These results demonstrate that combining a single-object skill with local interface adaptation can scale to larger numbers of objects in the evaluated environments. This finding has research value even without relying on cross-paper comparisons.
  2. Ablations within the same framework provide more direct evidence of the underlying mechanism than cross-algorithm comparisons. With the rest of the framework and the training budget held constant, the average success rates across object counts on the stacking task are 0.82 for the full method, 0.81 for input-only transformation, and 0.22 for output-only transformation. On the pyramid task, the corresponding rates are 0.47, 0.07, and 0.15 (Table 8). These results support the importance of input-side adaptation and the complementary role of two-sided adaptation in certain tasks. In the four-block stacking setting examined through interventions in Appendix G, the jointly trained policy relies on the output-side transformation primarily during the release phase. However, this finding should not be generalized to imply that two-sided transformations are necessary for all tasks.
  3. The combination of object selection and skill reuse has practical significance and is partially supported by experimental evidence. The method preserves a frozen skill without requiring access to its gradients or internal activations. The learned selector also operates without the explicit feasible-target filtering used by the baseline. On the pyramid task, the method achieves an average success rate of 0.47, outperforming the nearest-first selection baseline with feasibility filtering, which achieves 0.19 (Table 8). Although this advantage is task-dependent, it indicates that the paper's contribution extends beyond modifying individual actions to learning the sequencing of multi-step manipulation operations.
Critical weaknesses
  1. The semantics of the success metric remain inconsistent, affecting the assessment of complete arrangement capability. Section 4 defines success as all blocks being simultaneously within the target tolerance and supported by their intended supports, whereas Appendix C explicitly states that training may count an episode as successful while the final block is still held by the gripper. Table 5 transparently distinguishes between reaching the target configuration (8/10) and achieving a stable structure after release (5/10) in real-world experiments. The concern is that the main text emphasizes the former, which could be misinterpreted as successful completion of the entire placement task. The authors should clarify whether simulation results in Tables 2–4 use the same termination logic and how support is detected. The current evidence does not establish that the simulation results are incorrect, but neither does it justify interpreting them directly as stable arrangements with all blocks released.
  2. The connection between the conditional nature of the theoretical results and their empirical validation remains insufficiently established. Theorem 1 requires the lower bound on the success probability and the upper bound on the number of disrupted objects to hold conditional on every history. Appendix H explicitly acknowledges that aggregate averages are insufficient. Table 9 reports aggregate means, while δcov\delta_{\mathrm{cov}} in Appendix F.4 only checks whether the nearest-neighbor distance exceeds the minimum value in the training reference set. This does not establish that it upper-bounds the probability of encountering uncovered regions of the full crop distribution. Therefore, Table 9 provides useful diagnostics of progress but does not directly verify that the actual policy satisfies the task-completion probability guarantee. Establishing a linear attempt budget uniformly in N would additionally require a positive lower bound on the drift that does not vanish as N increases. In Table 9, ρ for stacking decreases from 0.036 at N=7 to 0.008 at N=30, while it is already negative for the pyramid task at N=10. Such a uniform lower bound has therefore not been established. This concern addresses the applicability of the guarantee to the learned policy, rather than presenting a counterexample to Theorem 1. Proposition 2 should also distinguish among exact dynamical correspondences that satisfy Equation (3), robustness bounds under additional Lipschitz assumptions, and the control expressivity afforded by action takeover. The latter does not, by itself, inherit the success guarantee of the frozen skill.
  3. The characterization of the closest prior work needs correction, and the novel contribution should be repositioned accordingly. Section 2 and Table 10 characterize the state/action projections of Zhang et al. (2023) as fixed throughout each episode. However, Section IV-C of the original paper specifies that the state projection depends on object features, while the action projection is defined as Ta=ρψ(f,sˉu)T_a=\rho_\psi(f,\bar{s}_u), depends on the current projected state, and is learned using PPO. Thus, the method cannot be characterized as a whole as remaining unchanged across control steps. This error does not negate WAVE's contributions to object-count extrapolation, local-neighborhood conditioning, and integration with an object selector, but it weakens the current argument for mechanistic novelty. The authors should clearly distinguish the mechanisms inherited from prior work from those newly introduced.
  4. The causal explanation for "why object-count generalization works" is stronger than what the current comparisons support. Several baselines already fail at object counts within the training range (Table 4). Consequently, these results alone cannot distinguish failure to learn the training tasks from failure to extrapolate after successful training. Equal training budgets, distillation, and anchored fine-tuning are useful experimental controls, and differences between PPO and TD7/TD3 do not automatically invalidate the comparisons. Table 8 and Appendix G provide mechanistic evidence for interface adaptation, but they do not isolate the contribution of local aggregation relative to global or variable-length representations. Therefore, they are insufficient to attribute EIT's failure specifically to input dimensionality. The cross-task and cross-robot comparisons in Figure 3 are better treated as contextual evidence. If the authors retain these stronger causal claims, they could provide existing training logs to support the diagnosis. Otherwise, narrowing the conclusions to the empirical advantages observed under the current training protocol would be sufficient, without requiring an entirely new set of baseline experiments.
Questions for authors

Q1: Please clarify the success and termination criteria separately for simulation training, simulation evaluation, and real-world experiments: Is releasing the final block required, and how is support by the intended supporting object verified? If simulation also counts an episode as successful while the final block remains grasped, could the authors first use existing logs or trajectories to distinguish success under the two criteria? If necessary, the authors could briefly continue executing the existing stacking policy at N=7 and one extrapolation object count (e.g., N=20), and report the proportion of arrangements that remain stable after release, without retraining. This clarification would determine whether the main results should be interpreted as reaching the target configuration or completing the full arrangement. Q2: Which cases in Proposition 2 satisfy Equation (3), and which provide only conditional error bounds or representational capacity? How do the statistics in Table 9 relate to the per-history conditional bounds and coverage of the full crop distribution? If a linear attempt budget uniform in the number of objects is claimed, how can a positive lower bound on ρ independent of N be established? If these quantities are merely empirical proxies, please explicitly characterize Table 9 as diagnostic evidence and clarify the scope of the theoretical guarantee. Exhaustive verification over all possible histories is not required. Q3: Please revisit the state-dependent action projection proposed by Zhang et al. and clarify which specific mechanisms are newly introduced by WAVE, along with the evidence supporting them, after correcting this comparison. It would be sufficient to explain the extent to which the existing ablations support these contributions; large-scale reproduction of prior work is not necessary.

3 comments

Thank you for the detailed review. We revised the paper on all four weaknesses. Short answers follow; the details are in the revision.

W1, Q1 success criterion In simulation, training and evaluation use indeed the same rule. An episode ends as a success at the first step at which every cube is within tolerance of its goal and on its intended support. The last cube can still be in the gripper, and support is determined from the cube positions (the rule is now given in Appendix A). Tables 1 to 4 report how often the target configuration is reached, and Section 4 now states this. We additionally ran the proposed evaluations on three seeds without retraining, in two settings: 1) only ending the episode after the success criterion is fulfilled for 2 full seconds (with a longer time limit) and 2) adding a scripted release and retract that takes over as soon as all cubes are placed and at rest.

Cube stacking (h=3h = 3)N = 7N = 20N = 30
Original termination on first success signal0.970.810.45
Termination after 2 s of continuous success0.990.920.59
All cubes in place 2 s after release0.930.730.38

So the policies also transfer zero shot to the stricter success criteria with limited loss of success rate. The full table is Table 14 (Appendix J). Thanks for raising this point, we think this result makes the paper stronger.

W2, Q2 scope of the theory We changed the following. (i) The proposition (now Proposition 1) distinguishes three cases. 1) exact correspondences that satisfy Eq. (4), which was Eq. (3) in the submission (retargeting, translation, per-channel recalibration) 2) a constant bias, for which we only give an error bound under a Lipschitz assumption and 3) take-over, which is control expressivity only and inherits no guarantee from the frozen skill. (ii) Table 10 (Table 9 in the submission) is described as a diagnostic of averages and no longer as a verification of the per-history hypotheses. (iii) The coverage statistic is removed from the theorem. (iv) The theorem is stated for a fixed NN and we claim no bound uniform in NN. The abstract and Section 3.6 are changed accordingly. (v) The net progress per attempt decreases from +0.64 at three cubes to +0.11 at thirty on cube stacking. On the pyramid it becomes negative between ten and twenty cubes, where success drops from 0.93 to 0.19. The theory appendix is rewritten with numbered assumptions, which we hope makes it easier to read.

W3 Zhang et al. (2023) We corrected Section 2 and Table 12 (Table 10 in the submission). Their state projection is constant over an episode, and their action projection depends on the current state. The revision lists what WAVE takes from their method (the frozen skill, per-channel edits on both sides, policy-gradient training, a state-dependent action edit) and what it adds (conditioning on the other objects, an input edit that changes within the episode, offsets, a learned selection policy). We also added a baseline that applies their interface to our setting: WAVE trained with the input edit chosen at the first step of an episode and kept until its end, while the output edit is still chosen at every step. Nothing else is changed, so the offsets, the crop and the learned selection policy, which are not part of their method, remain. On cube stacking (two seeds) it learns its training counts but drops to 0.08 at twenty cubes, compared to 0.95 for WAVE. On the line the gap is smaller and the two seeds differ considerably. This additional baseline shows that reactive input warp changes are a necessary requirement for true ood count generalization. See Section 4.3, Table 1 and Figure 7 in Appendix B. Thanks for raising this point!

W4 causal claims We narrowed the conclusions as you suggest. Section 4.3 now states that nine of the ten baselines fail within their training counts, so for these the comparison shows that they do not learn the task with our budget and cannot tell how they would extrapolate. Only the baseline with the fixed input edit learns its training counts, and it extrapolates far less than WAVE. We removed the attribution of the EIT's failure to its input size, and the caption of Figure 3 states that the gray series are shown for context only. Two further changes: 1) When checking the training logs we found an implementation error in plain fine-tuning and JSRL. The corrected runs retain the one-cube skill (0.88 and 0.87 on cube stacking) but still fail from three cubes on. 2) We ablated the aggregation method used in the crop. Replacing the minimum over the other objects by a mean or by a learned softmax-weighted mean (attention) lowers the mean success beyond the training range on cube stacking (0.55 and 0.64 against 0.82) but not on the pyramid (0.47 and 0.66 against 0.47), and we removed the sentence that a mean "would dilute as NN grows". These ablations are in the new Appendix J.

All changes are in the revised paper.

Correction to our response on W3 (Zhang et al. baseline). We wrote that the fixed-input-edit baseline was run with two seeds. That is wrong, it was run with three seeds, like WAVE and the other baselines (Table 1, Table 4, Figure 7).

For reference, on cube stacking it completes 0.60 of the episodes at ten cubes and 0.08 at twenty, compared to 0.98 and 0.95 for WAVE. On the line it completes 0.55 and 0.35, compared to 1.00 and 0.93, and the seeds vary a lot there (0.30 to 0.80 at ten cubes). The gap on the line task between fixed and reactive input warps is smaller than on the stacking task at twenty cubes. At ten cubes it is larger on the line than on stacking (0.45 vs. 0.38).

Dear Reviewer UajM,

Thank you again for your valuable feedback. As the discussion period is nearing its end, we would like to follow up to see if you have any further questions or anything else we can help clarify. We hope our responses have addressed your concerns, and if any remain, we would be glad to discuss them further.

Thank you :)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.