- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper tackles an issue of cardinality - how well can a manipulation task generalize to arbitrary numbers of objects, even when it hasn't seen such a large set in training. The approach taken is to not try and retrain a basic skill that works well for a small number of objects, but simply to crop/warp the input/output such that it is only aware of a smaller extent. In this way, the original skill does not become distracted by this domain shift. The paper shows extensive experiments on simulated (gantry and panda) and real (panda) block manipulation. This shows that basic skills can be extended greatly in cardinality e.g. up to 30 blocks for gantry.
Critical strengths
- It tackles a clear and well motivated problem: manipulation of large numbers of objects becomes challenging. The solution is also clean: rather than training a specific multi-object network, it effectively just tricks the original policy into only considering a limited view of the world. By warping its view of and actions in the world, it can now scale more gracefully to larger numbers of objects.
- Real world experiments are always useful! And hard to do, so well done on this.
Critical weaknesses
- Generalization: Whilst I admire the approach of simply cropping out bits of the world to warp the policy (and I think it is a good "divide and conquer" approach in general), I do have concerns about generalization: generalization to different sized blocks even, generalization to different objects, generalization to heterogenous collections of objects, generalization to more complex/cluttered scenes. I am not fully convinced that just warping the interface is sufficient for robust control policies that generalize well.
- Complex planning: Related to the above point is that of more complex planning and manipulation. The warp policy effectively has to take care of collision avoidance for the naive policy which now doesn't know about stacks. And it also has to learn when to take over. This seems quite challenging to get right in all situations, as you are blending the original skill with the warped policy.
- Dynamics: Cropping is good to attend to only a limited view of the world, but it then cannot handle moving objects/changes that are outside its warped view.
- Foundational models: I would be very interested to see how your approach fares against something like GPT Astra 6 or Fable. These models can do a relatively good job about reasoning about a scene to then decompose the planning task - I suspect the warping is doing something similar - it is balancing a global (warp) versus a local (crop) view of the world.
Questions for authors
I would be interested in more complex scenarios and failures - the starting position always seems like a nice collection of spread out blocks. How about a transformation challenge - turn a pyramid into a square? How well could it do this? Or simpler: can it take apart a pyramid without knocking it over?
3 comments
Good feedback
Thank you for your clear and honest feedback - I particularly think limitations like poor generalization to smaller blocks are significant and important to report as limitations in your paper, as it shows that whilst warping might help with cardinality it certainly does not help with scale variations. This is in itself an interesting finding as humans can adapt quite rapidly e.g. to using a microscope or loupe to have finer feedback motor control. Overall, I think this remains an interesting wrapper to add extra capability to an existing network, but I do wonder whether it is just a data problem i.e. will next month's foundation models solve this simply by throwing more and more data at the problem without needing your adaptor?
Thanks for the response. Regarding the reduced performance on the smaller cubes we would like to add that we did not change any other parameter in our simulation, training scheme or on the robot, meaning the robot itself is compared to the cube size much larger and needs to act with a lot more precision than is required in the other tests. The smaller cube task thus represents a significantly harder task than the other generalization experiments we ran. So maybe some smaller tunings like longer training, smaller LRs, better LR scheduling or simply also changing the robot dynamics could provide a different result. In the human analogy, many people have no problem stacking together lego blocks but struggle eg when it comes to gluing together minature skulptures even though they might require similar skills just on a different scale.
Regarding the question if next months foundational model could solve this challenge as well, we would concede that maybe in the limited setting that we are testing in our paper there is a chance that a huge monolith could handle these inputs as well, but in the long run, adding more and more objects to the scene (think of 100s or 1000s), the input will always drift ood for these models unless they have a special read on the scene that cannot drift. Our crop design would be one way to input a scene state into a model in a way that cannot drift regardless of the number of objects put into it.
Thank you for the review and for the experiments you suggested. We ran them and report them in a new Appendix J of the revised paper, together with videos. All of these experiments use the gantry and the frozen single-cube skill of the paper without changes, and only the warping policy and the selection policy are trained, with 1 to 10 objects.
W1 other sizes, other objects, heterogeneous collections, clutter The skill, the warping and the selection policy all read the four corner points of the currently selected cube, so they are able to distinguish different object shapes. In the table, N = 20 is twice the largest training count. All generalization experiments train on line and square tasks.
Boxes, mixed sizes and scenes with cubes that have to stay in place reach 0.95 or more at N = 20. Larger cubes reach 0.77. Smaller cubes fail beyond five cubes. This decrease begins within the training counts, so it is not a failure to generalize over the count. We attribute it to the unchanged gripper, which has to work between goals that lie closer together, but we did not isolate the cause. Objects of another class, for example cylinders, would need their own single-object skill. We also trained one warping policy and one selection policy on five tasks at once without a task identifier. This pair reaches at least 0.90 on every task at N = 10.
W3 changes outside the crop The crop restricts only the warping policy. The selection policy reads every object that is not at its goal, and the crop and the warp are recomputed at every control step. Section 3 now states this. As a test, a disturbance resets one resting cube to a free position on the floor every 10 s, using the policies of the paper without retraining. All episodes in this test have twice the usual time limit, because each disturbance adds work. At N = 20, success decreases by at most 0.10 (from 0.98 to 0.88 on the line), so our method adapts to this kind of disturbance zero shot. The method does not anticipate objects that move on their own, and we added this to the limitations.
Q: turning a pyramid into a square, and taking a pyramid apart without knocking it over. We trained both tasks. Starting from a built pyramid, the square is completed in 0.84 / 0.89 of the episodes with six cubes. Starting from stacks of height three, the line is completed in 0.88 / 0.92 with ten cubes. In these runs the policies first knock the structure down. We therefore trained a second time with an added condition under which an episode ends as a failure when a cube falls before the gripper has held it. Under this condition, a pyramid is taken apart and rebuilt as a square in 0.86 / 0.98 of the episodes with three cubes, in 0.34 / 0.50 with six, and in 0.02 / 0.17 with ten. These experiments are in Appendix J as well.
W2 collision avoidance and take-over We agree that the take-over is learned and has no guarantee from the frozen skill, and the revision now states this. Appendix G examines what is learned by switching off each side of the warp at every step. The output side is needed mainly for the release, and the input side is needed in every phase. The input side also has to change during the episode. A new baseline in which the input edit is fixed per episode learns its training counts but extrapolates far less than WAVE (Section 4.3). The experiment above on taking a structure apart shows a case in which the learned take-over is not sufficient.
W4 foundation models A language or vision-language model would likely only replace the selection policy, since it would decide which object to move next. The baseline "skill alone" in the paper corresponds to this setting with a scripted planner. A rule selects a cube whose goal can currently be fulfilled, and the unchanged skill executes the step. This baseline completes no episode with three or more cubes. When the same scripted rule is combined with the warp, it reaches 0.85 on cube stacking, compared to 0.82 with the learned selection policy, which is better on the pyramid (0.47 against 0.19). These results suggest that the execution next to other objects limits performance more than the order of the steps. A foundation model could therefore serve as the planner together with WAVE. We did not run an experiment with a foundation model.
Changes in the revision New Appendix J, a summary of its results in Section 4.1, revised limitations, a clarification about the crop in Section 3, and a paragraph on foundation-model planners in the related work.