- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
LAMP uses an image-editing model to show what a scene should look like after a task. It then estimates the required 3D object motion by matching the observed and edited point clouds. The method includes point cloud filtering, scale correction, and grasp selection. The authors evaluate it in simulation and on 13 real-world manipulation tasks.
Critical strengths
-
The method offers a useful way to turn generated images into robot actions. The filtering and scale correction address practical problems with noisy depth and inconsistent generated geometry.
-
The experiments cover a good range of tasks, including insertion, pouring, stacking, and drawer manipulation. LAMP outperforms the tested baselines on most real-world tasks and performs well in randomized simulation scenes without task-specific robot training.
Critical weaknesses
-
It is unclear how the image editor is chosen for each task. Table 6 shows that Qwen and Gemini perform quite differently, and Table 4 matches the better result for each of the five shared tasks. The paper also mentions generating multiple candidates but does not explain how many are generated or how one is selected. These details matter for interpreting the reported success rates.
-
The main component ablations only cover ring stacking. This makes it hard to tell whether filtering and scale correction explain the improvements on other tasks. A comparison with GoalVLA using the same image editor and execution setup would help. The claim that sparse keypoints cannot represent continuous geometric relations is also too strong; the failures shown could come from inaccurate keypoints or poorly generated constraints.
Questions for authors
-
Which image editor is used for each task, and how is it chosen? How many candidate images are generated per trial? Is selection fully automatic, and are rejected trials counted as failures?
-
Can you test filtering and scale correction on insertion or assembly tasks, beyond ring stacking? Does LAMP still outperform GoalVLA when both use the same edited images and execution setup?
-
What happens when the editor changes one object’s size or moves the supposedly stationary object? Can the system detect these errors?
-
Table 1 reports about 30 degrees of rotation error for block assembly. Does this reflect object symmetry, multiple valid goal poses, or an alignment error? Also, what threshold is used to replace baseline errors with dashes?
5 comments
Where do trials fail after editing? For Gemini, we followed each trial from its two candidate edits through perception, registration, and robot execution. Each column reports the number of trials reaching that stage, divided by the original number of trials for the task.
| Task | Edit | Registration | Execution |
|---|---|---|---|
| Pencil insertion | 10/10 | 8/10 | 7/10 |
| Block assembly | 8/10 | 6/10 | 4/10 |
| place_object_stand | 5/5 | 5/5 | 5/5 |
| move_pillbottle_pad | 5/5 | 4/5 | 4/5 |
| stack_bowls_two | 5/5 | 5/5 | 4/5 |
| Total | 33/35 | 28/35 | 24/35 |
With two candidates and verification, editing succeeds in all 15 simulated trials, where scenes are clean, and in 18 of 20 real-world trials, where scenes are cluttered and the tasks demand high precision.
[W2, Q2] Ablation of filtering and scale correction on insertion and assembly tasks
We extended the component ablation beyond ring stacking to the five registration tasks of Sec. 4.1. Filtering suppresses spurious depth points that can shift the object centroid and distort its estimated orientation. Without filtering, task-average translation and rotation RMSE increased from 11.15 mm / 14.80° to 27.01 mm / 23.07°. Scale alignment primarily improved translation. Removing it increased translation RMSE from 15.98 to 34.30 mm on toast insertion and from 5.04 to 25.01 mm on block assembly, while rotation RMSE remained unchanged on both tasks.
| Method | Lid covering | Toast insertion | Tea pouring | Block assembly | Drawer opening | Average |
|---|---|---|---|---|---|---|
| LAMP (w/o filter) | 31.53 / 22.95 | 27.15 / 29.02 | 27.59 / 24.15 | 15.01 / 30.05 | 33.77 / 9.20 | 27.01 / 23.07 |
| LAMP (w/o scale) | 21.21 / 8.73 | 34.30 / 11.10 | 59.26 / 21.53 | 25.01 / 30.05 | 88.67 / 2.61 | 45.69 / 14.80 |
| LAMP (full) | 3.38 / 8.73 | 15.98 / 11.10 | 14.24 / 21.53 | 5.04 / 30.05 | 17.14 / 2.61 | 11.15 / 14.80 |
We further ablate both components in real-world experiments on pencil insertion, toast insertion, and block assembly, with 10 trials per task. Removing either component reduces the success rate on all three tasks. Removing filtering causes the largest drop on pencil and toast insertion, where the camera viewpoint induces severe depth noise along the boundary of the active object. Removing scale alignment barely affects pencil insertion, but substantially degrades the precision-sensitive toast insertion and block assembly, where small translation errors are sufficient to cause failure.
| Task | LAMP (full) | LAMP (w/o filter) | LAMP (w/o scale) |
|---|---|---|---|
| Pencil insertion | 7/10 | 3/10 | 6/10 |
| Toast insertion | 6/10 | 2/10 | 3/10 |
| Block assembly | 4/10 | 2/10 | 1/10 |
[W2, Q2] More comparison between LAMP and GoalVLA
Quantitative comparison. Tab.4 we use the same edited images and the same (VGGT+DINOv3). We also evaluate on the five tasks of Sec. 4.1, each with 10 distinct initial configurations, and compare three methods: (i) LAMP; (ii) the official GoalVLA implementation, which uses the official Depth Anything V2 metric checkpoint and SD-DINO features; (iii) a GoalVLA variant whose depth estimation and feature extraction are replaced by the same VGGT + DINOv3 used in LAMP. We report the translation RMSE (mm) and the symmetry-aware rotation RMSE (°) of the estimated goal pose. Symmetry is handled as follows: for lid covering, any rotation of the lid about its normal (object Y-axis) is treated as equivalent; for toast insertion, a front–back flip of the toast (a 180° rotation about the object Z-axis) is treated as equivalent; no symmetry is applied to the remaining tasks. The Average column is the unweighted mean over the five tasks.
| Method | Lid covering | Toast insertion | Tea pouring | Block assembly | Drawer opening | Average |
|---|---|---|---|---|---|---|
| GoalVLA official (DA-V2 metric + SD-DINO) | 25.59 / 132.19 | 35.58 / 66.11 | 87.87 / 70.57 | 63.30 / 55.85 | 14.61 / 12.90 | 45.39 / 67.52 |
| GoalVLA (VGGT + DINOv3) | 57.85 / 12.13 | 62.72 / 37.24 | 89.28 / 33.63 | 37.29 / 42.86 | 94.72 / 20.32 | 68.37 / 29.24 |
| LAMP | 3.38 / 8.73 | 15.98 / 11.10 | 14.24 / 21.53 | 5.04 / 30.05 | 17.14 / 2.61 | 11.15 / 14.80 |
As shown in the table, LAMP achieves the lowest task-average error. Benefiting from object-centric interaction estimation, LAMP achieves significantly lower translation error under the same depth estimator and feature extractor. The difference is particularly clear in tasks that require alignment with a receiving object. For toast insertion, LAMP achieves 15.98 mm, compared with 35.58 mm and 62.72 mm for the two GoalVLA variants. For block assembly, the corresponding errors are 5.04 mm, 63.30 mm, and 37.29 mm. LAMP also yields the lowest rotation error on all five tasks.
We also find that the rotation error mainly depends on the quality of feature correspondence. The official GoalVLA has an average rotation error of 67.52°, largely because several lid-covering samples are estimated upside down (about 170°). With DINOv3 features, the lid-covering rotation error drops to 12.13°, as DINOv3 provides more accurate correspondences than SD-DINO.
We thank Reviewer m8FY for the constructive review and for recognizing the practicality of turning generated images into robot actions, the role of filtering and scale correction in handling noisy depth and inconsistent generated geometry, and the range of tasks evaluated without task-specific training.
[W1, Q1, Q3] Editor selection protocol
We consider two failure modes of image editing: (1) changes of the passive object. It will corrupts the passive reference from which LAMP recovers the goal frame and its scale; and (2) an incorrect active–passive interaction, e.g., wrong placement of the active object. Before registration, a vision-language verifier compares the observation and the edited image against the instruction. It checks whether the active object moved to the correct location and whether the passive reference retained its position and shape.
Candidate selection in Table 4. For each trial, the editor of Qwen-Image-3.0 and Gemini-3.1-Flash-Image generates 2 candidates, and a vision-language verifier compares each candidate with the observation against the instruction. The first accepted candidate is executed, and trials without an accepted candidate are counted as failures.
Quantitative benchmark of editors and verifier. We evaluate Gemini-3.1-Flash-Image, GPT-Image-2, and Qwen-Image-3.0 on 35 scenes across five tasks: three RoboTwin tasks (place_object_stand, move_pillbottle_pad, stack_bowls_two; 5 trials each) and two real-world tasks (pencil insertion and block assembly; 10 trials each). And generates two edited candidates per editor per scene. To assess the verifier, we additionally label every candidate by human annotation and treat the human label as ground truth. We used their labels as the reference for verifier to report precision and recall as table below.
Editor quality. Gemini has the highest human approval rate in both domains, while Qwen has the lowest on RoboTwin. Overall, above 75% of candidates are human-approved. Verifier precision. Verifier has high precision. Among candidates accepted by the verifier, 96.3% (real world) and 98.0% (RoboTwin) are human-approved. Verifier recall. The verifier recovers 84.6% (real world) and 69.6% (RoboTwin) of the valid candidates. It is therefore highly precise but stricter than the human annotators.
Gemini had the highest human approval rate in both domains, while Qwen had the lowest on RoboTwin. Verifier has high precision. Among candidates accepted by the verifier, 96.3% (real world) and 98.0% (RoboTwin) are human-approved. The verifier recovers 84.6% (real world) and 69.6% (RoboTwin) of the valid candidates. It is therefore highly precise but stricter than the human annotators. Editor choice also affected execution in the separate comparison in Table 6: Gemini achieved 6/10 successes on toast insertion versus 1/10 for Qwen, while Qwen performed better on ring stacking.