acceptodds
Under review as a conference paper at ICLR 2027

LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulation

Abstract

Human-like generalization in open-world remains a fundamental challenge for robotic manipulation. Existing learning-based methods, including reinforcement learning, imitation learning, and vision-language-action-models (VLAs), often struggle with novel tasks and unseen environments. Another promising direction is to explore generalizable representations that capture fine-grained spatial and geometric relations for open-world manipulation. While large-language-model (LLMs) and vision-language-model (VLMs) provide strong semantic reasoning based on language or annotated 2D representations, their limited 3D awareness restricts their applicability to fine-grained manipulation. To address this, we propose LAMP, which lifts image-editing as 3D priors to extract inter-object 3D transformations as continuous, geometry-aware representations. Our key insight is that image-editing inherently encodes rich 2D spatial cues, and lifting these implicit cues into 3D transformations provides fine-grained and accurate guidance for open-world manipulation. Extensive experiments demonstrate that \codename delivers precise 3D transformations and achieves strong zero-shot generalization in open-world manipulation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
open-world manipulation, 3D registration, zero-shot manipulation, image-editing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_m8FYOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LAMP uses an image-editing model to show what a scene should look like after a task. It then estimates the required 3D object motion by matching the observed and edited point clouds. The method includes point cloud filtering, scale correction, and grasp selection. The authors evaluate it in simulation and on 13 real-world manipulation tasks.

Critical strengths
  1. The method offers a useful way to turn generated images into robot actions. The filtering and scale correction address practical problems with noisy depth and inconsistent generated geometry.

  2. The experiments cover a good range of tasks, including insertion, pouring, stacking, and drawer manipulation. LAMP outperforms the tested baselines on most real-world tasks and performs well in randomized simulation scenes without task-specific robot training.

Critical weaknesses
  1. It is unclear how the image editor is chosen for each task. Table 6 shows that Qwen and Gemini perform quite differently, and Table 4 matches the better result for each of the five shared tasks. The paper also mentions generating multiple candidates but does not explain how many are generated or how one is selected. These details matter for interpreting the reported success rates.

  2. The main component ablations only cover ring stacking. This makes it hard to tell whether filtering and scale correction explain the improvements on other tasks. A comparison with GoalVLA using the same image editor and execution setup would help. The claim that sparse keypoints cannot represent continuous geometric relations is also too strong; the failures shown could come from inaccurate keypoints or poorly generated constraints.

Questions for authors
  1. Which image editor is used for each task, and how is it chosen? How many candidate images are generated per trial? Is selection fully automatic, and are rejected trials counted as failures?

  2. Can you test filtering and scale correction on insertion or assembly tasks, beyond ring stacking? Does LAMP still outperform GoalVLA when both use the same edited images and execution setup?

  3. What happens when the editor changes one object’s size or moves the supposedly stationary object? Can the system detect these errors?

  4. Table 1 reports about 30 degrees of rotation error for block assembly. Does this reflect object symmetry, multiple valid goal poses, or an alignment error? Also, what threshold is used to replace baseline errors with dashes?

5 comments

We thank Reviewer m8FY for the constructive review and for recognizing the practicality of turning generated images into robot actions, the role of filtering and scale correction in handling noisy depth and inconsistent generated geometry, and the range of tasks evaluated without task-specific training.

[W1, Q1, Q3] Editor selection protocol

We consider two failure modes of image editing: (1) changes of the passive object. It will corrupts the passive reference from which LAMP recovers the goal frame and its scale; and (2) an incorrect active–passive interaction, e.g., wrong placement of the active object. Before registration, a vision-language verifier compares the observation and the edited image against the instruction. It checks whether the active object moved to the correct location and whether the passive reference retained its position and shape.

Candidate selection in Table 4. For each trial, the editor of Qwen-Image-3.0 and Gemini-3.1-Flash-Image generates 2 candidates, and a vision-language verifier compares each candidate with the observation against the instruction. The first accepted candidate is executed, and trials without an accepted candidate are counted as failures.

Quantitative benchmark of editors and verifier. We evaluate Gemini-3.1-Flash-Image, GPT-Image-2, and Qwen-Image-3.0 on 35 scenes across five tasks: three RoboTwin tasks (place_object_stand, move_pillbottle_pad, stack_bowls_two; 5 trials each) and two real-world tasks (pencil insertion and block assembly; 10 trials each). And generates two edited candidates per editor per scene. To assess the verifier, we additionally label every candidate by human annotation and treat the human label as ground truth. We used their labels as the reference for verifier to report precision and recall as table below.

Editor quality. Gemini has the highest human approval rate in both domains, while Qwen has the lowest on RoboTwin. Overall, above 75% of candidates are human-approved. Verifier precision. Verifier has high precision. Among candidates accepted by the verifier, 96.3% (real world) and 98.0% (RoboTwin) are human-approved. Verifier recall. The verifier recovers 84.6% (real world) and 69.6% (RoboTwin) of the valid candidates. It is therefore highly precise but stricter than the human annotators.

DomainTaskEditorHuman-approvedPrecisionRecall
Real worldPencil insertionGemini19/2018/1818/19
Real worldPencil insertionGPT12/2011/1311/12
Real worldPencil insertionQwen15/2015/1615/15
Real worldBlock assemblyGemini18/2015/1515/18
Real worldBlock assemblyGPT13/2010/1010/13
Real worldBlock assemblyQwen14/208/88/14
Real worldTotalAll91/120 (75.8%)77/80 (96.3%)77/91 (84.6%)
RoboTwin3 tasksGemini27/3024/2424/27
RoboTwin3 tasksGPT23/3016/1616/23
RoboTwin3 tasksQwen19/308/98/19
RoboTwinTotalAll69/90 (76.7%)48/49 (98.0%)48/69 (69.6%)

Gemini had the highest human approval rate in both domains, while Qwen had the lowest on RoboTwin. Verifier has high precision. Among candidates accepted by the verifier, 96.3% (real world) and 98.0% (RoboTwin) are human-approved. The verifier recovers 84.6% (real world) and 69.6% (RoboTwin) of the valid candidates. It is therefore highly precise but stricter than the human annotators. Editor choice also affected execution in the separate comparison in Table 6: Gemini achieved 6/10 successes on toast insertion versus 1/10 for Qwen, while Qwen performed better on ring stacking.

Where do trials fail after editing? For Gemini, we followed each trial from its two candidate edits through perception, registration, and robot execution. Each column reports the number of trials reaching that stage, divided by the original number of trials for the task.

TaskEditRegistrationExecution
Pencil insertion10/108/107/10
Block assembly8/106/104/10
place_object_stand5/55/55/5
move_pillbottle_pad5/54/54/5
stack_bowls_two5/55/54/5
Total33/3528/3524/35

With two candidates and verification, editing succeeds in all 15 simulated trials, where scenes are clean, and in 18 of 20 real-world trials, where scenes are cluttered and the tasks demand high precision.

[W2, Q2] Ablation of filtering and scale correction on insertion and assembly tasks

We extended the component ablation beyond ring stacking to the five registration tasks of Sec. 4.1. Filtering suppresses spurious depth points that can shift the object centroid and distort its estimated orientation. Without filtering, task-average translation and rotation RMSE increased from 11.15 mm / 14.80° to 27.01 mm / 23.07°. Scale alignment primarily improved translation. Removing it increased translation RMSE from 15.98 to 34.30 mm on toast insertion and from 5.04 to 25.01 mm on block assembly, while rotation RMSE remained unchanged on both tasks.

MethodLid coveringToast insertionTea pouringBlock assemblyDrawer openingAverage
LAMP (w/o filter)31.53 / 22.9527.15 / 29.0227.59 / 24.1515.01 / 30.0533.77 / 9.2027.01 / 23.07
LAMP (w/o scale)21.21 / 8.7334.30 / 11.1059.26 / 21.5325.01 / 30.0588.67 / 2.6145.69 / 14.80
LAMP (full)3.38 / 8.7315.98 / 11.1014.24 / 21.535.04 / 30.0517.14 / 2.6111.15 / 14.80

We further ablate both components in real-world experiments on pencil insertion, toast insertion, and block assembly, with 10 trials per task. Removing either component reduces the success rate on all three tasks. Removing filtering causes the largest drop on pencil and toast insertion, where the camera viewpoint induces severe depth noise along the boundary of the active object. Removing scale alignment barely affects pencil insertion, but substantially degrades the precision-sensitive toast insertion and block assembly, where small translation errors are sufficient to cause failure.

TaskLAMP (full)LAMP (w/o filter)LAMP (w/o scale)
Pencil insertion7/103/106/10
Toast insertion6/102/103/10
Block assembly4/102/101/10

[W2, Q2] More comparison between LAMP and GoalVLA

Quantitative comparison. Tab.4 we use the same edited images and the same (VGGT+DINOv3). We also evaluate on the five tasks of Sec. 4.1, each with 10 distinct initial configurations, and compare three methods: (i) LAMP; (ii) the official GoalVLA implementation, which uses the official Depth Anything V2 metric checkpoint and SD-DINO features; (iii) a GoalVLA variant whose depth estimation and feature extraction are replaced by the same VGGT + DINOv3 used in LAMP. We report the translation RMSE (mm) and the symmetry-aware rotation RMSE (°) of the estimated goal pose. Symmetry is handled as follows: for lid covering, any rotation of the lid about its normal (object Y-axis) is treated as equivalent; for toast insertion, a front–back flip of the toast (a 180° rotation about the object Z-axis) is treated as equivalent; no symmetry is applied to the remaining tasks. The Average column is the unweighted mean over the five tasks.

MethodLid coveringToast insertionTea pouringBlock assemblyDrawer openingAverage
GoalVLA official (DA-V2 metric + SD-DINO)25.59 / 132.1935.58 / 66.1187.87 / 70.5763.30 / 55.8514.61 / 12.9045.39 / 67.52
GoalVLA (VGGT + DINOv3)57.85 / 12.1362.72 / 37.2489.28 / 33.6337.29 / 42.8694.72 / 20.3268.37 / 29.24
LAMP3.38 / 8.7315.98 / 11.1014.24 / 21.535.04 / 30.0517.14 / 2.6111.15 / 14.80

As shown in the table, LAMP achieves the lowest task-average error. Benefiting from object-centric interaction estimation, LAMP achieves significantly lower translation error under the same depth estimator and feature extractor. The difference is particularly clear in tasks that require alignment with a receiving object. For toast insertion, LAMP achieves 15.98 mm, compared with 35.58 mm and 62.72 mm for the two GoalVLA variants. For block assembly, the corresponding errors are 5.04 mm, 63.30 mm, and 37.29 mm. LAMP also yields the lowest rotation error on all five tasks.

We also find that the rotation error mainly depends on the quality of feature correspondence. The official GoalVLA has an average rotation error of 67.52°, largely because several lid-covering samples are estimated upside down (about 170°). With DINOv3 features, the lid-covering rotation error drops to 12.13°, as DINOv3 provides more accurate correspondences than SD-DINO.

Reviewer_UZkKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LAMP, a framework that uses image-editing models for zero-shot robotic manipulation. Given an RGB-D observation and a language instruction, the method generates an edited goal image and estimates the required inter-object SE(3) transformation through point cloud registration. To improve alignment accuracy, the authors introduce DINOv3-guided point cloud filtering and a unified scale alignment method to handle depth noise and scale inconsistencies in generated images. The estimated transformation is then used for grasp selection and motion planning. Experiments on RoboTwin 2.0, CALVIN-D, and real-world manipulation tasks show improvements over existing baselines, particularly in tasks requiring precise object alignment.

Critical strengths
  • S1. Strong performance on real-world manipulation tasks. The paper evaluates LAMP on 13 real-world tasks, including insertion, assembly, and articulated manipulation. The results show clear improvements over existing baselines, especially on tasks requiring precise object alignment. The experiments also cover different types of manipulation, which helps demonstrate the practical value of the method.

  • S2. Effective handling of depth noise and scale inconsistencies. The proposed point cloud filtering and scale alignment methods address two important challenges when converting edited images into 3D transformations. The qualitative results and ablation studies show that both components help improve alignment accuracy and manipulation success. This makes the overall approach more reliable for fine-grained manipulation.

Critical weaknesses
  • W1. Limited novelty compared with GoalVLA. The overall pipeline is quite similar to GoalVLA, which also converts generated visual goals into 3D transformations for robot manipulation. The main differences appear to be the point cloud filtering and unified scale alignment methods. While these improvements are useful, the paper could better explain what is fundamentally new compared with existing approaches.
  • W2. Limited ablation studies across different tasks. The ablation studies for point cloud filtering and scale alignment are mainly conducted on the ring-stacking task under three camera viewpoints. Although the results show that both components improve performance, it is unclear whether the same improvements hold for other manipulation tasks. More ablations on insertion and assembly tasks would help support the generality of these components.
Questions for authors
  • Q1. What are the main technical differences between LAMP and GoalVLA beyond point cloud filtering and scale alignment? Could the authors further clarify the novelty of the proposed approach?
  • Q2. Could the authors provide additional ablation results for point cloud filtering and scale alignment on other manipulation tasks, such as insertion and assembly, to verify whether these components consistently improve performance?
  • Q3. How does LAMP handle cases where the image-editing model changes the appearance or position of the passive object? Could the authors provide more quantitative results on how editing failures affect the final manipulation success rate?
4 comments

We thank Reviewer UZkK for the constructive review and for recognizing the strong real-world performance on precision-demanding tasks and the effectiveness of filtering and scale alignment in handling depth noise and scale inconsistencies.

[W1, Q1] Technical difference from GoalVLA

Limitations of GoalVLA's depth calibration. The key difference between LAMP and GoalVLA lies in how they account for the imperfections of real-world perception. GoalVLA relies on an idealized assumption: a single scene-level affine mapping, fitted between predicted and sensor depth on the observation, is taken to be accurate throughout the image and to transfer unchanged to the goal image. Real-world observations violate this assumption in three ways.

(1) Sensor depth is locally unreliable. Commodity depth sensors produce holes, as well as errors at sharp depth discontinuities and at long range. Holes reduce the valid support of the fit, while the erroneous regions bias it. Because the manipulation-relevant region occupies only a small part of the image (a median of 2.03% of the fitted pixels across our experiment pairs), the fit is dominated by the background and leaves an object-level depth bias. This bias displaces the reconstructed object along the viewing ray and corrupts the estimated translation, even under correct 2D correspondences.

(2) The observation and the edited image do not share one calibration. Image editing may alter regions beyond the manipulated object, for example by introducing or modifying other objects. The depth predictor's output on the goal image then no longer follows the mapping fitted on the observation, and transferring that mapping adds further error.

(3) The calibration is sensitive to camera viewpoint. As shown in our viewpoint ablation (Sec. 4.4), the background exhibits large depth variation as the camera moves from 0° to 45°. In several scenes, depth far from the near plane shifts substantially, so the scene-level fit places the object away from its true location before registration begins.

LAMP instead builds on the object-centric interaction prior carried by the region where the manipulated and stationary objects interact, rather than on whole-image depth. It estimates the object-centric scale from the passive object and expresses the motion of the manipulated active object relative to this reference. Errors from the background depth and other areas does not enter the estimate, which largely enhance its robustness to depth noises and editing errors.

Quantitative comparison. We evaluate on the five tasks of Sec. 4.1, each with 10 distinct initial configurations, and compare three methods: (i) LAMP; (ii) the official GoalVLA implementation, which uses the official Depth Anything V2 metric checkpoint and SD-DINO features; (iii) a GoalVLA variant whose depth estimation and feature extraction are replaced by the same VGGT + DINOv3 used in LAMP. We report the translation RMSE (mm) and the symmetry-aware rotation RMSE (°) of the estimated goal pose. Symmetry is handled as follows: for lid covering, any rotation of the lid about its normal (object Y-axis) is treated as equivalent; for toast insertion, a front–back flip of the toast (a 180° rotation about the object Z-axis) is treated as equivalent; no symmetry is applied to the remaining tasks. The Average column is the unweighted mean over the five tasks.

Results. As shown in the table below, LAMP achieves the lowest task-average error. Benefit from the that object-centric interaction estimation, LAMP achieves lower translation error under the same depth estimator and feature extractor. The difference is particularly clear in tasks that require alignment with a receiving object. For toast insertion, LAMP achieves 15.98 mm, compared with 35.58 mm and 62.72 mm for the two GoalVLA variants. For block assembly, the corresponding errors are 5.04 mm, 63.30 mm, and 37.29 mm. LAMP also yields the lowest rotation error on all five tasks.

We also find that the rotation error mainly depends on the quality of feature correspondence. The official GoalVLA has an average rotation error of 67.52°, largely because several lid-covering samples are estimated upside down (about 170°). With DINOv3 features, the lid-covering rotation error drops to 12.13°, as DINOv3 provides more accurate correspondences than SD-DINO.

MethodLid coveringToast insertionTea pouringBlock assemblyDrawer openingAverage
GoalVLA official (DA-V2 metric + SD-DINO)25.59 / 132.1935.58 / 66.1187.87 / 70.5763.30 / 55.8514.61 / 12.9045.39 / 67.52
GoalVLA (VGGT + DINOv3)57.85 / 12.1362.72 / 37.2489.28 / 33.6337.29/42.8694.72 / 20.3268.37 / 29.24
LAMP3.38 / 8.7315.98 / 11.1014.24 / 21.535.04 / 30.057.14 / 2.6111.15 / 14.80

Why object-centric registration helps. In toast insertion, the quantity that determines placement is the toast’s displacement relative to the slot. GoalVLA transfers a scene-wide depth calibration from the observation to the goal. In our diagnostic pairs, the manipulated object accounted for a median of only 2.03% of the pixels used to fit this calibration. A mapping can therefore fit most of the scene while placing the toast’s reconstructed centroid too far forward or backward along the viewing ray. Even correct 2D correspondences then yield an incorrect translation relative to the slot. Sharp depth boundaries and changes in visibility with viewpoint can further reduce reliable depth support around the object. LAMP uses the stationary object visible in both frames to estimate scale and expresses the manipulated object’s motion relative to that reference. A reconstruction shift shared by the toast and the slot can thus cancel in their relative placement.

[W2, Q2] Component ablations on insertion and assembly tasks

Registration ablation on five tasks. We extended the component ablation beyond ring stacking to five registration tasks of Sec. 4.1. Filtering suppresses spurious depth points that can shift the object centroid and distort its estimated orientation. Without filtering, task-average translation and rotation RMSE increased from 11.15 mm / 14.80° to 27.01 mm / 23.07°. Scale alignment primarily improved translation. Removing it increased translation RMSE from 15.98 to 34.30 mm on toast insertion and from 5.04 to 25.01 mm on block assembly, while rotation RMSE remained unchanged on both tasks.

MethodLid coveringToast insertionTea pouringBlock assemblyDrawer openingAverage
LAMP (w/o filter)31.53 / 22.9527.15 / 29.0227.59 / 24.1515.01 / 30.0533.77 / 9.2027.01 / 23.07
LAMP (w/o scale)21.21 / 8.7334.30 / 11.1059.26 / 21.5325.01 / 30.0588.67 / 2.6145.69 / 14.80
LAMP (full)3.38 / 8.7315.98 / 11.1014.24 / 21.535.04 / 30.0517.14 / 2.6111.15 / 14.80

Real-world ablation. We further ablate both components in real-world experiments on pencil insertion, toast insertion, and block assembly, with 10 trials per task. Removing either component reduces the success rate on all three tasks. Removing filtering causes the largest drop on pencil and toast insertion, where the camera viewpoint induces severe depth noise along the boundary of the active object. Removing scale alignment barely affects pencil insertion, but substantially degrades the precision-sensitive toast insertion and block assembly, where small translation errors are sufficient to cause failure.

TasksLAMP (full)LAMP (w/o filter)LAMP (w/o scale)
Pencil insertion7/103/106/10
Toast insertion6/102/103/10
Block assembly4/102/101/10

[Q3] Editing effects analysis

Editing failure modes and verification. We consider two failure modes of image editing: (1) changes of the passive object. It will corrupts the passive reference from which LAMP recovers the goal frame and its scale; and (2) an incorrect active–passive interaction, e.g., wrong placement of the active object. Before registration, a vision-language verifier compares the observation and the edited image against the instruction. It checks whether the active object moved to the correct location and whether the passive reference retained its position and shape.

Quantitative benchmark of editors and verifier. We evaluate Gemini-3.1-Flash-Image, GPT-Image-2, and Qwen-Image-3.0 on 35 scenes across five tasks: three RoboTwin tasks (place_object_stand, move_pillbottle_pad, stack_bowls_two; 5 trials each) and two real-world tasks (pencil insertion and block assembly; 10 trials each). And generates two edited candidates per editor per scene. To assess the verifier, we additionally label every candidate by human annotation and treat the human label as ground truth. We used their labels as the reference for verifier to report precision and recall as table below.

Editor quality. Gemini has the highest human approval rate in both domains, while Qwen has the lowest on RoboTwin. Overall, above 75% of candidates are human-approved. Verifier precision. Verifier has high precision. Among candidates accepted by the verifier, 96.3% (real world) and 98.0% (RoboTwin) are human-approved. Verifier recall. The verifier recovers 84.6% (real world) and 69.6% (RoboTwin) of the valid candidates. It is therefore highly precise but stricter than the human annotators.

DomainTaskEditorHuman-approvedPrecisionRecallTial with >=1 valid images
Real worldPencil insertionGemini19/2018/1818/1910/10
Real worldPencil insertionGPT12/2011/1311/128/10
Real worldPencil insertionQwen15/2015/1615/1510/10
Real worldBlock assemblyGemini18/2015/1515/188/10
Real worldBlock assemblyGPT13/2010/1010/137/10
Real worldBlock assemblyQwen14/208/88/148/10
Real worldTotalAll91/120 (75.8%)77/80 (96.3%)77/91 (84.6%)18/20 (90.0%)
RoboTwin3 tasksGemini27/3024/2424/275/5
RoboTwin3 tasksGPT23/3016/1616/235/5
RoboTwin3 tasksQwen19/308/98/195/5
RoboTwinTotalAll69/90 (76.7%)48/49 (98.0%)48/69 (69.6%)5/5 (100.0%)

Where do trials fail after editing? For Gemini, we followed each trial from its two candidate edits through perception, registration, and robot execution. Each column reports the number of trials reaching that stage, divided by the original number of trials for the task.

TaskEditRegistrationExecution
Pencil insertion10/108/107/10
Block assembly8/106/104/10
place_object_stand5/55/55/5
move_pillbottle_pad5/54/54/5
stack_bowls_two5/55/54/5
Total34/3528/3524/35

With two candidates and verification, editing succeeds in all 15 simulated trials, where scenes are clean, and in 18 of 20 real-world trials, where scenes are cluttered and the tasks demand high precision.

Reviewer_kc4COpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LAMP, a framework that leverages pretrained image-editing models to derive explicit 3D geometric priors for zero-shot robotic manipulation. The central idea is to generate a visual goal state from a language instruction and recover the corresponding inter-object SE(3) transformation through cross-state point cloud registration.

To improve the accuracy and robustness of this process, the authors introduce a hierarchical point-cloud filtering strategy using DINOv3 features and spatial clustering, together with a unified scale-alignment mechanism that preserves the relative geometry between active and passive objects. The resulting transformation is used for goal-informed grasp selection and collision-aware motion planning.

The method is evaluated on simulation benchmarks, including RoboTwin 2.0 and CALVIN-D, as well as 13 real-world manipulation tasks involving insertion, assembly, stacking, and articulated objects. The results demonstrate substantial improvements over the evaluated baselines, particularly in high-precision manipulation and randomized environments, without task-specific policy training.

Critical strengths

1. An effective and interpretable approach to geometry-aware manipulation.

The paper presents a well-motivated framework for converting generative image-editing priors into explicit inter-object SE(3) transformations. Compared with approaches that rely on sparse keypoints or language-based constraints, this representation provides a more direct way to capture relative object poses and fine-grained geometric alignment. The hierarchical point-cloud filtering and unified scale-alignment mechanisms address practical challenges arising from noisy single-view depth estimation and inconsistencies in generated goal images. The resulting framework is interpretable and can be integrated with conventional robotic motion planning.

2. Convincing real-world validation across diverse manipulation tasks.

A notable strength is the extensive evaluation on physical robots. The authors evaluate 13 real-world manipulation tasks with 10 trials per task, covering placement, insertion, assembly, stacking, and articulated manipulation. The reported results are encouraging, with 66% average success across the first 10 tasks compared with 28% for GoalVLA, and 60% across three articulated manipulation tasks.

The additional simulation experiments, viewpoint ablations, and spatial-registration evaluations provide complementary evidence of the approach's effectiveness. In particular, the improvements on precision-demanding tasks support the practical relevance of explicit geometric reasoning.

Critical weaknesses

1. The methodological novelty relative to existing image-editing-based manipulation approaches could be better established.

While LAMP demonstrates strong empirical performance, its overall formulation is closely related to existing approaches such as GoalVLA, which also converts generated visual goals into 3D transformations for manipulation. The main methodological differences appear to lie in the proposed hierarchical point-cloud filtering and unified scale-alignment strategies.

Although these components are practically meaningful, the current ablation studies are relatively limited. In particular, Table 5 evaluates their contributions primarily on a single ring-stacking task under three camera viewpoints. It remains unclear whether the improvements generalize consistently across the full range of manipulation tasks or how much each component contributes to the substantial performance gap over GoalVLA.

A more controlled comparison using identical edited goals and perception inputs, together with broader component-wise ablations across multiple tasks, would help establish the specific methodological contributions of LAMP.

2. The dependence on image-editing models and the evaluation protocol limit the strength of the open-world generalization claim.

A central assumption is that pretrained image-editing models can reliably generate geometrically meaningful target states. However, the framework remains sensitive to editing failures, including unintended changes to passive objects, inaccurate object scales, and unsuccessful goal generation. According to the failure analysis in Appendix A.3, image-editing errors account for the largest single category of failures.

Furthermore, Table 6 reveals substantial differences between image-editing models across tasks. For example, success rates on toast insertion and pen-cap covering change from 1/10 with Qwen-Image-Edit to 6/10 with Gemini, whereas the opposite trend is observed for ring stacking. Interestingly, the results reported for LAMP in Table 4 match the better-performing editor for each of the five tasks examined in Table 6. The paper does not clearly explain whether editor selection is predetermined, automatic, or based on task-specific empirical performance. Clarifying this protocol is important for evaluating the extent of zero-shot generalization.

Finally, although the 13 real-world tasks provide meaningful evidence of generality, each task is evaluated with only 10 initial configurations. The long-horizon demonstrations are also manually decomposed and primarily qualitative. Consequently, the evidence for robust performance across substantially unseen objects, task compositions, and extended autonomous execution remains limited.

Questions for authors

Q1. Contribution of the proposed geometric components.

Could the authors provide additional ablation studies of hierarchical point-cloud filtering and unified scale alignment across multiple real-world tasks beyond ring stacking? In particular, could they compare LAMP against GoalVLA under matched edited goal images, depth estimation, and segmentation inputs to isolate the improvements attributable to the proposed registration and scale-alignment mechanisms?

Q2. Image-editor selection and reproducibility.

How are different image-editing models selected for individual tasks? In Table 6, the success rates reported in Table 4 correspond to the better-performing editor for all five evaluated tasks. Is the editor selected before evaluation using predefined criteria, through an automatic selection mechanism, or based on empirical performance? Could the authors clarify the selection protocol and report the overall performance using a single fixed editor across all tasks?

Q3. Scope of real-world generalization.

While the real-world evaluation covers 13 task categories, each task is evaluated over 10 configurations. To what extent do these experiments involve unseen object instances rather than rearrangements of the same objects? Could the authors clarify the diversity of objects and environments used, and provide additional evidence for generalization to unseen instances or task compositions?

Q4. Long-horizon reliability.

The long-horizon tasks in Figure 12 are manually decomposed into subtasks and presented through successful execution examples. Could the authors report quantitative end-to-end success rates over repeated trials, including the frequency of failures arising from image editing, geometric registration, or accumulated execution errors?

9 comments

We thank Reviewer kc4C for the constructive review and for recognizing the well-motivated conversion of image-editing priors into explicit SE(3) transformations, the practical value of hierarchical filtering and scale alignment, and the extensive real-world validation.

[W1, Q1] Difference from GoalVLA, controlled comparison, and component ablations

The key difference between LAMP and GoalVLA lies in how they account for the imperfections of real-world perception. GoalVLA relies on an idealized assumption: a single scene-level affine mapping, fitted between predicted and sensor depth on the observation, is taken to be accurate throughout the image and to transfer unchanged to the goal image. Real-world observations violate this assumption in three ways.

(1) Sensor depth is locally unreliable. Commodity depth sensors produce holes, as well as errors at sharp depth discontinuities and at long range. Holes reduce the valid support of the fit, while the erroneous regions bias it. Because the manipulation-relevant region occupies only a small part of the image (a median of 2.03% of the fitted pixels across our experiment pairs), the fit is dominated by the background and leaves an object-level depth bias. This bias displaces the reconstructed object along the viewing ray and corrupts the estimated translation, even under correct 2D correspondences.

(2) The observation and the edited image do not share one calibration. Image editing may alter regions beyond the manipulated object, for example by introducing or modifying other objects. The depth predictor's output on the goal image then no longer follows the mapping fitted on the observation, and transferring that mapping adds further error.

(3) The calibration is sensitive to camera viewpoint. As shown in our viewpoint ablation (Sec. 4.4), the background exhibits large depth variation as the camera moves from 0° to 45°. In several scenes, depth far from the near plane shifts substantially, so the scene-level fit places the object away from its true location before registration begins.

LAMP instead builds on the object-centric interaction prior carried by the region where the manipulated and stationary objects interact, rather than on whole-image depth. It estimates the object-centric scale from the passive object and expresses the motion of the manipulated active object relative to this reference. Errors from the background depth and other image regions do not enter the estimate, which largely enhances its robustness to depth noise and editing errors.

Quantitative comparison. We evaluate on the five tasks of Sec. 4.1, each with 10 distinct initial configurations, and compare three methods: (i) LAMP; (ii) the official GoalVLA implementation, which uses the official Depth Anything V2 metric checkpoint and SD-DINO features; (iii) a GoalVLA variant whose depth estimation and feature extraction are replaced by the same VGGT + DINOv3 used in LAMP. We report the translation RMSE (mm) and the symmetry-aware rotation RMSE (°) of the estimated goal pose. Symmetry is handled as follows: for lid covering, any rotation of the lid about its normal (object Y-axis) is treated as equivalent; for toast insertion, a front–back flip of the toast (a 180° rotation about the object Z-axis) is treated as equivalent; no symmetry is applied to the remaining tasks. The Average column is the unweighted mean over the five tasks.

MethodLid coveringToast insertionTea pouringBlock assemblyDrawer openingAverage
GoalVLA official (DA-V2 metric + SD-DINO)25.59 / 132.1935.58 / 66.1187.87 / 70.5763.30 / 55.8514.61 / 12.9045.39 / 67.52
GoalVLA (VGGT + DINOv3)57.85 / 12.1362.72 / 37.2489.28 / 33.6337.29 / 42.8694.72 / 20.3268.37 / 29.24
LAMP3.38 / 8.7315.98 / 11.1014.24 / 21.535.04 / 30.0517.14 / 2.6111.15 / 14.80

As shown in the table, LAMP achieves the lowest task-average error. Benefiting from object-centric interaction estimation, LAMP achieves significantly lower translation error under the same depth estimator and feature extractor. The difference is particularly clear in tasks that require alignment with a receiving object. For toast insertion, LAMP achieves 15.98 mm, compared with 35.58 mm and 62.72 mm for the two GoalVLA variants. For block assembly, the corresponding errors are 5.04 mm, 63.30 mm, and 37.29 mm. LAMP also yields the lowest rotation error on all five tasks.

We also find that the rotation error mainly depends on the quality of feature correspondence. The official GoalVLA has an average rotation error of 67.52°, largely because several lid-covering samples are estimated upside down (about 170°). With DINOv3 features, the lid-covering rotation error drops to 12.13°, as DINOv3 provides more accurate correspondences than SD-DINO.

Component ablations beyond ring stacking. We extended the component ablation beyond ring stacking to the five registration tasks of Sec. 4.1. Filtering suppresses spurious depth points that can shift the object centroid and distort its estimated orientation. Without filtering, task-average translation and rotation RMSE increased from 11.15 mm / 14.80° to 27.01 mm / 23.07°. Scale alignment primarily improved translation. Removing it increased translation RMSE from 15.98 to 34.30 mm on toast insertion and from 5.04 to 25.01 mm on block assembly, while rotation RMSE remained unchanged on both tasks.

MethodLid coveringToast insertionTea pouringBlock assemblyDrawer openingAverage
LAMP (w/o filter)31.53 / 22.9527.15 / 29.0227.59 / 24.1515.01 / 30.0533.77 / 9.2027.01 / 23.07
LAMP (w/o scale)21.21 / 8.7334.30 / 11.1059.26 / 21.5325.01 / 30.0588.67 / 2.6145.69 / 14.80
LAMP (full)3.38 / 8.7315.98 / 11.1014.24 / 21.535.04 / 30.0517.14 / 2.6111.15 / 14.80

We further ablate both components in real-world experiments on pencil insertion, toast insertion, and block assembly, with 10 trials per task. Removing either component reduces the success rate on all three tasks. Removing filtering causes the largest drop on pencil and toast insertion, where the camera viewpoint induces severe depth noise along the boundary of the active object. Removing scale alignment barely affects pencil insertion, but substantially degrades the precision-sensitive toast insertion and block assembly, where small translation errors are sufficient to cause failure.

TaskLAMP (full)LAMP (w/o filter)LAMP (w/o scale)
Pencil insertion7/103/106/10
Toast insertion6/102/103/10
Block assembly4/102/101/10

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.