acceptodds
Under review as a conference paper at ICLR 2027

Give the Policy Its View Back: Recovering Frozen Visuomotor Policies from Camera Drift

12 views

Abstract

Visuomotor policies can fail under small camera drift even when the task and action space remain unchanged. We study how to recover such policies without updating their weights. We introduce **ReView**, a test-time input calibration method that fits a six-parameter image warp from stored demonstration images and only four unlabeled reset images from the displaced camera. ReView matches early spatial features across the two unpaired image sets, requiring no camera poses, depth, target actions, or interaction rollouts. The fitted warp corrects incoming images while the policy remains frozen. Across six tasks in robomimic and ManiSkill, ReView improves mean success over no adaptation by 28, 22, and 24 percentage points for Diffusion Policy, ACT, and SmolVLA, respectively, and by 27 and 30 percentage points on two real-robot tasks. Ablations show that spatial correspondence is critical for recovery. Our analysis further shows that geometric alignment and policy recovery need not coincide: under parallax, an imperfect scene alignment can better preserve the visual relationships that matter for control. Recovering a policy therefore requires a view it can act on, not necessarily an exact reconstruction of its original view.

Primary area:
applications to robotics, autonomy, planning
Keywords:
visuomotor policies, camera drift, viewpoint robustness, test-time adaptation, input calibration, robot manipulation

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_6sbMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ReView, a test-time calibration method for recovering frozen visuomotor policies after camera displacement. It fits a six-parameter image warp by matching mean spatial features from stored nominal-view demonstration images and four unpaired reset images from the displaced camera. The fitted warp corrects subsequent observations without policy updates, depth, measured camera extrinsics, or target action labels. Experiments across three policy families, six simulation tasks, and two real-robot tasks demonstrate substantial recovery over uncorrected inputs. Ablations and geometric analysis examine the importance of spatial information and explain why accurate image alignment does not necessarily yield the best control performance.

Critical strengths
  1. The calibration procedure is lightweight and requires little additional data. ReView fits only six warp parameters using stored demonstration images and four unlabeled reset images, without updating the policy or requiring depth, measured camera extrinsics, action labels, or interaction rollouts. This makes it an appealing recovery procedure for an existing policy after a camera is displaced.

  2. The experiments demonstrate substantial recovery and provide useful analysis. ReView improves average simulation success by approximately 28, 22, and 24 percentage points for Diffusion Policy, ACT, and SmolVLA, respectively, with additional gains on two real-robot tasks. The ablations show the importance of retaining spatial information, while the parallax analysis explains why accurate tabletop alignment may still leave control-relevant gripper–object relationships distorted.

Critical weaknesses
  1. The methodological advance over existing methods is unclear. Correcting observations before a frozen policy is already explored by AnyCamVLA and GS-VLA, and optimizing geometric alignment through feature matching is well established. ReView’s distinction is its use of a few unpaired RGB images without measured geometry. However, simpler alignment methods recover much of the same performance: on robomimic with Diffusion Policy, ReView achieves 59.5% success versus 58.7% for pixel correlation and 58.5% for ECC. These results leave the significance of the proposed formulation insufficiently established.

  2. Evaluation on stronger generalist robot policies is needed. The experiments emphasize Diffusion Policy and ACT, with real-robot validation limited to Diffusion Policy. Although SmolVLA is included, it provides a narrow basis for claims of applicability to generalist policies. Moreover, ReView underperforms pixel correlation on SmolVLA in ManiSkill (37.4% versus 43.3%). Evaluation with an additional generalist policy such as π0.5 would make the study more representative and test whether the benefits extend beyond the policy families where the method currently performs best.

  3. The spatial-matching assumption remains insufficiently stress-tested. ReView assumes that source and target images retain enough comparable spatial structure for feature mismatch to identify camera displacement. With unpaired scenes, changes in object placement or robot configuration can be mistaken for camera motion; occlusion and parallax further weaken correspondence. The same-state reference and reference-selection experiments provide useful evidence, but do not establish robustness to systematic differences between source and target reset distributions. Averaging only four target images does not necessarily resolve these ambiguities. Separately, the reported 16-percentage-point degradation on Square even without camera displacement demonstrates that calibration can harm an otherwise unchanged deployment.

Questions for authors
  1. Clarification: Under what conditions does ReView offer a meaningful advantage over simpler spatial RGB or pixel-correlation alignment? Given their similar performance in several settings, please clarify what benefit is specifically attributable to the proposed feature-based formulation.

  2. Additional evaluation request: Does the benefit extend to a stronger generalist policy such as π0.5? This would be particularly informative given that pixel correlation outperforms ReView on SmolVLA in ManiSkill.

  3. Clarification and robustness evidence: What similarity between source and target reset distributions is required for reliable calibration? Have you tested systematic changes in object placement or robot configuration while holding camera displacement fixed? Please also explain whether the nominal-camera degradation on Square could be detected from the calibration objective or validity checks without executing the policy.

5 comments
AuthorsOpenReview ↗

Advantage over pixel alignment, and results on π0.5 (1/2)

We thank the reviewer for the detailed assessment, and for finding the method lightweight and the parallax analysis useful. The suggestions to test π0.5\pi_{0.5} and to examine the Square case led to two useful additions.

Unless stated otherwise: success rate (%), averaged over three tasks and drift levels L0–L3 (0–10 cm / 0–10°).

W1 & Q1: "The methodological advance over existing methods is unclear. Correcting observations before a frozen policy is already explored by AnyCamVLA and GS-VLA … simpler alignment methods recover much of the same performance …" / "Under what conditions does ReView offer a meaningful advantage over simpler spatial RGB or pixel-correlation alignment?"

On robomimic, the two are close, as the reviewer notes. On ManiSkill, feature matching is clearly better whenever the features suit the encoder.

  • robomimic's scenes are textured enough for pixels to determine the warp.
  • On ManiSkill, the right features are the policy's own for task-trained CNNs, and generic early convolutional features for ViT-based policies.
  • The advantage holds in all the harder settings we tested:
ManiSkillReView-pixReView
Main setting, DP / ACT57.3 / 57.565.1 / 65.5
Second training seed (L2), DP / ACT52.4 / 46.963.5 / 64.4
Translation-only drift (L2), DP / ACT27.9 / 26.448.9 / 48.8
Larger drift (L4), DP / ACT26.8 / 18.032.5 / 26.9
Persistent occluder, DP / ACT43.9 / 46.049.3 / 55.4
Shifted reset distribution (full strength), DP32.344.4
ViT policies with ImageNet features, SmolVLA / π0.5\pi_{0.5}43.3 / 48.847.3 / 56.8

Relation to AnyCamVLA and GS-VLA. Both also correct the input of a frozen policy, but both need the calibrated pose of the displaced camera and a view-synthesis model; GS-VLA also needs metric depth (Table 17). ReView uses four unlabeled RGB images and one fixed warp.

Contribution. We do not regard feature registration itself as the contribution. The contribution is showing that a frozen policy can be recovered from such minimal data, together with the analysis of why image alignment and action recovery can disagree. The analysis is borne out by ReView exceeding the ground-truth table-plane rectification under translation (DP 48.9 vs. 36.3; ACT 48.8 vs. 32.1).

W2 & Q2: "Evaluation on stronger generalist robot policies is needed … Does the benefit extend to a stronger generalist policy such as π0.5\pi_{0.5}?"

Yes. We fine-tuned π0.5\pi_{0.5} on both benchmarks and evaluated it with the same cameras, initial states and calibration images:

π0.5\pi_{0.5}NominalNo adapt.ReView-pixReViewReView, ImageNet features
robomimic80.043.468.567.971.2
ManiSkill60.047.148.855.256.8
  • On robomimic, ReView raises success from 43.4 to 67.9. On ManiSkill the gain is smaller.
  • Unlike SmolVLA, π0.5\pi_{0.5} on ManiSkill does not show pixels ahead of ReView.
  • π0.5\pi_{0.5} also uses a SigLIP image tower. Matching generic early convolutional features, as the criterion above suggests, gives the best result on both benchmarks.
AuthorsOpenReview ↗

Reset-distribution shift and a rollout-free check for the nominal camera (2/2)

W3 & Q3 (a): "With unpaired scenes, changes in object placement or robot configuration can be mistaken for camera motion … Have you tested systematic changes in object placement or robot configuration while holding camera displacement fixed?"

Yes. We fixed the camera displacements and shifted the reset distribution. Objects and goal moved by 3 cm in x and in y, and the robot's initial joints were offset by about 4 cm at the gripper. We applied the shift at half, full and double strength, to both the calibration resets and the rollouts (ManiSkill, DP):

ShiftNo adapt.ReView-pixReView
None36.857.365.1
Half30.646.057.4
Full22.632.344.4
Double8.19.017.4
  • Up to full strength, ReView stays more than 20 points above no adaptation, and it beats pixel matching at every strength.
  • At double strength, the policy itself mostly fails even without adaptation, which limits what any input correction can recover.
  • In Proposition 1, such a mismatch is the bias term d₀. This is why we calibrate on initial frames, which are not affected by the displaced policy's own actions.

W3 & Q3 (b): "… whether the nominal-camera degradation on Square could be detected from the calibration objective or validity checks without executing the policy."

Yes.

Why Square drops. At the nominal camera, the fitted warp on Square is only 0.26 px, so the drop is not caused by a large wrong correction. Passing the images through the warp sampler with near-identity warps already lowers Square by about 5 points. The closed loop on this task is sensitive to tiny pixel changes.

The check. It uses only the calibration images, and estimates the noise term of Proposition 1.

  • We resample the 16 source and 4 reset images 20 times and refit the warp each time.
  • If the fitted displacement lies within the 95th percentile of the resampled fits, we keep the identity. The warp is then skipped, and the policy sees the raw frames.
  • The check takes 2–3 minutes on one GPU and needs no rollouts.

Results.

  • At the nominal camera, the check keeps the identity on all three robomimic tasks and on all nine ManiSkill task–policy combinations. This removes the degradation; for example, DP on PushCube returns to 91%.
  • Under real drift, it triggers only in a few cells at the smallest level, L0, where the unadapted policy already does well. The overall cost is 0.2–0.3 points.

We will add this check to Sec. 3.

We hope these answers address the reviewer's concerns, and we would be glad to discuss further.

AuthorsOpenReview ↗

Kind Request for Feedback on Our Rebuttal

Dear Reviewer 6sbM,

Thank you again for your detailed assessment and helpful suggestions. As the discussion period is drawing to a close, we wanted to kindly follow up on our rebuttal and the revised paper.

Following your comments, we added results on π0.5\pi_{0.5}, tests with shifted reset distributions, and a rollout-free check for the nominal camera.

We would be grateful to know whether our responses resolve your concerns. If any remain, we would be happy to provide further clarification or additional results before the deadline.

Best regards, The Authors

Reviewer_vH7ZOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReView, a lightweight method that recovers frozen visuomotor policies after camera displacement by fitting a six-parameter image warp using 16 nominal-view demonstration frames and four unpaired reset images. Calibration matches spatially resolved early features from the frozen policy encoder, and the fitted warp is reused during deployment. Across six simulated tasks, the paper reports average success improvements of approximately 28%, 22%, and 24% for Diffusion Policy, ACT, and SmolVLA, respectively. Two real-robot tasks provide additional evidence of practical recovery. The paper also analyzes spatial identifiability and why geometric alignment and action accuracy can favor different corrections.

Critical strengths
  1. The method addresses a practical deployment problem with limited requirements: no target actions, depth, camera extrinsics, or policy updates are needed for calibration.
  2. The internal comparisons examine spatial versus pooled features, RGB and pixel-correlation objectives, classical registration, and warp parameterization. These controls help distinguish the value of spatial alignment from that of learned features.
  3. The analysis connects alignment to control through regional sensitivity and action-error measurements. Reporting negative results and cases where simpler objectives perform better makes the empirical conclusions more informative and convincing.
Critical weaknesses
  1. The additional benefit of policy features over simple image matching is inconsistent, limiting the method-specific novelty. RGB or pixel matching is close to ReView on robomimic, and ReView-pix achieves higher average success for SmolVLA across the six tasks, supporting spatial registration more strongly than the particular feature-matching choice.
  2. The real-robot experiments compare ReView only against no adaptation. They demonstrate practical recovery but do not establish whether feature matching offers an advantage over pixel correlation or ECC in physical deployments.
  3. The comparison with network adaptation is limited by the ActMAD configuration, which uses a single SGD update with ImageNet hyperparameters without explaining why these settings are appropriate for this calibration task.
  4. Section 4 introduces several local models and sensitivity matrices without clearly explaining their roles or connecting them to the actual calibration procedure. Although brief intuitions are provided, I find the part a bit hard to follow.
Questions for authors
  1. Do existing results suggest a practical criterion for choosing feature matching over pixel correlation?
  2. Are pixel-correlation or ECC results available for either real-robot task?
  3. What development-set evidence supports the ActMAD learning rate and single-update budget?
4 comments
AuthorsOpenReview ↗

Feature-choice criterion, real-robot baselines, tuned ActMAD and Sec. 4 overview

We thank the reviewer for the careful review, and for finding the internal controls and the reported negative results valuable.

Unless stated otherwise: success rate (%), averaged over three tasks and drift levels L0–L3 (0–10 cm / 0–10°).

W1 & Q1: "The additional benefit of policy features over simple image matching is inconsistent …" / "Do existing results suggest a practical criterion for choosing feature matching over pixel correlation?"

Yes: choose the matching features by the type of image encoder. We agree that spatial registration is the main driver. The new results show which features to match:

ManiSkillNo adapt.ReView-pixPolicy's own featuresImageNet layer-1 features
DP (task-trained CNN)36.857.365.161.5
ACT (task-trained CNN)44.757.565.562.3
SmolVLA (SigLIP tower)22.143.337.447.3
π0.5\pi_{0.5} (SigLIP tower)47.148.855.256.8
  • Task-trained CNN encoder (DP, ACT): the policy's own early features work best, about 8 points above pixel correlation. A second training seed of both policies, evaluated at L2, shows the same: +11 and +17 points over pixels.
  • Generic ViT image tower (SmolVLA, π0.5): generic early convolutional features (ImageNet ResNet-18, layer 1) beat both pixels and the policy's patch embeddings. For SmolVLA, the gap to pixels grows at larger drift (L4/L5: 28.4 / 20.8 vs. 13.2 / 9.9).
  • robomimic: all spatial objectives are close (Table 1), because the scenes are textured enough for pixels to determine the warp.

We will add this criterion to Sec. 5.3.

W2 & Q2: "Are pixel-correlation or ECC results available for either real-robot task?"

Yes, we have now run both. We used the same displaced poses, the same reset images and 10 trials per pose. The table gives successes out of 30:

Real robot (DP)No adapt.ECCReView-pixReView
Close Box20242828
Pick Cube0559

On Close Box, pixel correlation matches ReView. On Pick Cube, which needs precise alignment between the gripper and the cube, ReView does better than both baselines. This is consistent with Sec. 4.2: policy features weight the gripper and the object, while pixels follow the textured table.

W3 & Q3: "What development-set evidence supports the ActMAD learning rate and single-update budget?"

In the submission we used ActMAD's published settings (App. A.6). To check whether this was unfair, we tuned ActMAD in its favor. We swept 24 configurations (optimizer × learning rate × number of updates; same layers and objective as in the paper) and selected the best one per task directly on the test cells. ActMAD uses the same 16 + 4 images as ReView.

DPNo adapt.ActMAD, published settingsActMAD, best of sweepReView
robomimic31.632.134.759.5
ManiSkill36.837.237.765.1

Matching feature statistics by updating the encoder does not express a geometric displacement, whereas a six-parameter warp does. We will add the sweep to App. A.6.

W4: "Section 4 introduces several local models and sensitivity matrices without clearly explaining their roles or connecting them to the actual calibration procedure."

We agree. Section 4 will open with the question it answers: why can a geometrically better warp give worse actions? An overview will follow:

PartExplainsWhere it shows up
4.1, Prop. 1How many reference images are needed, and why resets should resemble the demonstration statesChoice of 16 + 4 images (App. B.7); same-state references (App. B.6); the new check in our response to 6sbM
4.2, Eq. 6Which part of the scene one warp aligns under parallaxPixels follow the table, policy features follow the gripper (Sec. 5.4, Fig. 4)
4.3, Prop. 2Why exact image alignment need not minimize action errorReView above the table-plane oracle under translation; App. D.4

In response to Reviewer 6sbM we also evaluated π0.5\pi_{0.5}. On robomimic, ReView raises its success from 43.4 to 67.9.

We thank the reviewer again for the constructive review, and we are happy to discuss any remaining questions.

AuthorsOpenReview ↗

Kind Request for Feedback on Our Rebuttal

Dear Reviewer vH7Z,

Thank you again for your careful review and positive assessment. As the discussion period is drawing to a close, we wanted to kindly follow up on our rebuttal.

Following your suggestions, we added a criterion for choosing the matching features, real-robot comparisons with ECC and pixel correlation, and a tuned ActMAD baseline.

We would be grateful to know whether these additions address your concerns. If any questions remain, we would be happy to provide further clarification before the deadline.

Best regards, The Authors

I thank the authors for the additional results they provided. Despite small sample sizes in some comparisons, the results provide meaningful evidence that addresses most of my concerns.

However, the writing still needs substantial improvement, as the relatively simple idea and supporting analysis are not communicated clearly enough.

My current rating already reflects my support for the novelty and practical value of the work, so I will maintain my score.

Thank you for taking the time to consider our responses and additional results. We are glad that we could address most of your concerns, and we sincerely appreciate your continued support for our work. We will further improve the clarity of the writing in the final version.

Reviewer_JcZzOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper targets the setting where someone has bumped the camera and knows it. The goal is to bring the camera images back in distribution for the trained policy without retraining it. The proposed method takes the first frames of 16 training demonstrations and four unlabeled images of the scene at reset from the moved camera. It feeds both sets through the first few layers of the policy's own image encoder and averages the feature maps within each set. It then solves for a homography with six parameters (three for rotation and three for translation) that makes the feature map of the warped new images match the one from the training images. The warp makes the new images look as if they were taken from the training camera pose. During deployment, the same fixed warp is applied to every camera frame before the policy sees it. The paper shows that this improves success rates over the unadapted policy for Diffusion Policy, ACT, and SmolVLA on six simulated tasks, and for Diffusion Policy on two real-robot tasks.

Critical strengths
  1. The problem is very realistic. Cameras get bumped all the time.
  2. The method does not require retraining the policy, which is nice. Besides the training demos, it only uses four images from the moved camera, and it does not need camera poses or depth.
Critical weaknesses
  1. The method only handles what one homography can correct, and the main tests (drift up to 10 cm and 10°, little occlusion) are chosen so that this is enough. Gains are larger for pure rotation, where a homography is exact, than for pure translation (Table 16). Would it help with occlusions, or with 1 m or 50° azimuth changes?
  2. Prior works already map test-time images back to the training view. The two closest are not cited. Mirage (Chen et al., 2024) reprojects new images to the training camera pose for an unchanged policy, and MimicDreamer (Li et al., 2025) warps frames to a reference view with a homography.
Questions for authors

The policies are trained with only small random crops (76×76 from 84×84) and no rotation augmentation. Would training with larger random crops and random rotations make the policy robust to this drift without any test-time calibration?

4 comments
AuthorsOpenReview ↗

Translation, larger drift, occlusion, related work and augmentation

We thank the reviewer for recognizing that the problem is realistic and that the method needs no camera poses or depth, and for the concrete suggestions.

Unless stated otherwise: success rate (%), averaged over three tasks and drift levels L0–L3 (0–10 cm / 0–10°).

W1: "The method only handles what one homography can correct, and the main tests (drift up to 10 cm and 10°, little occlusion) are chosen so that this is enough. Gains are larger for pure rotation, where a homography is exact, than for pure translation (Table 16). Would it help with occlusions, or with 1 m or 50° azimuth changes?"

It helps under translation, larger drift and occlusion; it cannot handle 1 m or 50° changes. We agree that a homography is exact only for rotation or a planar scene, and that this defines the method's scope. We also agree that gains are larger under rotation. Below, we test each case the reviewer raises.

(a) Translation.

robomimic (Table 16, DP, L2): translation-only drift still helps Square (18.4 → 43.2) and Lift (80.8 → 93.6), but lowers Can (86.4 → 80.8). Translation barely hurts the unadapted Can policy (nominal success 89.0), so there is little to recover, and the residual parallax costs a few points.

ManiSkill (new, L2): here translation hurts the policy much more.

ManiSkill, L2No adapt.Table-plane oracleReView
DP, rotation only41.377.977.1
DP, translation only32.136.348.9
ACT, rotation only48.784.583.5
ACT, translation only28.832.148.8
  • As the reviewer expects, gains are larger under rotation (DP: +36 vs. +17 points).
  • Under translation, ReView still recovers 17–20 points, and stays above the oracle computed from the true pose. Aligning the table exactly leaves the gripper above it displaced (Sec. 4.2).

(b) Larger drift (new, 10–18 cm/°). This extends the robomimic L4/L5 results (Sec. 5.5, Table 6) to ManiSkill:

ManiSkill, L4 / L5No adapt.SIFTECCReView-pixReView
DP1.2 / 0.48.4 / 6.913.9 / 6.426.8 / 9.332.5 / 19.5
ACT2.4 / 1.18.9 / 6.311.5 / 5.318.0 / 5.726.9 / 14.5

At these levels, ECC and SIFT fail to register in 7–12 of 15 cells; failed cells fall back to the unadapted policy.

(c) Occlusion (new, ManiSkill). We placed a persistent occluder in both the calibration images and the rollouts. ReView raises DP from 29.8 to 49.3 and ACT from 36.2 to 55.4. ECC reaches 39.0 and 36.1.

(d) 1 m or 50° azimuth. Such changes alter which surfaces are visible, and no single image warp can undo that; it would need view synthesis with measured geometry. We will state this boundary explicitly in Sec. 5.5. ReView targets the common case the reviewer describes: a camera that was bumped or re-mounted.

W2: "Prior works already map test-time images back to the training view. The two closest are not cited. Mirage (Chen et al., 2024) … and MimicDreamer (Li et al., 2025) …"

Thank you for the pointers; we will cite and discuss both.

  • Mirage addresses cross-embodiment transfer. It renders ("cross-paints") the source robot over the target robot using URDFs, and can optionally reproject images to the source camera. That reprojection requires known camera transforms and depth.
  • MimicDreamer turns human videos into robot training data. Its EgoStabilizer warps egocentric frames to a reference view with a homography and inpaints the gaps.
  • ReView needs no depth or extrinsics. It fits the warp from unpaired images at test time, for a policy that is already deployed.

Q1: "Would training with larger random crops and random rotations make the policy robust to this drift without any test-time calibration?"

No. We retrained DP with moderate and strong crop + rotation augmentation. We used three training seeds on ManiSkill and two on robomimic:

NominalDrifted, no adapt.Drifted, ReView
ManiSkill, original85.736.865.1
ManiSkill, moderate aug.76.853.569.1
ManiSkill, strong aug.46.445.257.4
robomimic, original86.331.659.5
robomimic, moderate aug.82.843.466.5
robomimic, strong aug.67.551.158.6
  • Without adaptation, the augmented policies stay 8–20 points below the original policy with ReView.
  • Strong augmentation also costs nominal performance (85.7 → 46.4 on ManiSkill).
  • ReView still adds 7.5–23 points on top of every augmented policy, so the two are complementary. Augmentation, however, requires retraining.

We hope these results address the soundness concern, and we are happy to discuss any remaining questions.

AuthorsOpenReview ↗

Kind Request for Feedback on Our Rebuttal

Dear Reviewer JcZz,

Thank you again for your detailed review and concrete suggestions. As the discussion period is drawing to a close, we wanted to kindly follow up on our rebuttal.

Following your comments, we added results on translation, larger drift, occlusion, and stronger augmentation, and we now discuss Mirage and MimicDreamer.

We hope these results address your soundness concerns, and we would be grateful to hear your thoughts. If any concerns remain, we would be happy to provide further clarification or additional results before the deadline.

Best regards, The Authors

I thank the authors for the reply. It is good to see how the method performs under small versus large drift and under rotation versus translation, and that it handles the easy case of occlusion. However, I think occlusion in general is still not addressed.

In the rebuttal, the authors have now clarified the difference from previous works and addressed my augmentation question.

Overall, I think the method is not as general for larger drift. However, solving this problem for slight perturbations of the camera is also very useful, and the authors have clearly demonstrated where it works and where it does not. I will therefore raise my score.

Thank you for your thoughtful follow-up and for recognizing our efforts to address your concerns. We sincerely appreciate your willingness to raise the score, and we will clearly state the scope of the method regarding occlusion and larger drift in the revised version.

Reviewer_GFX3OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a deep learning based alternative to image rectification without explicitly inferring camera extrinsic. Given source images, it learns a inverse model that predicts a 6DoF 2D vector to warp uncalibrated images into its targeted source distribution.

Critical strengths

The paper is well presented. The idea is quite interesting.

Critical weaknesses

My biggest concern is that it's unclear to me what additional strength it brings to OTS methods, such as

  • Traditional image rectification -- calibration is not very labor intensive and is quite a standard approach
  • Feed-forward gaussian splatting methods?
Questions for authors

Please see my questions in the critical weakness sections. Happy to increase scores if authors convince me the above weaknesses ignore certain aspects of the paper. In addition,

  • how robust is the method to visual confounders? E.g. if I swap the background / the scenes rearrange a bit, will the model robustly rectify the images?
  • how robust is it to artifacts such as motion blur?

Overall, it was not clear to me the problem requires training a dedicated visual network to solve.

3 comments
AuthorsOpenReview ↗

ReView trains no network; comparison with rectification and robustness tests

We thank the reviewer for finding the paper well presented and the idea interesting. The main concern seems to stem from a misunderstanding of what ReView does, so we clarify this first.

Unless stated otherwise: success rate (%), averaged over three tasks and drift levels L0–L3 (0–10 cm / 0–10°).

Summary & Q3: "it learns a inverse model that predicts a 6DoF 2D vector …" / "it was not clear to me the problem requires training a dedicated visual network to solve."

ReView trains no network and has no inverse model. It solves a small optimization problem once, at deployment:

  • What is optimized: the six parameters of a plane-induced homography (Eq. 1), three for rotation and three for translation.
  • Objective: after warping, the mean early features of four unlabeled reset images should match those of 16 stored demonstration frames. The features come from the frozen policy encoder, which is never updated.
  • Cost: three fits take a median of 24 s on one RTX 4090 (App. A.7).

W1: "Traditional image rectification -- calibration is not very labor intensive and is quite a standard approach"

For DP and ACT, ReView outperforms rectification computed from the ground-truth camera pose, while measuring nothing.

Rectification needs the extrinsics of the displaced camera and a scene model. The paper already compares against an idealized version, the table-plane oracle, which uses the ground-truth pose and the true table plane (Tables 1, 4, 5). Averaged over all six tasks (Table 1), ReView is higher: 62.3 vs. 50.8 for DP and 73.1 vs. 64.9 for ACT.

Splitting the drift into rotation and translation shows why (ManiSkill DP, level L2):

Drift typeNo adapt.Table-plane oracleReView
Rotation only41.377.977.1
Translation only32.136.348.9
  • Rotation: the two match, because a homography is exact for pure rotation.
  • Translation: aligning the table exactly leaves the gripper and objects, which lie above the table, displaced (Sec. 4.2–4.3). ReView instead fits the warp that best restores what the policy uses.

In practice, re-calibrating after a bump means placing a calibration target in the workspace and re-estimating the extrinsics. ReView only needs four images that the robot already sees at reset.

W2: "Feed-forward gaussian splatting methods?"

These methods need the calibrated pose of the displaced camera; ReView does not (Introduction; Table 17):

Pose of moved cameraMetric depthView synthesis
AnyCamVLArequired–required
GS-VLArequiredrequiredrequired
ReView––– (one fixed image warp)

Q1: "how robust is the method to visual confounders? E.g. if I swap the background / the scenes rearrange a bit, will the model robustly rectify the images?"

ReView stays well above no adaptation in every case we tested, although success drops under large lighting or layout changes.

Condition (ManiSkill DP unless noted)No adapt.ReView
Unchanged36.865.1
Lighting changed (×0.5 / ×1 / ×2)37.2 / 33.2 / 27.958.1 / 46.0 / 48.5
Scene rearranged: objects and robot start pose shifted (×0.5 / ×1)*30.6 / 22.657.4 / 44.4
Persistent occluder29.849.3
Distractor in the calibration images only36.861.1
robomimic (Square, Can): green table and dim light13.833.4

*×1: objects and goal moved 3 cm in x and y; robot start pose offset by about 4 cm at the gripper.

  • Except for the distractor row, each change is present in both the calibration images and the rollouts.
  • The green-table setting is the closest we tested to a background swap.
  • Classical registration is less stable. ECC reaches 45.1 with the distractor and 39.0 with the occluder. At the nominal camera with the occluder, ReView's spurious warp stays at most 0.7 px, while ECC's reaches 3.2 px for DP and 7.7 px for ACT.

Q2: "how robust is it to artifacts such as motion blur?"

Calibration is essentially unaffected. We blurred the four calibration images, which are the only displaced-view images ReView uses for fitting. Success changed by −1.3 points on ManiSkill DP, and was unchanged on robomimic DP (Square and Can: 51.9 with blur vs. 51.3 without).

We hope these responses clarify the method. Please let us know if anything remains unclear; we are happy to provide further details or results.

AuthorsOpenReview ↗

Kind Request for Feedback on Our Rebuttal

Dear Reviewer GFX3,

Thank you again for your time and for finding the idea interesting. As the discussion period is drawing to a close, we wanted to kindly follow up on our rebuttal.

In our response, we clarified that ReView trains no network and only fits six warp parameters. We also added comparisons with ground-truth rectification and robustness tests under scene changes and motion blur.

You mentioned that you would be open to raising your score if these concerns were addressed, so we would be grateful to know whether our responses resolve them. If anything remains unclear, we would be happy to provide further clarification before the deadline.

Best regards, The Authors

Dear Reviewer GFX3,

We apologize for following up again. As the author-reviewer discussion ends in a few hours, we would greatly appreciate it if you could take a brief look at our response and let us know whether it addresses your concerns.

Thank you very much for your time.

Best regards, The Authors

Comments on the paper

1 comment

We sincerely thank all reviewers for their thoughtful evaluations and constructive feedback. We are encouraged that they found the problem realistic, the method lightweight, requiring no camera poses, depth, action labels, or policy updates, and the analysis of why image alignment and policy recovery can disagree informative.

We have carefully addressed the questions and concerns raised in the reviews with additional experiments and clarifications. These include results on π0.5\pi_{0.5}, a criterion for choosing the matching features, real-robot comparisons with ECC and pixel correlation, translation-only, severe-drift, occlusion, and scene-change tests, retraining with stronger augmentation, a tuned ActMAD baseline, and a rollout-free check for an unmoved camera. We also clarify that ReView trains no network and only fits six warp parameters, and we now discuss Mirage and MimicDreamer.

We have updated both the main paper and the appendix accordingly, with all revisions highlighted in red and the new experiments collected in Appendix G. We hope these results and clarifications address the reviewers’ concerns, and we would be happy to answer any further questions during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.