- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
INVAR learns visual invariance from paired clean and randomized simulation renders. Its frozen encoder enables policies trained on nominal simulation to transfer zero-shot to real robots, substantially improving dexterous and bimanual manipulation success over pretrained-encoder baselines.
Critical strengths
- Reusable visual robustness. Learning invariance during encoder pretraining allows downstream policies to share the same robust representation without visual randomization during policy training. This addresses a practical bottleneck in sim-to-real learning.
- Substantial real-world gains. Success improves from 0.35 to 0.62 on dexterous grasping and from 0.32 to 0.65 on bimanual tasks. Evaluation across two platforms and held-out objects and task concepts provides persuasive evidence of practical value.
Critical weaknesses
All downstream evaluations use task-specific BC policies, but demonstration counts, state-distribution coverage, and policy validation procedures are insufficiently documented. The “unseen” objects and tasks are held out from encoder pretraining, while their downstream policies still receive task-specific demonstrations. Consequently, the experiments do not sufficiently establish whether the gains reflect broadly transferable visual invariance or improved fitting within restricted demonstration distributions. Performance across demonstration budgets, controlled shifts in initial states, and independent training seeds would materially strengthen this assessment.
The symmetric-DR baseline uses crops from one randomized render, whereas INVAR aligns different appearances of the same state. Thus, the comparison supports paired appearance alignment but does not establish that a clean teacher is necessary.
The statistical support and generalization scope is limited. Non-grasping main evaluations use 24 real-world trials per method, while ablations report only 12 rollouts, without uncertainty estimates or clear variability across independently trained models. This weakens conclusions based on smaller differences. Both evaluated embodiments also appear in encoder pretraining, leaving transfer to unseen embodiments untested.
Questions for authors
- How many independently trained encoders and policies contribute to each real-world result?
- How does INVAR compare with teacher and student views independently randomized from the same simulator state, using matched augmentations and compute? This would distinguish the contribution of paired appearance alignment from that of a clean teacher and directly test the central claim.
4 comments
Transfer to unseen embodiments
Both evaluated embodiments also appear in encoder pretraining, leaving transfer to unseen embodiments untested.
INVAR transfers to unseen embodiments without additional encoder pretraining or fine-tuning, suggesting that it learns a visual invariance prior that generalizes beyond its pretraining embodiments. We support this with three complementary evaluations: (1) visual sim-to-real transfer on new embodiment; (2) leave-one-out generalization during encoder pre-training; and (3) encoder generalization to new domain-randomized sim embodiment zero-shot. In all settings, the encoder remains frozen, and only a new action head is trained for the target task (Sec. 4.3, Tables 2–3).
1. Visual sim-to-real transfer on new embodiment: KUKA–SHARPA stirring (Sec. 4.3; Table 2). We evaluate the task “pick up the spoon and stir clockwise” with a KUKA iiwa arm (unseen during pretraining) and SHARPA hand (setup shown in Table 2). INVAR and both baselines use the same inputs and identical policy architecture. INVAR outperforms both baselines in simulation and the real world (64 simulation and 25 real-world rollouts per method).
| Encoder / training | Real-world SR | Nominal Sim SR | Sim SR w/ visual DR |
|---|---|---|---|
| INVAR (ours, frozen) | 52% | 89.1% | 90.6% |
| DINOv3 (end-to-end) | 28% | 76.6% | 82.8% |
| CAIP (frozen) | 24% | 54.7% | 51.6% |
2. Leave-one-out generalization during encoder pre-training (dexhand → parallel gripper, Sec. 4.3; Table 3). We use a frozen INVAR encoder pretrained exclusively on dexterous hand data, then test on the bimanual parallel-gripper task “put bottle in bin” on YAM arms. The experiment shows that our learned encoder INVAR transfers to a morphology unseen in pretraining: the dexterous-hand-only encoder outperforms all baselines in the real world despite never seeing parallel-gripper data during encoder pretraining.
On real-camera parallel-gripper observations, the dexhand-only INVAR encoder attends more to task-relevant bottles, bins, and robot arms, and less to backgrounds than DINOv3 and CAIP (Appendix C.3, Fig. 11).
| Encoder | Policy training | Real-world SR | Nominal Sim SR | Sim SR w/ visual DR |
|---|---|---|---|---|
| INVAR (full pretraining, frozen) | Clean | 83% | 94% | 54% |
| INVAR (dexhand-only, frozen) | Clean | \underline{\mathit{67\\%}} | 90% | \underline{\mathit{52\\%}} |
| DINOv3 (frozen) | Clean | 21% | 60% | 38% |
| DINOv3 (fine-tuned) | DR | 42% | 86% | 44% |
| DINOv3 (frozen) | DR | 33% | 80% | 34% |
| CAIP (frozen) | Clean | 33% | 90% | 52% |
| CAIP (frozen) | DR | 46% | \underline{92\\%} | 54% |
Simulation success is measured over 50 episodes per method (nominal or visual DR), and real-world success over 24 rollouts per method.
3. Zero-shot encoder transfer to an unseen simulated embodiment under visual DR (Sec. 4.3; Table 2). On RoboMimic/robosuite Square and Stack, both the Franka arm and third-person camera viewpoint are unseen during pretraining. With frozen encoders and matched action-head training and evaluation settings, INVAR outperforms DINOv3 and CAIP in simulation with visual DR on both tasks (100 episodes per method).
| Frozen encoder | Square | Stack | ||
|---|---|---|---|---|
| Sim SR w/ visual DR | Nominal Sim SR | Sim SR w/ visual DR | Nominal Sim SR | |
| INVAR (ours) | 76% | 85% | 77% | 83% |
| DINOv3 | 70% | 83% | 71% | 81% |
| CAIP | 39% | 72% | 49% | 85% |
4. Qualitative cross-embodiment evidence (Sec. 4.4; Appendix C.3, Fig. 10). Across unseen human egocentric views (EgoDex), dexterous hands (Wuji and Allegro), and robot-arm configurations (Franka and KUKA–SHARPA), INVAR shows more object- and interaction-focused attention than DINOv3 and CAIP on identical observations. Wuji cart-pushing and bed-making examples from WB-WAM extend this qualitative analysis beyond tabletop manipulation to humanoid locomanipulation.
5. Scalability of embodiment pretraining. INVAR is designed to scale efficiently to a broad range of robot embodiments for two reasons:
- Generalization beyond pretraining embodiments: Our results demonstrate that INVAR transfers to unseen robot configurations without additional paired renders or encoder retraining. Moreover, most manipulation robots fall into a manageable set of embodiment families, such as single-arm or bimanual systems with dexterous hands or parallel grippers, suggesting that a sufficiently diverse pretrained encoder could support many downstream robots.
- Efficient expansion of pretraining coverage: Even when additional embodiments need to be included in pretraining, all encoder pretraining data are generated in simulation, allowing us to expand the dataset at low cost without collecting real-world data. Visual robustness can then be learned once and reused across downstream tasks and embodiments, rather than requiring task-specific domain randomization and representation learning for every policy.
Demonstration coverage and controlled variation
The “unseen” objects and tasks are held out from encoder pretraining, while their downstream policies still receive task-specific demonstrations. Consequently, the experiments do not sufficiently establish whether the gains reflect broadly transferable visual invariance or improved fitting within restricted demonstration distributions.
Our goal is to evaluate the sim-to-real generalization of a reusable visual encoder, rather than zero-shot transfer of an action policy across tasks. INVAR is pretrained once and then frozen and reused across all downstream tasks and embodiments. Each task has its own behavior-cloning action head, trained using task-specific demonstrations on nominal simulation observations. Importantly, the evaluation objects and tasks are held out from encoder pretraining. Thus, our central contribution is that a single task-agnostic encoder provides transferable visual invariance across previously unseen downstream settings, without requiring task-specific visual domain randomization or encoder retraining. We have added clean-versus-randomized feature visualizations to directly illustrate the learned visual invariance (Sec. 4.4; Appendix C.4, Fig. 12).
All downstream evaluations use task-specific BC policies, but demonstration counts, state-distribution coverage, and policy validation procedures are insufficiently documented. Performance across demonstration budgets, controlled shifts in initial states, and independent training seeds would materially strengthen this assessment.
Demonstration counts, state-distribution coverage, and policy validation: We report the demonstration counts and initial-state distributions below and in the revised paper (Appendix B.2, Tables 10–11).
Simulation demonstrations per downstream task.
| Embodiment | Task/object | Policy training | Held-out validation |
|---|---|---|---|
| Dexhand | Spray bottle | 1,600 | 400 |
| Dexhand | Wine glass | 1,300 | 320 |
| Dexhand | Watering can | 230 | 55 |
| Dexhand | Tape | 1,100 | 270 |
| Dexhand | Hammer | 1,600 | 400 |
| Dexhand | Wine pouring | 400 | 100 |
| Parallel gripper | Put bottles in bin | 1,700 | 190 |
| Parallel gripper | Flip mug | 1,800 | 200 |
We retain only successful simulation rollouts during demonstration generation. Consequently, demonstration counts vary across tasks due to differences in data-collection success rates.
Initial-state distribution coverage.
| Task/object | Position variation | Orientation and additional variation |
|---|---|---|
| Spray bottle, tape | ±5 cm in X/Y | ±20° yaw |
| Wine glass, watering can | ±4 cm in X/Y | ±10° yaw |
| Hammer | ±5 cm in X/Y | −10° to +30° yaw |
| Wine pouring | Bottle: 5 × 5 cm; bowl: 3 × 5 cm | ±10° yaw for both |
| Put bottles in bin | Object X: 0.319–0.881 m; Y: −0.632–0.633 m | 19 object geometries; 0.9–1.1× object scale; 360° object yaw. Bin X: 0.57–0.73 m, Y: ±0.25 m; ±43° yaw; 1.0–1.4× scale |
| Flip mug | Object X: 0.469–0.783 m; Y: −0.343–0.347 m | 18 object geometries; 0.9–1.1× object scale; 360° object yaw subject to valid placement. Tray X: 0.55–0.70 m, Y: ±0.30 m; ±14° yaw |
Our initial-state randomization follows established practices in robot manipulation benchmarks. Our ±4–5 cm reset ranges are comparable to standard benchmarks, such as RoboMimic Lift (±3 cm; Mandlekar et al., 2022) and a representative LIBERO-Spatial task (±1–7 cm; Liu et al., 2023), while our parallel-gripper tasks cover larger position ranges, object geometries, and orientations.
Overall, our evaluation spans substantially different demonstration budgets, object geometries, initial-state distributions, and manipulation tasks. INVAR consistently achieves strong performance across these diverse settings, providing evidence that its learned representations are effective beyond a single narrowly constrained demonstration distribution.
Matched comparisons isolate the encoder’s contribution: For each task, INVAR and all baselines use identical demonstrations, initial-state distributions, object variations, policy architectures, and training hyperparameters. These controls attribute the relative performance gains to the visual representations.
Dear Reviewer,
Thank you again for your time and thoughtful feedback. Please let us know if you have any further questions or concerns; we would be very happy to address them.
We thank the reviewer for their thoughtful feedback. We are glad that the reviewer recognizes the practical value of reusable visual robustness and the strong real-world results. We address their questions and concerns below.
We pretrain one INVAR vision encoder, which is frozen and reused across all tasks and embodiments. For each downstream task, only a new task-specific behavior-cloning action head is trained on top of the frozen encoder. Unlike approaches requiring task-specific encoder fine-tuning or domain randomization, INVAR learns transferable visual invariance once, achieving strong performance across diverse held-out objects and tasks without retraining the encoder.
Clean teacher and matched randomized-view ablation
Our ablation suggests that the main benefit comes from learning invariance across paired views, while a canonical teacher target provides additional gains. We ran an ablation where the teacher and student receive independently randomized renders of the same simulator state, while keeping the pretraining data, augmentations, and optimization budget identical (Sec. 4.5; Table 7).
This variant outperforms the best baseline in simulation with visual DR on both dexterous-hand tasks and in the real world on the ABC put-bottle-in-bin task, but remains below INVAR (canonical teacher) in simulation with visual DR on all three tasks and in the real-world bottle task (Table 7). These results suggest that aligning independently randomized views can improve visual invariance, but aligning randomized observations with a clean teacher provides additional benefits, supporting our asymmetric design choice.
Why do we need a canonical teacher?
1) Align with downstream inputs: Since downstream policies using INVAR train only on clean simulation images, clean teacher targets encourage randomized observations to map to the same representations used during policy training.
2) Provide a consistent target: The teacher is updated only through EMA, without direct gradients. Clean inputs keep its visual target consistent, encouraging invariance around the nominal representations used by downstream policies.
Statistical evidence and independent model counts
Physical trial counts per object/task: For each method, our main real-world experiments use 25 rollouts per dexterous-hand task/object (five unseen grasping objects and one unseen pouring task, so 150 rollouts in total) and 24 rollouts per bimanual parallel-gripper task. Our additional real-world ablation studies use 12 rollouts per evaluated condition. These trial counts are consistent with the evaluation scale commonly adopted in foundational robotics research, where hardware constraints and evaluation costs often limit real-world trials (e.g. ACT (Zhao et al., 2023; 25 trials), Diffusion Policy (Chi et al., 2023; 20 trials), OpenVLA (Kim et al., 2024; 5–10 trials), π0.5 (Black et al., 2025; 10 trials), RobustVLA (2026; 10 trials)).
Uncertainty for the main and ablation results: We report ±1 SE (one standard error) to quantify variability (Figure 4, Tables 4–6). For the embodiment-diversity ablation (Table 4), we ran three random seeds and found that the embodiment-diversity effect is consistent across training runs. For our main real-world experiments, INVAR consistently achieves approximately 2× the success rate of the strongest baseline across different tasks and embodiments, demonstrating substantial performance improvements rather than marginal gains.