acceptodds
Under review as a conference paper at ICLR 2027

HIVE-Bench: Evaluating Patch-Level Visual Representations for Egocentric Robot Manipulation

28 views

Abstract

Pretrained visual encoders are widely reused in robot manipulation, yet most evaluations read them through a pooled embedding of third-person images, whereas bimanual policies consume dense patch tokens from onboard cameras. We introduce HIVE-Bench, to our knowledge the first benchmark that compares dense patch-level representations for egocentric bimanual manipulation under one shared policy. A flow-matching action head reads the patch tokens of 20 encoders on 24 RoboCasa-GR1 and RoboTwin 2.0 tasks, across frozen and fine-tuned encoders, training regimes, and data, model, and layer sweeps. We further relate more than fifty diagnostics and published vision-benchmark scores to closed-loop success. Three findings run against common practice. Averaging each view's patch tokens into a single token removes most of the success of frozen DINOv2 and DINOv3. Measures of world reconstruction, from segmentation, depth, and correspondence scores to probes that decode robot state or object position, either do not track success or lose this relationship once vision-language models are included. In contrast, an action readout that decodes commanded motion from two head-camera frames tracks success in every setting we test, with Spearman absolute rho of at least 0.54. Finally, better vision is not necessarily better control: larger models and newer pretraining improve vision benchmark scores without consistently improving manipulation success.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Manipulation, Visual Representation, Egocentric Vision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AhMqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper evaluates 20 pre-trained visual encoders across 24 dual-arm manipulation tasks within the RoboCasa-GR1 and RoboTwin 2.0 frameworks, employing a shared patch-conditioned policy architecture. It investigates the impacts of fine-tuning, task sharing, data volume, model scale, and specific feature layers. Furthermore, the study analyzes token statistics, probing for state and object localization, the use of Inverse Dynamics Models (IDM), and the correlation between standard visual benchmarks and closed-loop control performance, while also examining combinations of metrics and consistency across different experimental settings. The authors propose a two-stage selection process: first, using an IDM to filter out weaker candidates, followed by closed-loop experiments to finalize the encoder choice--emphasizing that standard visual benchmarks and model scale are not direct substitutes for control-based evaluation.

Critical strengths
  1. The paper features an extensive experimental scope, encompassing multiple encoder families, two dual-arm manipulation benchmarks, and a variety of training and adaptation configurations

  2. By utilizing a consistent patch-conditioned policy backbone, while keeping data, camera configurations, optimization protocols, and evaluation pipelines fixed across benchmarks, the paper enhances the comparability of different visual representations.

  3. The paper provides a comprehensive analysis of the experimental results, covering representation interfaces, fine-tuning, data and model scale, as well as the relationship between diagnostic metrics and success rates. This offers valuable insights into the applicability of various encoders.

Critical weaknesses
  1. The robotic control evaluations in this paper are derived exclusively from two simulation suites; consequently, the extent to which these findings generalize to the visual discrepancies encountered in real-world environments remains unverified. Real-world experiments are critical for robotic tasks. The authors should, at a minimum, incorporate open-loop probing evaluations using real-world robotic data, and, if feasible, include closed-loop results from physical robot experiments. Alternatively, drawing on Reference [1], the authors could leverage SimplerEnv and its corresponding real-world tasks to assess whether simulation-based metrics or rankings can effectively predict performance in real-world control scenarios.

[1] Capturing Visual Environment Structure Correlates with Control Performance (ICLR 2026) https://arxiv.org/abs/2602.04880

  1. The novelty of the diagnostic metrics proposed in this paper is somewhat limited: Reference [2] has already utilized IDMs to assess action recoverability, while Reference [1] has explored proxy metrics such as state decoding, segmentation, and depth estimation. Although the subjects of evaluation differ, the paper should more clearly articulate its specific, novel contributions to the field. While the paper emphasizes the effectiveness of IDM, there are counterexamples in the rankings, such as V-JEPA 2.1 and VGGT-Ω, where the method fails to align with performance. Consequently, beyond overall correlation, I would like to see a clearer delineation of the method’s capabilities: is IDM merely capable of filtering out weak candidates, or can it reliably identify the optimal encoder?

[2] Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models, https://arxiv.org/abs/2605.06388

  1. Although the paper provides extensive diagnostic analysis, the results still do not offer clear guidance on the actual selection of encoders. Existing experiments of leaving encoder regression and candidate screening support the value of the indicators for prediction and preliminary screening, but have not directly explained whether probe-assisted screening can reduce the cost of closed-loop evaluation while retaining high-performance candidates compared to not using these indicators. In particular, the authors also pointed out that IDM has limited sorting ability for strong candidates, and the final selection still depends on closed-loop experiments. Therefore, it is recommended to further clarify the method's ability boundary between excluding weak candidates and identifying strong candidates, and through the comparison of selection quality and evaluation cost, to more directly reflect its practical guiding value.
Questions for authors
  1. What is the specific novelty of this paper’s egocentric setting, will the analysis results change for non-egocentric settings?
  2. What is the rationale for introducing 32 additional "query tokens" into the action head? How do these differ from standard "action tokens," and what is the basis for choosing this specific number? While the appendix explains how these tokens participate in flow-matching velocity prediction via attention, I would like to see ablation studies, removing these query tokens or varying their count, to justify this design choice and assess its impact on encoder comparisons.
  3. Different encoders produce varying numbers of visual tokens (ranging from 64 to 576) and feature different widths. Simply standardizing the number of training steps and the policy backbone does not equate to matching the FLOPs in the cross-attention layers; furthermore, variations in context width alter the parameter counts of the projection layers and the action decoder. Could the authors report the downstream parameter counts and computational costs for each configuration?
1 comment

Thank you for asking what the benchmark adds and when its diagnostics help choose an encoder.

Weaknesses

W1. Do the findings extend to physical control?

We added physical closed-loop evaluation on an ARX A7 bimanual robot with an egocentric RealSense D405. Three encoders share multi-task training with 50 demonstrations per task and 100 trials per encoder/task: 900 trials (App. D).

Success (%)Drawer Cube PlacePick Two CubeGrab Stick
DINOv2423176
DINOv3322764
CLIP171123

DINOv2 leads throughout: 49.67% on average versus 41.00% and 17.00%, preserving the simulation ordering. The real-robot policies are trained with state input. The consistent ordering adds physical evidence for the encoder comparison.

W2. What is new beyond the two cited studies?

Thank you for pointing us to these studies. Related Work and Table 1 now credit their contributions and clarify how our evaluation extends this line of work:

  • Capturing Visual Environment Structure Correlates with Control Performance studies environment-state recovery, including geometry and object attributes, as a proxy for control. It also evaluates segmentation/depth and sim-to-real prediction. Its probes include dense features. We extend the evaluation to a controlled evaluation of patch-conditioned egocentric bimanual policies, across task sharing and adaptation, with several families of diagnostics.
  • Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models studies latents for action-conditioned world models. Its IDM measures action recoverability and degradation in generated latents; success is evaluated with fixed OpenVLA inside those world models. We train a common action head for each encoder and correlate diagnostics with simulator task success.

Our contribution is comprehensive evaluation of this policy setting and how diagnostic associations change with training. Your V-JEPA/VGGT counterexamples show why correlation need not identify the best encoder. On Twin, IDM–success correlation is −0.80 across frozen/adapted checkpoints, but −0.40 within adapted checkpoints. Improving on IDM across settings remains an open problem.

W3. Does screening beat random selection at the same budget?

We compare five lowest-IDM candidates with five random candidates and the full cohort. Random is the exact expected best over uniformly sampled subsets.

SettingFull bestRandom-5IDM-5
Casa ST43.1736.3232.61
Casa MT35.3932.1731.22
Twin MT53.6750.5253.22

On Twin, five instead of twenty policy runs retain success within 0.45 points of the best. On both Casa settings IDM loses to random; state and single-frame probes also lose on Casa ST. Even on Twin, IDM-3 gives 45.17%, below random-three's 47.33%. The practical benefit therefore depends on the setting and shortlist size. We report policy-run budgets here, with probe training as an additional cost (Sec. 5.4, App. E).

Questions

Q1. Why egocentric patches, and would other views change the answer?

Moving head/wrist views bring occlusion and fine local contact detail. Pooling patches lowers DINOv2/DINOv3 Casa success from 32.28/31.22% to 9.94/6.44%. Keeping spatial tokens therefore matters to the evaluation itself. This motivates evaluating the spatial interface increasingly used by egocentric policies. A matched camera-swap study would help determine which findings also hold under third-person views.

Q2. Why 32 queries rather than only action tokens?

The 16 action tokens represent the noisy action chunk and are decoded into velocity. The 32 queries provide extra learned attention positions without action targets; their count follows the common head. Removing them lowers Casa success from 32.28→27.78% for DINOv2 and 31.22→23.50% for DINOv3. The queries help both encoders and preserve their order (App. C.5). We retained 32 from the shared architecture; optimizing their count remains a separate design question.

Q3. Are downstream capacity and compute matched?

Appendix A reports all 20 encoders' counts. Thirteen share a 138.669M action head; the largest is 156.478M (12.8% more). Downstream totals add external projections and include the allocated state MLP even when unused; pretrained encoders are excluded. Table 3 reports per-encoder training costs: DINOv2 uses 54.84 TFLOPs/0.662 s per Casa MT step, VGGT 113.35/0.869, and VLMs about 195.35/1.12–1.14. Encoder size, token interfaces, and context-dependent projections all contribute to the observed compute differences. Equal training steps therefore do not imply equal computational cost.

Reviewer_ZiqKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces HIVE-Bench, a benchmark for comparing pretrained visual encoders in the way they are actually used by modern bimanual policies, namely through dense patch tokens from egocentric head and wrist cameras rather than through a single pooled embedding of a third-person view.

Twenty encoders from seven families are evaluated through one shared flow-matching action head on 12 RoboCasa-GR1 and 12 RoboTwin 2.0 tasks, across frozen and fine-tuned encoders, single-task and multi-task training, and sweeps over the number of demonstrations, model size, and VLM layer, amounting to more than 500 trained policies and 250,000 evaluation rollouts. The authors then relate more than fifty diagnostics and published vision-benchmark scores to closed-loop success.

The main findings are that averaging the patch tokens of each view into a single token removes most of the success of DINOv2 and DINOv3, that an inverse-dynamics readout probe is the only measurement that tracks success in every setting considered, whereas readouts of the robot state and object positions lose their association once the VLMs are included, and that gains in standard vision benchmarks from scale or newer pretraining do not translate into better control.

Critical strengths

The first critical strength is the scale and degree of control of the evaluation. By holding the action head and training recipe fixed while varying only the encoder and its token interface, the benchmark makes it possible to compare twenty encoders on equal terms across two suites and several training regimes, and the per-task results and code are released together with the paper. This is consequential for my assessment because such a controlled and reusable comparison is currently missing for patch-conditioned egocentric policies, and the resource is likely to be useful to the community regardless of how the individual findings hold up.

The second critical strength is the care taken in the statistical analysis and the informativeness of the negative results. The paper uses permutation tests with false-discovery control across diagnostics, bootstrap intervals, leave-one-family-out checks, and explicit evidence classes, and it openly discusses the dependence between encoders that share families and teachers. This is consequential because it lends credibility to the finding that commonly used proxies, such as segmentation, depth, and correspondence scores, do not track closed-loop success, which is a practically important message for researchers selecting encoders for manipulation.

Critical weaknesses

The first critical weakness is that the headline claim that reading the action is more predictive than reconstructing the world is not separated from the ability to decode the robot state. The target of the inverse-dynamics probe contains the current state, no policy receives proprioception, and among the vision encoders the inverse-dynamics and state probes are highly correlated and track success equally well. The two probes mainly diverge for the three VLMs, which share one base model and whose state probe reads a different view than their policy uses, and the probes also differ considerably in capacity. This is consequential because this contrast is the most novel message of the paper, and although the authors acknowledge the limitation, the abstract presents a stronger claim than the evidence supports.

The second critical weakness is that the variance across training runs is not measured, even though the paper draws conclusions about rankings. Each configuration is trained with a single seed and the reported uncertainty reflects only evaluation noise, yet the paper states that a suite score is uncertain by about one point. This is consequential because rankings are the primary output of a benchmark, and some of the reported conclusions, such as the reordering of mid-ranked encoders, involve differences that training variance alone could produce.

Questions for authors

Clarification requests

  1. Since patch tokens enter the action head without normalization and a single learning rate is used for all encoders, could the very low success of the RADIO models partly reflect differences in feature scale, and do the rankings change if the tokens are layer-normalized?

  2. In the matched-size comparison between DINOv3 and DINOv2, the two models produce different token grids at 224 pixels (196 versus 256 tokens per view), so how much of the gap on RoboTwin could be explained by the spatial resolution of the tokens rather than by pretraining?

Analyses that could change my assessment

  1. Does the inverse-dynamics probe still outperform the state probe among all 20 encoders when it is compared with its single-frame and repeated-frame variants, as the authors themselves propose, and when both probes have comparable capacity?

  2. How large is the variance across training seeds for a few representative encoders from the top, middle, and bottom of the ranking, and how many rank changes does it induce?

2 comments

Thank you for the careful assessment. We clarified the abstract and added controls for probe capacity, temporal input, and training variation.

Weaknesses

W1. Does “action over world reconstruction” reduce to state decoding?

We replaced that headline with the finding supported by the revised comparison: State recovery remains useful under the corrected, capacity-matched comparison. The more informative question is whether observing a transition adds anything beyond recognizing the current configuration.

The targets differ: state readout predicts sts_t; IDM predicts at+15−sta_{t+15}-s_t from two images. Imagine the same joint at 0.3 rad followed by a command of either 0.5 or 0.1 rad. The state answer is unchanged, but the relative-command answers are +0.2 and −0.2. Thus current-state knowledge alone does not determine the IDM target. Identifying configurations in both images can help recover this relative command, which also explains why IDM and state readout overlap.

The probes overlap: their frozen vision-encoder error ranks correlate at 0.80 on Casa MT and 0.82 on Twin MT. Matched-capacity state correlations with success remain −0.53/−0.50/−0.50 on Casa ST/Casa MT/Twin MT, including VLMs. We withdrew the earlier VLM-based argument. The three VLMs share a base model, and head-only probes see less than the multi-view Twin policy.

The results show a consistent pattern: IDM is informative on Twin, spatial-frequency statistics stand out on Casa, and state recovery matters in both. Fine-tuning further separates these measurements: VC-1's Twin success rises 18.94 points as IDM error falls 0.206→0.032, while state error changes only 0.104→0.102 (Sec. 5.1, Fig. 8).

W2. Are small rank differences larger than training noise?

We measured training variation by repeating DINOv2, VC-1, RADIO, and V-JEPA on Casa MT with seeds 1, 42, and 78. Their training SDs are 1.74, 1.10, 0.18, and 0.93 success points. DINOv2 varies most, from 28.22 to 32.28%. The same ordering holds in all three runs. These SDs give a more useful reference for close scores than rollout uncertainty alone; Appendix C.7 now reports the two sources of variation separately.

Questions

Q1. Could feature scale explain RADIO's low scores?

Your question led us to check the full input path. We found duplicate image normalization for SPA, RADIOv2.5, and C-RADIOv4, removed it, and replaced the affected successes and diagnostics. RADIO now reaches 24.39%/31.11% on Casa/Twin MT; C-RADIO reaches 34.78%/45.17%. These corrected results substantially improve their position in the comparison.

The correction concerns image preprocessing; feature LayerNorm remains untested. The shared recipe uses 10−410^{-4} for the action head and 10−510^{-5} for a fine-tuned encoder. We use these rates across encoders, so the rankings describe transfer under a common training recipe. Per-encoder learning-rate tuning remains unexplored.

Q2. Is DINOv3 disadvantaged by its coarser patch grid?

We tested your explanation by increasing DINOv3's input from 224 to 320 pixels: 196→400 patches, exceeding DINOv2's 256. Twin success improves from 30.44 to 34.61%, but remains below DINOv2's 49.39%. The gap shrinks by 4.17 points and still measures 14.78 points. On Casa, DINOv3 moves from 31.22 to 32.39%, essentially matching DINOv2's 32.28% (Fig. 6). The inputs are resized from stored 224-pixel images. Denser token sampling helps, although a substantial Twin gap remains.

Q3. Does IDM still outperform matched controls?

On Twin it has a stronger association; on Casa the readouts are close. All use width-256, four-layer transformers; the single-frame variant predicts the same target without the later image.

Error (Spearman ρ\rho, all 20 encoders)Casa MTTwin MT
Two-frame IDM−0.53−0.70
Single-frame IDM−0.53−0.56
State−0.50−0.50

Adding the later frame reduces error for every encoder, with median reductions of 40.7%/46.9%. After controlling for state-error ranks among vision encoders, IDM–success correlations are −0.28/−0.55. Lower target error and better policy ranking are therefore different outcomes: Casa improves on the former without much separation on the latter. Our control uses a single frame with the original target. Adding the later frame also increases token count; a repeated-frame comparison remains untested.

Q4. How many ranks change across training seeds?

None among the four tested encoders: every seed orders DINOv2, VC-1, RADIO, then V-JEPA. Appendix C.7 lists each run and mean/SD. The ordering of these four encoders is stable across the tested seeds.

Thank you for the response.

Additional experiments maek the results more reliable, and the claims now appear better calibrated. Implementation error was aslo identified and corrected.

However, the IDM readout largely overlabs with robot-state recovery, and its additional valeu appears limitied to RoboTwin.

I hope the abstract and introduction will reflect this more claerly.

I am keeping my rating.

Reviewer_25DdOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HIVE-Bench compares 20 pretrained visual encoders for egocentric bimanual manipulation. All of them feed the same GR00T-style DiT flow-matching head, which cross-attends to their raw patch tokens. The tasks are 12 RoboCasa-GR1 and 12 RoboTwin 2.0, and no policy gets proprioception. The study covers frozen and fine-tuned encoders, single- and multi-task training, and sweeps over demonstrations, model size, and VLM layer, totaling over 500 policies. It then relates more than 50 diagnostics and published vision benchmark scores to closed-loop success.

Critical strengths
  1. It’s a large, well-controlled comparison that the field doesn’t really have yet. There is one head and one recipe, 20 encoders, two bimanual suites, and both frozen and fine-tuned settings, all with patch tokens rather than a pooled vector.

  2. Several results are practically useful. These include the pooling control, the gap between vision benchmark gains and control (DINOv3 vs. DINOv2, C-RADIOv4), and the finding that frozen rank mostly survives fine-tuning. Together they argue against choosing encoders by leaderboard scores.

Critical weaknesses
  1. I’m not convinced the IDM probe is really reading motion. Its target includes s_t, the policies get no proprioception, and IDM and state errors correlate at 0.9+. So it may just be measuring how well the robot’s state can be read from the image. The VLM case that separates the two probes is confounded: the state probe only sees the head view and is much bigger at VLM width. There’s also no single-frame control.

  2. With one training seed per config and no normalization on the input tokens, I can’t tell how much of the ranking comes from the representation and how much from training noise or token-scale quirks. For example, RADIO sits near the bottom despite strong correspondence scores, which looks more like an interface issue to me.

Questions for authors
  1. Can you run the IDM probe with single-frame and repeated-frame inputs? This would show whether it adds anything beyond state.

  2. Can you match probe capacity (e.g. a state probe at width 256) and re-check the VLM outlier result?

  3. For a subset of encoders, does adding proprioception change the ranking or weaken the state/IDM association?

2 comments

Thank you for highlighting state visibility and training choices. We added controls to understand how each affects the probe results and encoder rankings.

Weaknesses

W1. Is IDM just another state probe?

The two probes overlap, but ask different questions. State readout predicts the current configuration, sts_t, from ItI_t. Our IDM reads (It,It+16)(I_t,I_{t+16}) and predicts at+15−sta_{t+15}-s_t, the terminal command relative to that configuration.

For a simple one-joint example, suppose the current angle is 0.3 rad. A later command of 0.5 gives a target of +0.2; a command of 0.1 gives −0.2. State readout has the same answer in both cases. It cannot, by itself, tell which target occurred. The later image can help distinguish them. The actual probes use standardized coordinates.

Current state is part of the IDM target, so the probes share information. Their distinction is current-state recovery versus relative-command recovery. Recognizing configurations across two frames may help recover the command; this explains their overlap while leaving room for different behavior.

The data show overlap without equivalence. Across frozen vision encoders, IDM/state error ranks correlate at 0.80 on Casa MT and 0.82 on Twin MT. Yet after Twin fine-tuning, VC-1's state error barely changes (0.104→0.102), while IDM error falls from 0.206 to 0.032 and success rises by 18.94 points. The different changes within each probe suggest that adaptation improves relative-command recovery more than current-state recovery in this case (Sec. 5.1).

W2. Could training noise or the interface drive the ranking?

Your RADIO example prompted an important correction. SPA, RADIOv2.5, and C-RADIOv4 received image normalization twice: in our wrapper and internally. We removed the extra step and replaced the affected results. RADIO now reaches 24.39%/31.11% on Casa/Twin MT; C-RADIO reaches 34.78%/45.17%. The corrected scores substantially change their position in the comparison. Feature LayerNorm remains a separate interface question.

We also repeated four Casa MT encoders with three training seeds. Training SDs are 1.74, 1.10, 0.18, and 0.93 success points for DINOv2, VC-1, RADIO, and V-JEPA, respectively. Their ordering is unchanged in all three runs. DINOv2 ranges from 28.22 to 32.28%, which gives a useful scale for interpreting close scores. Appendix C.7 reports training and evaluation variation separately.

Questions

Q1. What does the single-frame control show?

We held the target fixed and removed the later image. Restoring it lowers prediction error for all 20 encoders, by a median 40.7% on Casa and 46.9% on Twin. For context, success correlations are:

Error (Spearman ρ\rho)Casa MTTwin MT
Two-frame IDM−0.53−0.70
Single-frame IDM−0.53−0.56
State−0.50−0.50

Lower error is better, hence the negative signs. The two-frame input reduces prediction error in both suites, while improving the association with success mainly on Twin. Without a repeated-frame control, we cannot isolate the contribution of the later observation from the additional tokens. Among vision encoders, controlling for state-error ranks leaves IDM–success correlations of −0.28 and −0.55. The additional association is stronger on Twin.

The single-frame control retains the original action target.

Q2. Does matching capacity remove the VLM contrast?

Yes. With matched capacity, state readout also tracks success across VLMs. We projected features to width 256 and used the same four-layer transformer width/depth as IDM. State remains associated with success when VLMs are included (table above), so we removed the claim that its association disappears. Both probes use the head view; Twin policies additionally use wrist views.

Q3. Does giving the policy state change the result?

This tests whether our image-only setup makes state visibility unusually important. Adding a state token raises DINOv2/RADIO success from 32.28/24.39 to 35.25/26.17% on Casa, and from 49.39/31.11 to 49.78/33.06% on Twin. Both benefit and their order stays the same (Fig. 6). For this pair, the encoder difference persists with proprioception. Measuring how the overall correlations change would require extending this comparison across encoders.

Thank you author for the rebuttal. The single-frame IDM control and the correction to RADIO preprocessing help address my concerns about probe interpretation and encoder ranking. The rebuttal has clarified several of my concerns, and I will maintain my original rating.

Comments on the paper

1 comment

Shared updates and corrections

We thank all three reviewers for their careful reading and insightful questions. Your comments helped us identify an implementation error and improve the fairness of our diagnostic comparisons. We summarize the shared updates below; the individual responses address each reviewer's questions in detail.

  • Corrected encoder preprocessing. SPA, RADIOv2.5, and C-RADIOv4 received an extra image-normalization step before entering the encoder, in addition to their internal normalization. This affected their representations and downstream results. We removed the extra step, reran the affected experiments, and replaced the corresponding success rates and diagnostics throughout the paper. We apologize for this error and the misleading comparisons it caused.

  • Matched IDM and state-probe capacity. Both probes now project encoder features to 256 dimensions and use four-layer, width-256 transformers and width-256 readout heads. The transformer capacity therefore no longer grows with the encoder's native feature width. We reran the state measurements and updated the comparisons, including those involving VLMs.

  • Updated data and conclusions. All reported success rates and diagnostic measurements now use the corrected results, and we recomputed their correlations and revised the figures, tables, and conclusions accordingly. Missing results remain excluded. In particular, state readout remains informative when VLMs are included; we have removed the earlier claim that its association disappears and narrowed the interpretation of IDM to reflect its overlap with state recovery.

  • Added evidence requested by the reviews. The revision includes single-frame IDM controls, multiple training seeds, proprioception and image-resolution ablations, query-token removal, and closed-loop real-robot evaluation. We also report complete downstream parameter counts and compare probe-based shortlists against random selection in all three settings. No tested readout reliably selects the best encoder across settings. Our contribution is the comprehensive evaluation of patch-level representations for egocentric bimanual control and their diagnostic relationships; finding metrics that improve on IDM remains an open direction.

We are grateful for the reviewers' observations: they led to substantive corrections and a clearer account of what the benchmark supports.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.