acceptodds
Under review as a conference paper at ICLR 2027

Compact Robot Policies Need Fine-Grained Visual Representations

Abstract

Multi-task manipulation policies differ in architecture, scale, and pretrained priors all at once, so published comparisons cannot attribute performance to any single component. We argue that most of it comes from the visual representation, and that parameter scale and generative priors are largely incidental. To test this, we build CoRP (Compressed Representation Policy), a deliberately compact policy (48.9M parameters, no vision-language model and no video-generative prior) that factorizes into a representation extractor and a flow-matching action generator. It reaches 97.0% on LIBERO and 75.78%/73.36% on RoboTwin 2.0 Clean/Randomized, matching systems 40.9–163.6× larger. Holding the action generator fixed, we then vary one extractor property at a time. Pretrained initialization is decisive: a random ViT-S/14 drops to 78.1% and an ImageNet ResNet-34 to 74.5% on LIBERO. Pretraining alone is not enough, as freezing the encoder costs 19.8 points. Compression matters as much: resampling each view to 48 tokens beats passing all patch tokens (97.0% vs 83.2%), and a variational information bottleneck over those tokens is worse than a hard token budget, cutting LIBERO-Goal from 95.8% to 33.0% by suppressing the instruction-dependent token selection the policy relies on. Language conditioning contributes only where the observation leaves the goal ambiguous (LIBERO-Goal: 9.2% to 95.8%), while on RoboTwin 2.0, where observations are unambiguous, removing it slightly improves success. Therefore, we argue that a compact policy works when its representation is pretrained, task-adapted, and compressed. Project page: https://corp-policy.github.io

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, Imitation Learning, Flow Matching, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ty4mOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper argues that the performance of multi-task manipulation policies depends more on visual representations than on model size or generative priors. To investigate this, the authors introduce CoRP, a compact policy consisting of a representation extractor (for visual observation, proprioceptive state, and language instruction) and a lightweight flow-matching action generator. This is trained with a flow-matching loss on action chunks and an auxiliary task-classification loss. Keeping the action generator architecture fixed, they study several factors affecting policy performance on LIBERO, including pretrained versus randomly initialized encoders, encoder architecture, frozen versus fine-tuned encoders, language conditioning, and token compression. The paper evaluates the proposed method on LIBERO and RoboTwin 2.0. The results suggest that compact policies can perform well when they use pretrained, task-adapted, and compressed visual representations, while language conditioning helps primarily when the goal is ambiguous from the scene alone.

Critical strengths
  1. Clean experimental design: CoRP has few interacting components, and the ablations in Sections 4.3-4.6 keep the action generator fixed while varying one property of the representation extractor at a time. The action probe (NMAE, R^2) and task-head accuracy provide complementary diagnostics beyond success rate. The gap between task identification and task execution is also a useful observation: for example, the pretrained ResNet achieves 100% task-head accuracy but only 37.0% success on Goal (Table 3).

  2. A compact policy that matches much larger VLA systems on closed-set benchmarks: The 48.9M-parameter CoRP reaches 97.0% on LIBERO, on par with pi-0.5 (3.3B) and OpenVLA-OFT (7B), and 75.8%/73.4% on RoboTwin 2.0, above X-VLA and pi-0.5 (Tables 1-2), while running at about 7ms per chunk. The design consists entirely of standard components, which makes it a cheap and reproducible testbed and a practical reference point for small policies.

Critical weaknesses
  1. The main findings are largely known, and the closest prior study is not cited: The three properties emphasized in this paper have already been established in prior work: pretrained visual features benefit manipulation (e.g., R3M, VC-1, and the studies cited by the authors), fine-tuning outperforms freezing, and Perceiver-style resamplers can improve performance. In particular, RoboVLMs [1] already reports that vision-language pretraining is essential, that frozen pretrained encoders perform substantially worse than fine-tuned ones, and that adding a Perceiver resampler to reduce the number of visual tokens yields substantial gains. These findings are demonstrated across eight backbones, four formulations, and real-robot generalization settings. The current manuscript does not cite RoboVLMs, and its conclusion that "parameter scale and generative priors are largely incidental" runs counter to RoboVLMs’ conclusions.

  2. The main claim is not tested in the regime where it matters: The abstract argues that performance depends primarily on visual representations, while model scale and generative priors play a minor role. However, all quantitative evaluations are conducted on closed-set, in-distribution tasks, and the authors acknowledge in Sec. 5 that they do not evaluate the open-world semantic and physical generalization that VLAs and WAMs are designed to support. RoboVLMs, by contrast, finds that vision-language pretraining is particularly important for such generalization. RoboVLM, by contrast, finds that vision-language pretraining is particularly important for such generalization. Moreover, Table 2 shows that CoRP remains about 13 points below the video-pretrained WAMs on RoboTwin 2.0, and that removing video pretraining from Motus costs 10-11 points. This directly suggests that generative priors do matter, yet the paper does not explain how this finding aligns with its central claim. The claim should therefore be scoped to VLM priors on closed-set benchmarks, where it is much weaker.

  3. The conclusion that "pretraining helps" (Sec. 4.3) is not supported by the results: Pretraining improves performance with the ViT by 18.9 points but reduces performance with the ResNet (74.5% with pretrained initialization vs. 82.1% with random initialization, Table 3).The evidence therefore supports the narrower conclusion that DINOv2 initialization benefits the ViT, rather than the broader claim in the abstract that "pretrained initialization is decisive". The experimental design cannot disentangle the effects of pretraining type (self-supervised vs. supervised) from those of architecture (ViT vs. ResNet), and the two architectures also provide different numbers of tokens to the resampler. Although the authors acknowledge the missing supervised-ViT control, they still draw a general conclusion about the benefits of pretraining.

Minor Weaknesses:

  • The task-classification loss is introduced but not ablated: Sec. 3.2 introduces this loss to preserve task-relevant information. However, the paper’s own results suggest that accurate task identification does not necessarily imply that the representation retains task-relevant details (Tables 3 and 7), yet no result with λ=0\lambda = 0 is reported. Moreover, this loss is the only non-standard component of the method, but it is excluded from the ablation study.
  • Benefits of pretraining are measured only at full data and a fixed training budget: The paper does not test data efficiency (reduced demonstrations) or convergence speed, which are the main practical reasons to prefer a pretrained encoder. Without this, the 18.9-point gap between pretrained and random ViT (Table 3) could partly reflect under-training of the random variant under the shared training schedule rather than a representational advantage.
  • Inconsistent bolding in tables: Multiple entries in the same column are bolded, some of the best results are not highlighted (e.g., GigaWorld-Policy in Table 2), and the "Ours" are bolded even when they do not achieve the best results. Please bold only the best result in each column or clearly state the bolding convention.

[1] Li, Xinghang, et al. "What matters in building vision–language–action models for generalist robots." Nature Machine Intelligence 8.2 (2026): 158-172.

Questions for authors

Please address the weaknesses.

3 comments

Thank you for your careful reading and constructive feedback. Please see our responses below; changes in the revised PDF are marked in blue.

(W1): Thank you for highlighting the connection to prior work. We agree that the importance of vision–language pretraining for robot-policy generalization has already been established and that our claims are overstrong.

However, our goal is to split the contribution of vision and language to the policy. First, when tasks can be distinguished from visual observations, language provides limited additional benefit. In our RoboTwin setting, removing language does not substantially change success (77.28%77.28\% vs. 74.57%74.57\%). Second, when visual observations alone are insufficient to identify the task, a small language encoder provides effective task-conditioning information within the evaluated closed task set. This does not imply that it matches a large VLM in open-vocabulary understanding, compositional generalization, or reasoning.

We also agree that our original statement that performance depends primarily on visual representations, with model scale and generative priors playing a limited role, was too broad. We will therefore narrow our conclusion: Under the benchmarks and training conditions studied, strong pretrained visual representations enable a compact policy to achieve competitive performance at relatively low cost. We hope this conclusion does not mislead readers to think that larger models or stronger generative priors are unnecessary.

(W2): We added LIBERO-Plus OOD experiments. Full results with per-perturbation breakdowns are in Table 12 (page 21, Appendix A.3).

ModelLIBERO-Plus (%)
OpenVLA15.6
OpenVLA-OFT69.6
OpenVLA-OFT$_w$55.8
NORA39.0
WorldVLA25.0
UniVLA43.9
π0\pi_053.6
π0\pi_0-Fast61.6
RIPT-VLA68.4
OpenVLA-OFT$_m$67.9
CoRP (ours)51.54±2.5851.54 \pm 2.58

This evaluation extends the evidence beyond the original in-distribution experiments. We will limit the resulting robustness claims to the perturbations and evaluation protocol covered by LIBERO-Plus.

Training with the official LIBERO-Plus training data. Separately, our policy achieves 72.62% success after training with the official LIBERO-Plus training data.

(W3): We added supervised-pretrained DeiT-III ViT-S baselines of similar size to DINOv2 ViT-S/14. The results are presented in Table 3 (page 7, Section 4.3) of the revised manuscript, and reproduced below for convenience.

EncoderPretrainingLIBERO (%)
DINOv2 ViT-S/14Self-supervised96.45±0.4896.45 \pm 0.48
DeiT-III ViT-SSupervised, IN-22K93.85±1.5193.85 \pm 1.51
DeiT-III ViT-SSupervised, IN-1K90.62±1.6890.62 \pm 1.68

We conclude that pretrained visual representations benefit both DINOv2 and DeiT-III, though not ResNet-34. DINOv2 achieves the highest success among all encoders tested. Due to differences in pretraining data, recipe, and patch size, this comparison does not isolate the effect of supervised vs. self-supervised pretraining, which is outside our scope.

We also address the token-count discrepancy: under a fair 49-token comparison, DINOv2 achieves 97.15%97.15\% and ResNet-34 achieves 74.5%74.5\%, confirming the gap is not explained by the extra patch token.

(Q1): Lambda sweep with/without language across three seeds, in Table 8 (page 18, Appendix A.3).

λ\lambdaWith language (%)Without language (%)Gain (pts)
093.62±0.3393.62 \pm 0.3369.17±0.3369.17 \pm 0.3324.5
0.00194.05——
0.01 (default)96.45±0.4896.45 \pm 0.4869.32±0.6069.32 \pm 0.6027.1
0.191.30——
189.20——

A small weight (λ=0.01\lambda=0.01) gives a modest gain over λ=0\lambda=0 (+2.8+2.8 points with language), and larger weights degrade success monotonically, so the loss is a small auxiliary regularizer.

(Q2): Data efficiency results are in Table 10 (page 21, Appendix A.3). The pretrained encoder improves substantially with more data (62.37%→96.45%62.37\%\to96.45\%), while the random-initialized encoder plateaus near 77%77\%.

Encoder initData (%)LIBERO (%)
Pretrained2562.37±0.5662.37 \pm 0.56
Random2557.18±1.3057.18 \pm 1.30
Pretrained5088.45±4.8588.45 \pm 4.85
Random5075.26±0.7975.26 \pm 0.79
Pretrained10096.45±0.4896.45 \pm 0.48
Random10077.40±0.6877.40 \pm 0.68

The random-initialized encoder needs 2× more steps and still underperforms at matched loss (72.67%72.67\% vs. 96.45%96.45\%). See Figure 10 (page 23, Appendix A.3) for convergence curves.

(Q3): The bolding error in Table 2 has been corrected; all tables bold only the best result within each track.

Thank you again for your time and thoughtful review. We hope our narrowed claims, new experiments, and corrections address your concerns. If so, we would be grateful if you would consider revising your score.

Reviewer_ty4mOpenReview ↗

Official comment by Reviewer

Thank you for your response. After reading the response and the updated manuscript, I still have concerns about the position of the paper’s claims. In particular, the abstract and conclusion has not been sufficiently narrowed. The new LIBERO-Plus results suggest that the compact policy’s competitiveness with larger systems is limited to in-distribution settings, which further reinforces my concern. Therefore, I will maintain my initial score.

Thank you for the follow-up. We agree, and we have made the following concrete revisions to the manuscript.

Abstract revision. We replaced "enable compact policies to match approaches using large VLMs or generative priors at lower cost" with "enable compact policies to approach the performance of much larger systems on closed-set manipulation benchmarks, without a VLM or video-generative prior." This removes the unqualified "match" claim.

Conclusion revision. We replaced "CoRP is competitive with systems up to 163× larger" with "CoRP substantially narrows the gap to systems up to 163× larger on the closed-set benchmarks studied." We also added an explicit sentence acknowledging that under distribution shift (LIBERO-Plus), a gap to the strongest systems remains.

On the LIBERO-Plus results. We agree they reinforce the concern: CoRP reaches 51.54%51.54\% while OpenVLA-OFT reaches 69.6%69.6\% — an 18-point gap that was not visible in-distribution. We do not claim OOD parity. What the results do show is that representation design still matters under distribution shift: CoRP at 51.5%51.5\% outperforms OpenVLA (15.6%15.6\%), NORA (39.0%39.0\%), WorldVLA (25.0%25.0\%), and UniVLA (43.9%43.9\%), and approaches π0\pi_0 (53.6%53.6\%), which benefits from large-scale robot pretraining. The gap to OpenVLA-OFT (69.6%69.6\%) and RIPT-VLA (68.4%68.4\%) is real, and we now state it explicitly.

Revised claim scope. Our contribution is to show which representation design choices support strong performance under a compact model budget, and to quantify the effect of each choice. The paper does not claim that scale and generative priors are unnecessary in general — only that, on the closed-set benchmarks studied, they are not strictly required to narrow the gap substantially.

We hope these targeted revisions address your concern about the positioning of the claims.

Reviewer_KQZYOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CoRP, a 48.9M-parameter multi-task manipulation policy. It combines a fine-tuned DINOv2 ViT-S/14, a T5-Efficient-Tiny language encoder, a FiLM-conditioned learned-query cross-attention resampler (48 tokens per view), and a single-block DiT action head trained with flow matching plus an auxiliary task-classification loss.

CoRP reaches 97.0% on LIBERO and 75.8%/73.4How does CoRP perform on Libero-Plus compared to the other baselines? How much variance is there across training seeds for the default model and the main ablations? What is the effect of the task-classification loss (λ = 0), with and without language? How does CoRP compare to ProgVLA under matched training conditions?% on RoboTwin 2.0 Clean/Randomized. The authors use it as a testbed to ablate four properties of the representation: encoder initialization, encoder fine-tuning, language conditioning, and token compression. They conclude that compact policies work when their visual representation is pretrained, task-adapted, and compressed, and that scale and generative priors are largely incidental. A small real-robot study on three bimanual tasks is included.

Critical strengths

At under 50M parameters, CoRP matches π0.5 and OpenVLA-OFT on LIBERO. On RoboTwin 2.0 it outperforms X-VLA (0.9B) and π0/π0.5 (3.3B), and approaches Motus without video pretraining at roughly 160× fewer parameters. Training takes about 8–13 hours on two GPUs and inference about 7 ms. If code is released as promised, this is a valuable, accessible baseline that raises the bar for papers arguing that large VLM backbones are necessary.

Fixing the action generator and varying one property of the representation extractor at a time is the right way to approach the question. The 2×2 designs (initialization × architecture, resampler × VIB) are more informative than standard leave-one-out tables.

The task-head accuracy, action probe, Δ_prefix, and t-SNE analyses expose a recurring dissociation between task identification and execution. The pretrained ResNet identifies tasks perfectly but reaches only 37% on LIBERO-Goal, and All Tokens + VIB reaches 99.9% accuracy but 33.6% success. This is the most interesting observation in the paper.

Critical weaknesses

Framing

The paper does not decide whether it is a method paper or an analysis paper, and neither framing is adequately supported. The introduction states that the benchmark result “is our starting point, not our contribution.” Yet the first listed contribution, Figure 1(b), and Section 4.7 present CoRP as a competitive new policy “competitive with systems up to 163x larger.”:

  • As a method paper, it lacks multi-seed results, robustness or generalization evaluation, like-for-like baselines (all baseline numbers are copied from prior work with different inputs and training regimes), and a comparison to its closest relative, ProgVLA, on RoboTwin. The comparison also omits efficient VLAs that pursue the same goal, most notably FLOWER (Reuss et al., CoRL 2025), a sub-1B VLA that retains a pruned VLM backbone. Including it would show whether CoRP’s efficiency advantage comes from removing the VLM or simply from avoiding unnecessary scale.

  • As an analysis paper, the headline claim cannot follow from the experimental design. The claim is that most performance comes from the visual representation while parameter scale and generative priors are largely incidental. But the action generator, scale, and priors are never varied systematically. Holding the action generator fixed shows that representation choices matter for this generator, not that they matter more than the factors the abstract dismisses. The paper’s own capacity results (93.9% at d=180 vs. 98.4% at L=3) suggest capacity is not incidental even within this family. All ablations also use a single architecture on a single benchmark, so it is unclear whether the insights transfer to other policy classes.

I would encourage the authors to commit to one framing and provide the evidence that framing requires.

Only In-Distribution Evaluation

Evaluation is entirely in-distribution, so overfitting cannot be ruled out. The authors acknowledge that LIBERO evaluation states follow the demonstration distribution and that RoboTwin Randomized is trained on 25,000 randomized demonstrations. A 49M-parameter model trained on all data without a validation split is exactly the setting where memorization is a concern, and the paper’s own Figure 6 hints at it: the no-language and All Tokens variants reach lower training loss but much lower success. Without the instruction, LIBERO-Goal actions should be irreducibly ambiguous, so a lower flow-matching loss suggests the model is exploiting episode-specific visual cues. Evaluating on a perturbation benchmark such as LIBERO-Plus is needed. It introduces controlled perturbations across seven dimensions, including camera viewpoints, robot initial states, language instructions, lighting, background textures, and sensor noise, and would directly test both robustness and the paper’s language claims. CALVIN ABC→D, which evaluates transfer to an unseen environment with long-horizon language-conditioned task chains, would be a complementary test. It is also a standard benchmark for efficient policies (e.g., FLOWER).

Statistical Rigor All results come from a single training seed with no confidence intervals, and some conclusions rest on small differences (e.g., removing language improves RoboTwin by about 2.7 points).

Limited novelty of the insights. Because CoRP itself is disclaimed as a contribution, the paper’s value rests on its insights, several of which are largely known:

  • Language matters only when the scene is goal-ambiguous; LIBERO-Goal was designed to test exactly this, and LIBERO-Plus found that models are largely insensitive to language variations and tend to ignore language instructions completely. The aux loss optimizing for task identification via the visual tokens further confound the stated experimental results.
  • Fine-tuned pretrained encoders outperform frozen ones.
  • Pretrained encoders outperform training from scratch. The architecture is also close to ProgVLA (separate pretrained encoders, Perceiver-style resampling, compact flow-matching head, auxiliary heads).

W6. The real-robot study is inconclusive. It covers three tasks without baselines and follows proprietary large-scale pretraining. As the authors note, it does not isolate the representation, so it does not support the analysis claims.

Questions for authors
  1. How does CoRP perform on Libero-Plus compared to the other baselines?
  2. How much variance is there across training seeds for the default model and the main ablations?
  3. What is the effect of the task-classification loss (λ = 0), with and without language?
  4. How does CoRP compare to ProgVLA under matched training conditions?
3 comments

Thank you for your careful reading and constructive feedback. Please see our responses to your comments, and changes in the revised PDF are marked in blue.

(W1): Thank you for pointing out the mismatch between our positioning and the supporting evidence. We agree that the current version emphasizes an analytical goal and that the paper is a controlled empirical study. We highlight CoRP's performance to show that, based on our analysis, it is possible to construct a compact policy with powerful capabilities, which supports our analysis.

We agree that current experiments do not support a relative ranking of "visual representation" against model scale or generative priors. What we really want to argue is that we do not need to incorporate expensive VLMs and generative priors into systems to reach competitive results, though this does not mean VLMs and generative priors are worse.

Although our paper is an analysis paper, we also agree that we should compare with other efficient VLAs with less than 1B parameters. We already included results for X-VLA (0.9B) and find its performance (72.88%/72.84%72.88\%/72.84\% on RoboTwin 2.0 and 98.1%98.1\% on LIBERO) is similar to CoRP, while CoRP uses only 48.9M parameters.

(Q1): Thank you for this suggestion. To keep the scope of the original study manageable, we evaluated only in-distribution performance. Following your advice, we added two experiments on LIBERO-Plus to demonstrate CoRP's performance in out-of-distribution scenarios. The full results with per-perturbation breakdowns are presented in Table 12 (page 21, Appendix A.3) of the revised manuscript, and a summary is reproduced below for convenience.

Trained on original LIBERO only, evaluated on LIBERO-Plus.

ModelSuccess rate (%)
OpenVLA15.6
OpenVLA-OFT69.6
OpenVLA-OFT$_w$ (no wrist camera)55.8
NORA39.0
WorldVLA25.0
UniVLA42.9
π0\pi_053.6
π0\pi_0-Fast61.6
RIPT-VLA68.4
OpenVLA-OFT$_m$67.9
CoRP (ours)51.54±2.5851.54 \pm 2.58

CoRP outperforms OpenVLA, NORA, WorldVLA, and UniVLA, and is close to π0\pi_0 (53.6%), although it remains below OpenVLA-OFT, RIPT-VLA, and π0\pi_0-Fast. Notably, π0\pi_0 benefits from large-scale robot-data pretraining, whereas CoRP is trained only on LIBERO demonstrations. We do not claim state-of-the-art robustness. However, we think this result indicates that the policy does not merely memorize episode-specific visual cues, since it retains non-trivial performance under camera, lighting, background, language, and other perturbations.

(b) Trained with LIBERO-Plus data. When we additionally train on the official LIBERO-Plus training data, CoRP's success rate further improves to 72.62%. Note that this setting is no longer zero-shot with respect to the perturbations.

(Q2): Thank you for raising this statistical rigor concern. We provide multi-seed results for the main ablations to address the reviewers' questions.

(Q3): Thank you for this question. To disentangle the effect of the auxiliary task-classification loss from the effects of language and visual representation, we ran an additional ablation with λ=0\lambda{=}0, both with and without language instructions. The full results with three training seeds are presented in Table 8 (page 18, Appendix A.3) of the revised manuscript, and reproduced below for convenience.

Table 8 (reproduced): Auxiliary-loss weight sweep on LIBERO.

λ\lambdaWith language (%)Without language (%)Language gain (pts)
093.62±0.3393.62 \pm 0.3369.17±0.3369.17 \pm 0.3324.5
0.01 (paper default)96.45±0.4896.45 \pm 0.4869.32±0.6069.32 \pm 0.6027.1

(Q4): Thank you for this question. We consider ProgVLA a methods paper while CoRP is an analysis paper: we conduct extensive comparative experiments to identify what leads to CoRP's success, which can serve as a design principle for others.

Since ProgVLA is not open-sourced, we reimplemented it from the description in the original paper and aligned its training conditions with those of CoRP. Under these matched conditions, the results are:

MethodSuccess rate (%)
CoRP (ours)96.45±0.4896.45 \pm 0.48
ProgVLA (our reimplementation)84.07±0.7384.07 \pm 0.73

(W6): The real-robot policy follows more than 10,000 hours of pretraining and has no matched baseline, since pretraining a zero-initialized large model is too expensive, while incorporating large models pretrained on other datasets is also incomparable. Moreover, our goal is to show that we do not need to incorporate VLM and generative priors to manage these tasks, so we present this as a qualitative feasibility study.

Thank you again for your time and thoughtful review. We hope our revised positioning, new OOD experiments, and additional ablations address your concerns. If so, we would be grateful if you would consider revising your score.

Thank you for the response. I still find the paper would benefit from restructuring and rework to clearly position it. also the additional baselines like X-VLA and Flower have apparently not been used for the additional ablations, which I find confusing since they are the close to the approach and given their size the necessary compute would be less than openVLA or the other models that have been compared against in the additional results.

For these reasons I will stick to my intitle score.

Thank you for the continued engagement and comments.

On positioning and framing. We have revised the abstract and introduction to more clearly frame CoRP as a controlled empirical study. Our goal is to identify which representation design choices — encoder initialization, fine-tuning, token compression, and language conditioning — matter most for compact manipulation policies, and to measure how much each one contributes. We have also toned down the competitiveness claims in the abstract and conclusion, limiting them to the closed-set benchmarks we actually evaluated.

On X-VLA and FLOWER as ablation baselines. We appreciate this suggestion. We want to clarify the distinction between ablation baselines and performance comparisons. Our ablations (Tables 3–7) vary one property of the representation extractor at a time — encoder initialization, fine-tuning, token budget, and language conditioning — while keeping the action generator fixed. X-VLA and FLOWER are independent systems with different architectures, training data, and action generators; varying their representation properties would require reimplementing them from scratch and would not isolate the same design choices our study targets. They therefore serve as performance reference points, not ablation controls.

We agree they are the most relevant compact-policy comparisons. We have added X-VLA (fine-tuned on LIBERO only), FLOWER, and SmolVLA to the LIBERO-Plus table in the revised manuscript. The full results are shown below.

MethodParamsCameraRobotLanguageLightBackgroundNoiseLayoutTotal
OpenVLA7.0B0.83.523.08.134.815.228.515.6
OpenVLA-OFT7.0B56.431.979.588.793.375.874.269.6
OpenVLA-OFT$_w$7.0B10.438.770.576.893.649.969.955.8
NORA3.0B2.237.065.145.758.612.862.139.0
WorldVLA7.0B0.127.941.643.717.110.938.025.0
UniVLA7.0B1.846.269.669.081.021.231.942.9
π0\pi_03.3B13.86.058.885.081.479.068.953.6
π0\pi_0-Fast3.0B65.121.661.073.273.274.468.861.6
RIPT-VLA7.0B55.231.277.688.491.673.574.268.4
OpenVLA-OFT$_m$7.0B55.621.781.092.791.078.668.767.9
X-VLA15.9B69.157.273.174.747.254.155.961.6
FLOWER1.0B34.943.842.455.349.455.846.046.8
SmolVLA0.45B2.64.58.326.420.64.019.311.2
CoRP (ours)0.049B45.0±2.345.0 \pm 2.333.6±3.633.6 \pm 3.671.8±2.571.8 \pm 2.570.6±8.670.6 \pm 8.637.2±4.237.2 \pm 4.235.2±5.835.2 \pm 5.869.2±3.669.2 \pm 3.651.5±2.5\mathbf{51.5 \pm 2.5}

Among compact models (under 1B parameters), CoRP (51.5%51.5\%) outperforms FLOWER (46.8%46.8\%) and SmolVLA (11.2%11.2\%) despite using 20×20\times and 9×9\times fewer parameters respectively. Compared with X-VLA, CoRP leads on Language (71.871.8 vs. 73.173.1 — roughly comparable), Layout (69.269.2 vs. 55.955.9), and Camera (45.045.0 vs. 69.169.1 — X-VLA leads here); X-VLA leads on Robot, Background, and Noise. The overall gap (51.5%51.5\% vs. 61.6%61.6\%) reflects a 325×325\times difference in parameter count (0.0490.049B vs. 15.915.9B).

We hope these additions address your concern about compact-model comparisons. Thank you again for your time and thoughtful review. If these clarifications resolve your concerns, we would be grateful if you would consider revising your score.

Reviewer_ZQijOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies where the performance of multi-task robot policies comes from. It proposes CoRP, a 48.9M-parameter policy that separates a visual/language representation extractor from a flow-matching action generator. CoRP uses a small DINOv2 ViT-S/14 encoder, an instruction-conditioned learned-query resampler, and a one-block DiT action head. The authors keep the action generator fixed and vary representation initialization, visual fine-tuning, language conditioning, and visual token compression. CoRP reaches 97.0% average success on LIBERO and 75.8%/73.4% on RoboTwin 2.0 Clean/Randomized. The ablations suggest that pretrained and task-adapted visual features are important, language is mainly useful when the scene is goal-ambiguous, and a small instruction-aware token budget is better than passing all patch tokens. Small real-robot experiments are also reported.

Critical strengths
  1. Useful controlled decomposition. By fixing the action generator and varying the representation extractor, the paper makes a meaningful attempt to separate representation quality from model scale and generative priors. The large gaps between pretrained/fine-tuned and frozen or random encoders support the practical value of this analysis.
  2. Clear and relevant ablations. The language and token-budget experiments are well motivated. The LIBERO-Goal result and the resampled versus all-token comparison provide a concrete explanation for when language and compression matter.
Critical weaknesses
  1. Some central controls are still missing. The encoder study lacks a supervised-pretrained ViT, so the gain cannot be tied specifically to DINOv2 pretraining. The task-classification loss is not ablated.
  2. The cross-system attribution is stronger than the evidence. The authors clearly limit the study to closed-set tasks, which is appropriate. Still, changing the representation within CoRP does not establish that scale and generative priors are incidental in other systems. Tables 1–2 use baseline scores from prior reports. The real-robot results follow large proprietary pretraining and have no matched baseline or representation ablation.
Questions for authors
  1. How much of the reported performance comes from the auxiliary task-classification loss? Please report lambda=0 and a small lambda sweep.
  2. What is the sensitivity to the VIB coefficient, and can the VIB comparison control for stochasticity and sequence/compute changes?
  3. Can you clarify the latency measurements? Section 4.7 reports about 7 ms per chunk, while Appendix A.5 reports about 20 ms for real deployment. What hardware and operations does each figure include?
1 comment

Thank you for your constructive feedback and for recognizing the value of the controlled decomposition. Please see our responses to your comments; changes in the revised PDF (which we resubmitted) are marked in blue.

(W1): We added a DeiT-III ViT-S control (ImageNet-1K and ImageNet-22K), of similar size to DINOv2 ViT-S/14, with the same action generator, data, and recipe. The results are presented in Table 3 (page 7, Section 4.3) of the revised manuscript, and reproduced below for convenience.

Table 3 (reproduced): Encoder initialization control.

Encoder initializationPretrainingLIBERO success (%)
DINOv2 ViT-S/14self-supervised96.45±0.4896.45 \pm 0.48
DeiT-III ViT-Ssupervised, IN-22K93.85±1.5193.85 \pm 1.51
DeiT-III ViT-Ssupervised, IN-1K90.62±1.6890.62 \pm 1.68

Supervised ViT pretraining also gives a large gain over random initialization, so the benefit is not specific to DINOv2, although DINOv2 is the strongest encoder we tested. Due to differences in pretraining data, training recipe, and patch size, this comparison does not isolate whether the supervised or self-supervised encoder is better. However, since our goal is not to understand how to pretrain the visual encoder for manipulation tasks, we do not dig further.

(W1 & Q1): We ran the sweep with and without language. The full results with three training seeds are presented in Table 8 (page 18, Appendix A.3) of the revised manuscript, and reproduced below for convenience.

Table 8 (reproduced): Auxiliary-loss weight sweep on LIBERO.

λ\lambdaWith language (%)Without language (%)Language gain (pts)
093.62±0.3393.62 \pm 0.3369.17±0.3369.17 \pm 0.3324.5
0.00194.05——
0.01 (paper default)96.45±0.4896.45 \pm 0.4869.32±0.6069.32 \pm 0.6027.1
0.191.30——
189.20——

A small weight (λ=0.01\lambda=0.01) gives a modest gain over λ=0\lambda=0 (+2.8+2.8 points with language), and larger weights degrade success monotonically, so the loss is a small auxiliary regularizer.

(W2): We agree that varying the representation inside CoRP cannot show that scale or generative priors are incidental in other systems, and we apologize for overclaiming this result. Our narrower claim is that, on the closed-set benchmarks we tested, a 48.9M-parameter policy with pretrained, task-adapted encoders is competitive with much larger systems. This suggests, but does not establish, that a strong pretrained representation may reduce the scale and generative priors needed to approach similar performance.

As for real-robot policy that has followed more than 10,000 hours of pre-training, we do not consider that the pretrained prior is useless, as our results also support that pretrained visual representation works better. What we really want to argue is that we do not need to incorporate expensive VLM and generative priors into systems to reach still competitive results, and our whole paper showed that it is possible.

(Q2): We swept β\beta at a fixed 48 tokens per view with the same sequence length and update budget, using three training seeds. The results are presented in Table 9 (page 21, Appendix A.3) of the revised manuscript, and reproduced below for convenience.

Table 9 (reproduced): VIB coefficient sweep (supplementary protocol, mean ±\pm sample SD over three seeds).

ConfigurationStochastic samplingβ\betaLIBERO success (%)
No VIBno—96.45±0.4896.45 \pm 0.48
VIByes095.73±0.6895.73 \pm 0.68
VIByes4×10−94\times10^{-9}94.22±1.4194.22 \pm 1.41
VIByes4×10−84\times10^{-8}81.57±6.1081.57 \pm 6.10
VIByes4×10−74\times10^{-7}37.85±3.8537.85 \pm 3.85

Stochastic sampling and the extra projection alone do not reproduce the drop; success falls monotonically as β\beta grows, so the KL constraint is what degrades the model. Budgets are close but not exactly matched, since the projection heads within VIB add parameters and compute. What actually degrades the model's capabilities is the KL regularization constraint we identified in our paper; as its weight increases, the model's performance progressively decreases.

(Q3): You have pointed out a significant inaccuracy in our phrasing. We are measuring end-to-end latency. In Section 4.7, since both the simulation environment and the inference service are deployed on the same local machine, the inference process requires only 7ms. However, in a real-world deployment, the latency increases because the process involves transmitting data to an inference server over the local area network (5ms), performing the inference (7ms), and returning the resulting command (8ms).

Thank you again for your time and thoughtful review. We hope our additional experiments and clarifications address your concerns. If our responses resolve your concerns, we would be grateful if you would reconsider your rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final author remark

Dear all reviewers and AC:

We sincerely thank the AC and all three reviewers for their careful and constructive evaluation. We summarize the contributions of the work and how each concern was addressed during the rebuttal.

Summary of contributions.

  1. Controlled decomposition — fixing the action generator and varying one representation property at a time, we measure the effect of encoder initialization, fine-tuning, token compression, and language conditioning, with multi-seed results for statistical reliability.
  2. Out-of-distribution evaluation — we evaluate CoRP on LIBERO-Plus, a seven-dimensional perturbation benchmark, showing that CoRP (51.5%51.5\%) outperforms all models under 1B parameters, including FLOWER (46.8%46.8\%) and SmolVLA (11.2%11.2\%).
  3. Narrowed and precise claims — we revised the abstract and conclusion to scope all competitiveness claims to the closed-set benchmarks studied and to explicitly acknowledge the gap to the strongest systems under distribution shift.

Strengths recognized by the reviewers.

  • Reviewer ZQij highlighted the useful controlled decomposition and clear and relevant ablations, noting that the gaps between pretrained/fine-tuned and frozen/random encoders "support the practical value of this analysis."
  • Reviewer KQZY praised the clean experimental design with 2×22 \times 2 factorial ablations, and called the dissociation between task identification and execution — where pretrained ResNet achieves 100%100\% task-head accuracy yet only 37%37\% goal success — "the most interesting observation in the paper."
  • Reviewer ty4m recognized the clean experimental design and acknowledged that CoRP matches much larger VLA systems on closed-set benchmarks at ~7 ms per chunk, calling it "a cheap and reproducible testbed and a practical reference point for small policies."

Below we list the concerns raised and their resolutions.

ConcernReviewerStatus and Response
Supervised-ViT encoder control missingZQijResolved. DeiT-III ViT-S baselines added in Table 3 (p. 7).
Cross-system attribution stronger than evidenceZQij, ty4mResolved. Claims scoped to closed-set benchmarks; OOD gap acknowledged.
λ\lambda sweep and VIB sensitivityZQijResolved. Three-seed sweeps in Tables 8 and 9 (pp. 18, 21).
Latency phrasing inconsistencyZQijResolved. Sec. 4.7 clarified: 7 ms local, 20 ms real-world end-to-end.
Framing unclear (method vs. analysis)KQZYResolved. Repositioned as controlled empirical study; introduction rewritten.
Only in-distribution evaluationKQZY, ty4mResolved. LIBERO-Plus OOD results in Table 12 (p. 21).
Multi-seed statistical rigorKQZYResolved. Three-seed results for all main ablations (Tables 2, 8–10).
X-VLA and FLOWER missingKQZYAddressed. Both added to Table 12 as reference points.
Prior work (RoboVLMs) understatedty4mAddressed. Connection acknowledged; claims narrowed.
Pretraining conclusion overly broadty4mResolved. DeiT-III baselines, data efficiency, and convergence added.
Abstract and conclusion overclaimty4mResolved. Both revised and scoped to closed-set benchmarks.

We have provided clarifications, new experiments, and targeted manuscript revisions for all concerns raised during review. Reviewer ZQij supports acceptance. We believe CoRP provides a useful and reproducible testbed for studying compact manipulation policies, and that our refined claims accurately reflect what the experiments support. We hope the AC will find that the rebuttal substantively addressed the reviewers' concerns.

Sincerely, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.