acceptodds
Withdrawn from ICLR 2027

When Does Sim-and-Real Co-Training Make Simulation a Reliable Evaluator?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Evaluating robot policies in the real world is expensive. Simulation offers a scalable alternative, but its value depends on reliably predicting real-world performance. A common belief is that sim-and-real co-training can improve this predictiveness, but when and why remain unclear. We identify co-training's two opposing effects on sim-and-real agreement: sim-transferring carries real-learned behaviors into simulation, whereas sim-overwriting lets simulated data determine simulated behavior regardless of the policies' real-domain behavioral differences. We use controlled synthetic experiments to isolate how gap type, simulation-data dose, and policy-set construction govern these effects. We then test the predictions with manipulation policies across behavioral and visual sim-to-sim gaps and with real-robot evaluation. For behaviorally diverse policies, small simulation-data doses can improve behavior-wise agreement through sim-transferring. Larger doses can hurt agreement through sim-overwriting on states covered by simulated data, while continuing to improve agreement on Sim-OOD states, where direct sim-overwriting is avoided. Step-wise correlation between simulated and target-domain scores generally improves with sim-and-real co-training, but this does not necessarily imply behavioral alignment. These results provide practical guidance for reliably designing and interpreting simulation-based policy evaluation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, real-to-sim evaluation, co-training, vision-language-action models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_BcNFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper examines sim-and-real co-training and its effects on making simulation an effective policy evaluator.

One contribution of the paper is the distinction b/w sim-transferring and sim-overwriting. The definition of these terms, however, is not clear. As best I can tell, they are effects rather than causes, i.e., they describe whether it helps or hurts to add sim data. In fact, the abstract (L015) calls them "effects". Clearly in some cases it will help, in others it will hurt. But I don't see any way to independently distinguish between these two other than to run the policy and see if it's better or worse.

Section 4 describes an experiment involves an R^3 to R^2 mapping, which allows a controlled study. The conclusion at the end of this section is that the simulated corpus should remain small, because too much sim data leads to sim-overwriting. But how small is small? And how much is too much?

Section 5 extends this analysis to sim-to-sim and real-to-sim evaluations of a VLA.

Critical strengths
  • Sim and real are important topics in robotics.
  • If we could pre-identify how much sim data to mix with real data in co-training, it would be useful.
Critical weaknesses
  • The paper is difficult to follow.
  • Two primary terms do not appear to be well-defined causal explanations, but rather are effects.
  • No specific guidance appears from the experiments to help future researchers determine how much sim data is needed in co-training.
Questions for authors
  • L020 "For behaviorally diverse policies, small simulation-data doses can improve behavior-wise agreement through sim-transferring. Larger doses can hurt agreement through sim-overwriting on states covered by simulated data, while continuing to improve agreement on Sim-OOD states, where direct sim-overwriting is avoided." These are the type of sentences that I find difficult to parse. What is "behaviorally diverse"? More specifically, are you saying that small sim-data doses would not help on a behaviorally non-diverse policy? Please explain how the statement to improve "through sim-transferring" avoids circular reasoning (since sim-transferring is the term used to describe improvement). Same for "sim-overwriting".
  • L074 "evaluation-oriented use of co-training" - what does this mean?
  • L132 Add the term "success rate" here for clarity (to explain the acronym SR)
  • L086 "sim-overwriting lets shared simulated data determine each policy’s simulated behavior independent of their different real data, making their differences unobservable from simulated rollouts." Please explain.
  • Is the point of Figure 1 that sometimes sim helps and sometimes it hurts?
  • Please explain Figure 2.
  • L291: What are "CIs"?
  • What is the conclusion of Figure 4? And how is "alignable" defined?
  • L477. "When using simulated rollouts to interpret policy behavior or diagnose differences in DR, researchers should verify that the rollouts preserve the relevant behavioral differences." How do you propose researchers do this?
  • What is the conclusion of Table 1?
Reviewer_fEesOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates when sim-and-real co-training improves the reliability of simulation-based policy evaluation. It identifies two mechanism that co-training can affect the sim-and-real agreement. Through synthetic experiments and controlled robot manipulation experiments (both sim-to-sim and sim-to-real), the authors find that

  1. When comparing checkpoints at different training steps, high sim-real correlation might arise because the performance are increasing in both domains during training.
  2. When comparing policies with different behaviors, co-train on shared simulation data can hide those differences.
  3. Using relatively little simulated data or a held-out task can help improve reliability.
Critical strengths
  1. The studied question is very important for recent Real2Sim and simulation-based evaluation works.
  2. The experiments are well-designed in general. The mechanisms behind the question turn out to be interesting and offer a useful perspective for understanding how co-training affects evaluation reliability.
Critical weaknesses
  1. My main concern is the setting of the co-training process:
  • In PolaRis, different checkpoints trained on real data (like pi0_droid_jointpos and pi0_droid_jointpos_100k) are co-finetuned on simulation data for the same steps, and then compared.
  • In this paper, the same pretrained policy is co-finetuned on simulation data (for different steps if in step-wise experiments) to generate two different checkpoints.

I think the setting in PolaRis is more appropriate, since we want to use co-training to adapt existing checkpoints to the simulator for evaluation, rather than creating different checkpoints via the co-training process. For the step-wise experiments, this might be problematic. If the PolaRis setting is used, then all checkpoints are co-finetuned on the cross task data for a fixed steps, so the same-task success rates may not increase with the success rates of the held-out task. Therefore, the conclusion that "an increase in step-wise correlation through co-training does not require behavior alignment" may depend on the particular protocol used in this paper. I expect this concern to have less impact on the behavior-wise experiments. I encourage the authors to evaluate at least a subset of their experiments using the PolaRiS-style protocol and examine whether the conclusions still hold. I will increase my score if this concern is addressed.

  1. The question is studied through highly controlled experiments. The robot experiments include only two relatively simple tasks. It's unclear whether the findings still hold in larger scale evaluation pipelines as well. For example, does the alignability discussion still hold? In the paper, different bottle geometries, mass are treated as an alignable gap, while the unalignable gap instead uses two different tasks. It's unclear how this connects to more realistic settings.
Questions for authors
  1. What specific guidance do the authors want to provide to practitioners using simulation evaluators based on co-training? Should they always use small simulation dataset and hold-out tasks? It would be helpful to summarize the recommendations more explicitly and discuss how practitioners could apply them.

  2. In Section 5.2 and Figure 7, can co-training on 2+3 improve the reliability as well?

  3. In Fig 4. (b) and (d), the bottom-right action prediction plots are missing Real pred. and Sim sample, why?

Reviewer_zG9qOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This study examines how sim-and-real co-training affects the reliability of simulated policy evaluation, distinguishing the preservation of target-domain behavioral differences from their overwriting by simulated demonstrations. The analysis compares step-wise and behavior-wise agreement through synthetic experiments and two manipulation tasks, with real-world validation limited to step-wise comparisons on PutBottles.

Critical strengths
  1. This study addresses a relevant issue in simulation-based policy evaluation through the sim-transferring and sim-overwriting framework. Distinguishing step-wise from behavior-wise agreement gives the analysis a clear structure for explaining why score correlation alone does not establish behavioral correspondence.
  2. The experimental design combines synthetic mappings, sim-to-sim manipulation settings, and a real-robot checkpoint comparison. It varies policy sets, domain gaps, and simulated dataset size and coverage, including cross-task comparisons and renderer changes with fixed dynamics. These choices provide a reasonable framework for examining the proposed explanation.
  3. The training and evaluation protocols are described in useful detail, including behavior-mode combinations, sampling weights, checkpoint selection, and rollout counts. The appendices also specify the synthetic mappings and training settings, making the experimental construction reasonably easy to follow.
Critical weaknesses
  1. The real-world validation is too limited for the central claims. This study contains only 300 real episodes on PutBottles and tests only step-wise agreement, whereas related studies report substantially larger real-world evaluations [1][2][3][4][5]. Behavior-wise agreement, effects of simulated dataset size, and the hold-out setting remain untested on real robots. The main recommendations therefore lack sufficient real-world validation.
  2. This study attributes the improved agreement in the hold-out setting to excluding the evaluation task from simulated training. However, this setting also adds 75 target-domain PutBottles demonstrations and changes MugFlip sampling frequencies. These changes could independently affect agreement, so the comparison does not isolate the benefit of excluding the evaluation task.
  3. This study uses a classifier on time-averaged joint positions to support the claim of sim-overwriting. However, different motion sequences and object interactions can produce similar averaged features, lowering classification accuracy while preserving distinct executed behaviors. The probe therefore does not sufficiently distinguish sim-overwriting from information lost through time averaging.
  4. This study constructs seven-policy comparisons to examine behavior-wise agreement, using different combinations of demonstrated behavior modes. These policies also receive 25, 50, or 75 target-domain demonstrations, so their performance differences can reflect dataset size as well as behavioral coverage. The comparisons therefore do not isolate the preservation of behavior-related differences from effects of target-domain dataset size.
  5. The suggestion to limit simulated dataset size is based on sweeps at a fixed sampling weight of 0.1 and fixed training durations. The preferred dataset size may depend on repeated exposure and adaptation under this training recipe. Without examining other sampling weights and durations, the results do not establish how broadly the small-dataset recommendation applies.
  6. This study examines gap alignability through constant versus state-dependent synthetic action corrections and cross-task comparisons. However, state-dependent corrections can follow transferable rules, such as action rescaling, while cross-task comparisons also change task goals. Poorer agreement could therefore reflect learning difficulty or task mismatch rather than an absence of transferable structure.

[1] Xuanlin Li et al. Evaluating Real-World Robot Manipulation Policies in Simulation. CoRL, 2024, pp. 3705-3728.

[2] Adam Wei et al. Empirical Analysis of Sim-and-Real Cotraining of Diffusion Policies for Planar Pushing from Pixels. IROS, 2025, pp. 5625-5632.

[3] Ruixing Jin et al. Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Models. ECCV, Part XXVII, 2026, pp. 20-37.

[4] Yu Lei et al. A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies. CoRR, abs/2604.13645, 2026.

[5] Shuo Wang et al. A Practical Recipe Towards Improving Sim-and-Real Correlation for VLA Evaluation. CoRR, abs/2606.10366, 2026.

Questions for authors
  1. Can real-world evaluation cover additional tasks with a substantially larger rollout budget and test behavior-wise agreement, effects of simulated dataset size, and the hold-out setting?
  2. Can the same-task and hold-out settings be compared with identical target-domain auxiliary data, matched MugFlip sampling frequencies, and equal training budgets, varying only whether simulated demonstrations include the evaluation task?
  3. Can the sim-overwriting claim be checked using executed-behavior labels and features that retain motion order and object interactions? Do the conclusions persist when these behavioral distinctions remain observable to the classifier?
  4. Can the seven-policy behavior-wise comparisons match the number of unique target-domain demonstrations and training exposure while varying behavior-mode coverage? Do the reported agreement trends persist under this control?
  5. Can the sweeps over simulated dataset size be repeated at additional sampling weights and training durations to establish whether the preference for smaller simulated datasets persists?
  6. Can you clarify how gap alignability is determined and test a nonconstant but structured correction, such as action rescaling, within the same task? This would help separate transferable structure from mapping complexity and task mismatch.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.