- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper examines sim-and-real co-training and its effects on making simulation an effective policy evaluator.
One contribution of the paper is the distinction b/w sim-transferring and sim-overwriting. The definition of these terms, however, is not clear. As best I can tell, they are effects rather than causes, i.e., they describe whether it helps or hurts to add sim data. In fact, the abstract (L015) calls them "effects". Clearly in some cases it will help, in others it will hurt. But I don't see any way to independently distinguish between these two other than to run the policy and see if it's better or worse.
Section 4 describes an experiment involves an R^3 to R^2 mapping, which allows a controlled study. The conclusion at the end of this section is that the simulated corpus should remain small, because too much sim data leads to sim-overwriting. But how small is small? And how much is too much?
Section 5 extends this analysis to sim-to-sim and real-to-sim evaluations of a VLA.
Critical strengths
- Sim and real are important topics in robotics.
- If we could pre-identify how much sim data to mix with real data in co-training, it would be useful.
Critical weaknesses
- The paper is difficult to follow.
- Two primary terms do not appear to be well-defined causal explanations, but rather are effects.
- No specific guidance appears from the experiments to help future researchers determine how much sim data is needed in co-training.
Questions for authors
- L020 "For behaviorally diverse policies, small simulation-data doses can improve behavior-wise agreement through sim-transferring. Larger doses can hurt agreement through sim-overwriting on states covered by simulated data, while continuing to improve agreement on Sim-OOD states, where direct sim-overwriting is avoided." These are the type of sentences that I find difficult to parse. What is "behaviorally diverse"? More specifically, are you saying that small sim-data doses would not help on a behaviorally non-diverse policy? Please explain how the statement to improve "through sim-transferring" avoids circular reasoning (since sim-transferring is the term used to describe improvement). Same for "sim-overwriting".
- L074 "evaluation-oriented use of co-training" - what does this mean?
- L132 Add the term "success rate" here for clarity (to explain the acronym SR)
- L086 "sim-overwriting lets shared simulated data determine each policy’s simulated behavior independent of their different real data, making their differences unobservable from simulated rollouts." Please explain.
- Is the point of Figure 1 that sometimes sim helps and sometimes it hurts?
- Please explain Figure 2.
- L291: What are "CIs"?
- What is the conclusion of Figure 4? And how is "alignable" defined?
- L477. "When using simulated rollouts to interpret policy behavior or diagnose differences in DR, researchers should verify that the rollouts preserve the relevant behavioral differences." How do you propose researchers do this?
- What is the conclusion of Table 1?