- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The manuscript mainly addresses the sim-to-real gap for dexterous hands and proposes a three-stage method called GEARS. The method design is reasonable, but the writing is unclear. There are also two suspected AI hallucinations in the paper involving references to the appendix. The most serious problem is that the experiments lack recent methods and classic baselines. The experimental setup is unfair, and maybe there is seed selection. The authors also acknowledge this.
Critical strengths
1.The details of the method are explained thoroughly, so readers can reproduce the method easily.
2.The ablation experiments are sufficient.
Critical weaknesses
1.From a statistical perspective, I question whether the reported positive results may come from the authors selecting random seeds and choosing the highest results. This cannot sufficiently support the conclusion that the method is consistently superior across different random seeds. In Table 1, none of the tasks reaches the conventional two-sided significance threshold of p < 0.05. The current sample size is not sufficient to confirm these improvements on its own. Of course, the authors also acknowledge this in lines 369 to 375. I would only consider increasing my score after the authors disclose the selection process for the highest GEARS result on each task. After the selection is complete, they should also reevaluate the method on an independent, pre-specified formal test set.
2.The compared baselines are relatively old. Recent methods are missing, and the representative baseline DP3 is also missing.
3.The real-robot experiments involve relatively simple tasks and are limited in scale.
4.The manuscript is not clearly written. Sections 3.2 and 3.3 are difficult to understand, and the motivation is not clearly explained. See the Questions section below for details.
Questions for authors
1.In Figure 2, are (3), the 6-DoF dexterous hand, and (4), the 7-DoF robotic arm, drawn in reverse?
2.Should the content in lines 175 to 178 be placed in Section 3.3? When readers look at the first part of Figure 1, the sudden mention of the student may confuse them.
3.In line 205, what is S in the MDP? Has it been mentioned? I may have missed it. If so, I hope the authors can point it out to me. If it has not been clearly explained, the authors could improve the wording here. Also, what does "γ is the discount factor" mean here? How should this be understood? I feel that this part is a little difficult for readers to understand. I suggest that the authors explain it clearly in natural language in the manuscript. The authors should ensure that every symbol here is clearly defined.
4.In lines 206 and 209, the sentence "The privileged expert observes ξ together with ..." is very difficult to understand. What does it mean? Is the sentence structure correct? For "joint positions and velocities, object poses and velocities, fingertip velocities, relative fingertip-object positions, contact state, and the previous action", are these items in a parallel relationship, or is there some other relationship? The repeated use of "and" here makes me very confused.
The sentence seems to mean that ξ is listed alongside object poses, velocities, and other quantities. However, line 209 says that ξ contains object poses. This is contradictory. The authors do not define many symbols here, but the explanation seems confusing. This makes it difficult for me to understand. I also question whether these symbols really need to be defined. Is it really impossible to explain the same content clearly in natural language?
If my understanding is incorrect, I hope the authors can correct me. I suggest that if the authors want to define equations and symbols, they should explain every symbol clearly. They should not leave readers and reviewers to guess what these symbols mean. Equations 1 and 2 are written clearly, but line 205 becomes very confusing.
5.In lines 222 to 223, what does "ten adaptive ranges" mean? I do not recommend referring to the appendix in this way. A reference like the one in line 215 is acceptable, but the description here is difficult to understand. The main text should allow readers to understand the content without having to read the appendix.
6.Does the manuscript explain why the "Automatic Domain Randomization via VP-SVGD" component is used? What problems does it solve?
7.Line 369 states, "The GEARS row contains the highest recorded per task rates for the full system." Does this mean that the comparison is unfair?
8.The No ADR curve in Figure 3(d) does not seem reasonable. It remains almost zero before 4 × 10^8 environment steps. Is there a problem with the experiment? Also, this contradicts the statement in line 429: "In Fig. 3(d), the VP-SVGD run reaches its final success level with fewer environment steps than the uniform-ADR run." Uniform ADR clearly reaches a stable level earlier than the method proposed by the authors.
9.Line 381 states, "Appendix C.1 states the remaining ambiguity in the recorded protocol for this control." Where is the relevant content in Appendix C.1? When using AI-assisted writing, the authors must carefully check the content.
10.Similarly, line 431 states, "Appendix B.3 reports the available cost information." I could not find the relevant content in the appendix either. This may be caused by an AI hallucination.
6 comments
Response to Reviewer 8m8E (2/5)
(Continued from Part 1/5.)
Q2. The baselines omit recent methods and the representative DP3 baseline. How does GEARS compare with these alternatives? (W2)
A2: Experimental setup: We extend the comparison to a pretrained VLA, two robot-specific encoders, and an RGB-based DP3 adaptation. For each training seed and task, all six methods use the same 400 successful simulated expert trajectories. They receive observations from the same two RGB cameras and robot proprioception. All methods predict the same 26 robot commands: 14 arm-joint velocity targets and 12 hand-joint position targets. We fine-tune [1] with a fixed instruction per task and pair frozen SPA and Theia encoders [2-3] with the same diffusion architecture as GEARS. DP3 [4] receives point clouds reconstructed from depth predicted by a separate, fixed metric-depth model based on Depth Anything V2 [5]. The same depth-prediction and calibrated back-projection pipeline processes simulated and real images. Task-specific policy training uses only simulated demonstrations. Each method is evaluated with three training seeds and 20 real-world trials per task and seed. Checkpoints, deployment settings, initial-condition sampling procedures, and success criteria are fixed before testing. The revised appendix will specify the pretrained checkpoints, trainable modules, training budgets, and input/output adaptations, including the metric-depth model and point-cloud preprocessing used for DP3.
Results and analysis: GEARS outperforms every baseline on all four tasks, with mean success versus for the strongest baseline, , and for DP3 with RGB-predicted depth. With the demonstrations and diffusion architecture fixed, replacing Geo with SPA or Theia reduces mean success by and , respectively, relative to GEARS. In a matched three-task ablation with the same Depth Anything V2 initialization, geometry fine-tuning raises success from to , more than doubling the success rate. These results demonstrate that task-specific geometric learning improves visual transfer and that GEARS achieves higher real-world success than the pretrained VLA and policies using alternative visual representations under the same demonstration budget.
Table 1. Real-world success rates (). Three seeds 20 trials per task; 60 trials per task and 240 per method. Avg. is the equally weighted mean over four tasks, computed before rounding.
Method Grasping Pouring Handover Lifting Avg. ACT + Geo (retest) 71.7 41.7 23.3 36.7 43.3 DP + SPA 46.7 41.7 38.3 35.0 40.4 DP + Theia 51.7 48.3 35.0 33.3 42.1 DP3 (RGB-pred. depth) 36.7 31.7 18.3 16.7 25.8 (sim fine-tuning) 76.7 50.0 43.3 48.3 54.6 GEARS 88.3 66.7 56.7 60.0 67.9 Note: ACT + Geo and GEARS are independently reevaluated. DP3 uses RGB-predicted metric depth. Relative decreases use GEARS as the denominator and are computed before rounding.
(References are provided at the end of Part 5/5.)
Response to Reviewer 8m8E (3/5)
(Continued from Part 2/5.)
Q3. The real-robot tasks are relatively simple and the evaluation is limited in scale. What conclusions does this evaluation support? (W3)
A3: Our study addresses three challenges in visual sim-to-real transfer for dexterous manipulation:
- Visual perception: The student must infer contact-relevant geometry from RGB despite appearance changes between simulation and reality. This is particularly challenging for transparent objects, whose appearance depends strongly on background and lighting.
- Policy learning: The RL teacher jointly controls the arms and hands using 14 arm-joint velocity commands and 12 hand-joint position commands across varying physical conditions. The student learns to perform these tasks from RGB and proprioception, while the teacher also observes object state, contact state, and physical parameters.
- Task execution: The difficulty lies in maintaining stable contact while coordinating orientation and timing across the arms and hands. Pouring requires regulating the amount of liquid transferred without losing grasp stability. Handover requires the receiving hand to secure the object before release, while bimanual lifting requires synchronized motion to keep the container level. Small errors in contact timing, vessel orientation, or relative hand motion can cause drops or spills.
The new comparison covers six methods, four tasks, three training seeds, and 20 trials per task and seed: 1,440 real-world executions. GEARS achieves the highest success rate on all four tasks, and the component experiments identify geometric learning and adaptive expert training as contributors to this advantage. The evaluation therefore demonstrates improved RGB-based sim-to-real transfer across distinct grasping, pouring, and bimanual coordination requirements. We will make these challenges and the task-specific success criteria explicit in the revised manuscript.
Q4. Why does No ADR remain near zero until approximately steps? Fig. 3(d) also appears inconsistent with the claim that VP-SVGD reaches its final success level earlier than uniform ADR. (Q8)
A4: Training behavior: No ADR retains physical and visual randomization within fixed ranges. The initial policy rarely succeeds under difficult conditions, so learning starts slowly. VP-SVGD adjusts the ranges to the teacher's progress: low success triggers contraction, intermediate success keeps the ranges fixed, and higher success permits particle-based updates. In Fig. 3(d), Uniform ADR reaches a lower plateau earlier, while VP-SVGD continues to improve and finishes at a higher success rate. We will replace the statement at line 429 with this distinction and define fixed-range DR consistently in the text, tables, and figure caption.
Experimental setup: We evaluate checkpoints from the three-seed SAC runs that provide the demonstrations for the students in Table 4. Evaluation covers Grasping, Pouring, and Handover, using the same fixed distribution of initial conditions and physical parameters for every method. Each checkpoint is tested for 100 episodes per task and seed. Training budgets include all simulator interactions, including Active DR's reference rollouts [6].
Results and analysis: Uniform ADR leads at interactions with success. VP-SVGD overtakes it by and continues from to over the remaining budget. Active DR and Uniform ADR improve from to and from to , respectively; fixed-range DR finishes at . Thus, VP-SVGD achieves the highest final success under the same interaction budget. Its advantage is higher performance with continued training, while Uniform ADR learns faster initially.
Table 2. Simulation success at matched training budgets (). Grasping, Pouring, and Handover; three training seeds per method and task; 100 evaluation episodes per task, seed, and checkpoint. Entries average equally over tasks and seeds.
Method GEARS / VP-SVGD 36.0 78.0 89.0 Uniform ADR 49.0 63.0 66.0 Active DR 39.0 71.0 74.0 No ADR (fixed-range DR) 2.0 8.0 48.0
(Table 4 appears in Part 5/5; references are provided at the end of Part 5/5.)
Response to Reviewer 8m8E (1/5)
Q1. Were the reported results obtained by selecting the best seeds or evaluation batches? The task-level comparisons are not significant at p<0.05. Please disclose the selection process and reevaluate the selected models on an independent, prespecified test set. (W1,Q7)
(Table 1 appears in Part 2/5; references are provided at the end of Part 5/5.)