- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Paper study if WAM follows the current language instruction when the same scene can supports different goals. It evaluates it via counterfactual (CF) instructions while having the scene and initial robot state fixed. The paper- LIFT-WAM builds upon Fast-WAM through CF adaptation, grounded intent extraction & corrective training through expert continuations from states reached while pursuing the wrong goal. The paper evaluate on LIBERO-obj and 10, RoboTwin & a real robot. The results show substantial improvements over the base model, with CF adaptation contributing most of the gain. An extra experiment also examines how language conditioning in the video pathway influences executed actions.
Critical strengths
- The evaluation addresses an important problem. I liked the use of different feasible instructions for the same scene and initial state. Which provides a useful test of if the policy follows language, beyond what standard task-success evaluations reveal. The distinction between behavioral persistence, grounding errors, and realization failures also helps explain where instruction following can break down.
- The video-pathway intervention is informative and I like dthat. I found the result that changing only the Video Expert language conditioning can redirect the adapted policy particularly interesting. It shows how language influences control in this architecture and adds insight beyond the overall performance gains.
Critical weaknesses
-
I am not yet convinced that the experiments isolate the contribution of each component. The overall improvement over the base model is substantial, but several gains from grounded intent and full-goal corrective training are small relative to the evaluation size, especially on LIBERO. Without confidence intervals or repeated training runs, it is difficult to judge their reliability. More importantly, plain fine-tuning on the same counterfactual data and a data-matched corrective-training control would help establish whether the gains come from the proposed objectives and supervision strategy. This matters because these components form a central part of the main contribution in the paper.
-
Important methodological details remain difficult to reconstruct. I had to infer how several components work. The counterfactual-adaptation objective should be stated explicitly in the main text, and the completed-clause indicators (m_t), deployment gate, and replay annotations need clearer definitions. For example, it is unclear how completed clauses are detected during execution or whether incorrect completion flags can be revised. The source of the grounding annotations, particularly for the real robot, also needs clarification. These details affect both reproducibility and my assessment of the supervision required.
-
Baseline is missing from the real robot experiment.
-
Hard to read the paper. Many terms are either undefined or defined very late in the paper. What is GIE? What is the purpose of the gain gate? and so on. Even with limited spac main architecture pattern should be clear to the readers.
-
Privileged supervision is not accepted clearly. GIE and routing are trained on replay annotations, (possibly) simulator ground truth for entities, positions, and predicate truth. Baseline dont gets comparable information, and how these labels are obtained for the real robot is unclear.
Questions for authors
-
Can you state the exact CF-adaptation objective and clarify why the negative-condition error is stop-gradiented in ints equation. Do you have a comparison against plain flow-matching fine-tuning on the same data?
-
How are (m_t) and the deployment gate computed and updated during execution? What annotations supervise grounding, and how are these obtained for the real robot?
-
Can you give uncertainty estimates for the component gains and clarify whether corrective-training comparisons use matched data or training budgets? Please also clarify whether the 53.3% diagnostic result and the 26% RoboTwin Clean result use the same checkpoint and evaluation protocol.
-
Where do LIBERO counterfactual demonstrations come from, given LIBERO has no default counterfactual demos?
-
Given Section 4.3’s finding that language acts mainly through the video pathway, did you try injecting grounded intent on the video side? Why the action expert?
-
Can you provide more details about the UNT David A1C robot and its capability and DOF in the paper