- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
EgoForge generates goal-directed egocentric video rollouts from minimal inputs: a single egocentric frame, a high-level instruction, and an optional exocentric reference image, with no camera trajectories, pose signals, video prefixes, or synchronized multi-view capture at inference. It is built on a Wan2.2-5B diffusion-transformer backbone with flow matching. The paper also curates X-Ego, 15,000 training clips plus 100 held-out test clips from Nymeria and Ego-Exo4D with hand-object annotations. Seven automatic metrics against five baselines plus matched-conditioning fine-tuned variants, component ablations, counterfactual conditioning tests, instruction sensitivity, a 20-rater human study, real-world ARGO smart-glasses deployment, and judge-reliability and reward-hacking analyses.
Critical strengths
The evaluation is thorough. Matched-conditioning baselines (+EV/+TT/+CI), an extra-SFT training-budget control, counterfactual conditioning tests, clip-start vs. clip-end exo references, instruction sensitivity, reward ablations, cross-judge and human agreement, a reward-hacking probe, and cross-backbone GRAFT transfer (Cosmos, HunyuanVideo) together make the results largely believable. The transparency about Wan2.2+CI being the authors' own Stage-1 and the separation of full-system versus method-only gains (Table 9) is exactly how this should be reported.
Critical weaknesses
- The headline numbers rely on an exocentric reference taken from the end of the clip, which reveals the final object state, i.e., the answer to the goal-completion question. The paper acknowledges this (Appendix H.6) and provides clip-start and no-exo controls showing the gains persist, which mitigates the concern. But Tables 2-4 and the abstract-level claims are built on the clip-end setting. If "goal-directed generation from minimal static inputs" is the claim, the clip-start or no-reference results should be the primary tables; otherwise the claim is weaker than presented.
- The evaluation scale is small: only 100 held-out test clips, 20 raters over 25 groups, and K=6 rollouts per condition for GRAFT training. FVD and flow MSE on 100 clips are noisy, and no variances or confidence intervals are reported. The "first large-scale benchmark" characterization of X-Ego also sits awkwardly on a 15k-sample derivative of Nymeria and Ego-Exo4D with LLM-refined captions, and the test set is much smaller than the training set.
Questions for authors
- Could the clip-start (or no-exo) results be made the primary tables? Given that the goal reward and the clip-end reference both carry future information, how much of the reported gains over baselines in Tables 2-3 depends on seeing the final state?
- With 100 test clips, what are the variances or confidence intervals of FVD, flow MSE, and DINO-Score? Are the reported gains statistically meaningful?
- At deployment (e.g., the smart-glasses scenario) there is no target-state image. How is Rgoal operationalized in that setting, and how does the model behave when the goal involves an unseen object state?