- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes TGPED for procedural action segmentation, error detection, explanation, and omission localization across multiple tasks. PGTS uses frozen models and generated procedural references to organize timestamped video captions into action segments. A shared LLM detector is then fine-tuned on synthetic caption sequences containing errors, constructed from normal training videos. Experiments on EgoPER and five selected CaptainCook4D tasks evaluate all four capabilities. Task-specific and leave-one-task-out comparisons provide evidence for sharing the detector across tasks and transferring to held-out tasks when procedural references are available.
Critical strengths
-
Reference-conditioned cross-task reuse. PGTS avoids training a separate segmentation model for each task, while the detector shares parameters across tasks. On EgoPER, leave-one-task-out detection F1 reaches 40.0, compared with 42.5 when all tasks are included. This supports reuse on held-out tasks given their procedural references.
-
Useful supervision from normal videos. The synthetic-error pipeline enables direct training for detection and explanation without collecting real erroneous training videos. On EgoPER, fine-tuning improves end-to-end detection F1 from 30.1 to 42.5, providing meaningful evidence for the value of this supervision.
-
Explicit localization of omitted steps. The method predicts both the missing action and where it should be inserted, making procedural feedback more actionable. On CaptainCook4D, omission localization F1 reaches 40.1 versus 19.8 for GTG2Vid at a time tolerance of 5% of the video length.
Critical weaknesses
-
Limited novelty. The method appears primarily incremental, and the paper does not sufficiently distinguish its synthetic-error construction from related work such as PIE-V, leaving the specific methodological advance unclear.
-
Generalization claims exceed the experimental evidence. The leave-one-task-out setting still constructs the target-task reference using normal videos and ground-truth action annotations from that task. Thus, deployment without target-task annotations is not fully validated.
-
Insufficient controlled ablations and quantitative quality validation of synthetic supervision. Different synthesis strategies are not compared under controlled conditions. Although manual inspection is reported, quantitative quality assessments are missing, leaving the necessity of full-sequence supervision and the realism of synthetic errors uncertain.
-
Insufficient validation of captions for fine-grained error detection. All downstream reasoning relies on captions, so omitted or misinterpreted visual details may directly cause detection failures. Robustness to caption noise and missing details is insufficiently evaluated, limiting confidence in practical reliability.
Questions for authors
see above