- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies subject consistency in generated videos. It breaks diagnosis into subject binding, attribute comparison, and issue filtering. Stage-specific teachers then transfer these skills to a single 9B model. Training uses 2,160 structured traces, including unchanged attributes and changes allowed by the prompt. On a 50-video benchmark, the method improves F1 from 36.68% to 41.88%. This exceeds the three-call fine-tuned pipeline and the other 9B distillation methods evaluated.
Critical strengths
1.The task decomposition is well motivated. The paper distinguishes visible differences from defects that should actually be reported. It also retains unchanged attributes and permitted changes. This supervision fits the task well and could help reduce false alarms. 2.Providing stage-specific feedback along the same student-generated trajectory is a meaningful extension. The distillation methods start from the same student checkpoint. The three-call baseline also reuses the same weights. These controls help assess the value of the integrated training approach.
Critical weaknesses
1.The main results need stronger validation. The benchmark contains only 50 videos. Checkpoints are selected by monitored F1, but it is unclear whether monitoring uses a separate validation set. The gain over Video-OPD is just 0.78 F1 points, with no uncertainty analysis. Also, under the stated protocol, the per-seed scores in Appendix D do not reconcile with the pooled counts in the main table. The authors need to check and explain this discrepancy. 2.The contribution of stage specialization is not yet clear. Table 3 provides some support, but the effects of teacher configuration, additional training, and loss reweighting remain difficult to separate. The paper does not report a continued fine-tuning baseline with the same starting checkpoint, supervision, and a comparable training budget. Without this control, it is hard to tell how much the on-policy stage feedback adds.
Questions for authors
1.Was the final evaluation set also used to select the best model? 2.Could you clarify how the results in Table 2 relate to Appendix D? Do they use the same model and evaluation setup? 3.The gain over Video-OPD is small. Does it hold on a separate set of videos? 4.What happens if you keep the same supervised losses and training budget, but leave out the teacher feedback? 5.How were disagreements resolved during annotation review? Were the automated matching decisions checked by human reviewers?