- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces CaReVOS, a benchmark for identifying event participants from the opening and closing segments of a video while the intervening interaction is hidden from the reasoner. The selected objects are then segmented throughout the full video. The authors propose Event-OPD, which combines supervised answer and mask training with privileged on-policy distillation from a frozen copy of the SFT model that observes the complete event. Distillation aligns both next-token distributions and segmentation-token representations. The method achieves 60.16 J&F, improving over its SFT initialization by 1.62 points.
Critical strengths
The combination of missing-event inference and privileged on-policy distillation is well motivated. A teacher with access to the complete event can provide additional evidence when guiding a student that only observes the setup and outcome. Extending this guidance to the segmentation-token representation also connects answer-level distillation to spatial grounding. I consider this combination the main methodological highlight.
CaReVOS provides a concrete benchmark for studying this setting, with human-verified questions and object masks across multiple video sources. Requiring pixel-level outputs makes target selection errors observable and adds a useful grounding component to missing-event reasoning.
Critical weaknesses
-
Ambiguity under partial observation is insufficiently quantified. The same opening and closing observations may support different hidden interactions and, potentially, different target identities. Although the authors describe manual verification and ambiguity filtering, the paper does not quantify how reliably independent annotators can identify the intended targets without seeing the hidden event. The multi-solution category addresses multiple jointly valid targets, but does not by itself resolve uncertainty between alternative explanations. This matters because evaluation against one reference target set could penalize another answer that is also defensible from the available evidence.
-
Evidence for improved reasoning remains limited. The reported gains are measured primarily through segmentation quality, which combines target selection and mask accuracy. It is unclear whether the model generates explicit reasoning traces, whether thinking mode is enabled, and which reasoning content is actually distilled. The improvement over SFT is 1.62 J&F points, while giving the teacher the complete event adds 0.86 points over the restricted-view teacher. These gains are encouraging, but their stability and reasoning-specific contribution remain uncertain without repeated runs, a training-budget-matched continued-SFT control, and a separate evaluation of target selection or event inference.
Questions for authors
-
How do you determine that the target identity is sufficiently constrained by the visible segments and question? Do you have an independent annotation study in which annotators never see the hidden event? How are alternative but defensible target sets handled during evaluation?
-
Is Qwen3.5's thinking mode enabled during SFT, on-policy rollouts, and evaluation? Do the generated responses contain explicit event-inference rationales, or mainly target descriptions? Please clarify which tokens receive distillation supervision and provide representative outputs. A controlled comparison with and without explicit reasoning would help assess its contribution.
-
How stable is the 1.62-point improvement across runs, and how does Event-OPD compare with continuing SFT for the same additional training budget? Can you separately evaluate target selection to distinguish reasoning gains from improvements in mask prediction?
As a possible extension, have you explored varying the hidden interval or constructing multiple validated queries and alternative explanations per clip? Such augmentation could help test robustness to different observation patterns, provided that each example is checked for answerability.