- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes ASSEMBLE: each video is first turned into a timestamped textual "evidence catalog", and a Qwen3.5-9B reader answers from it using 23 prompt-defined "atomic skills", citing catalog entries. Training is LoRA SFT on Qwen3-VL-235B teacher responses (4,416), then GRPO on Video-Holmes only, with a correctness-gated citation term A·P_cite in the reward. On Video-Holmes, VRBench and CG-Bench-mini, ASSEMBLE reports 59.2% macro accuracy (Gemini-2.5-Pro 58.3%) and 25.4% Grounded accuracy (vs 18.7%) (Table 1).
Critical strengths
- The correctness-gated citation reward is supported by a clean, controlled ablation. On Video-Holmes, SFT and outcome-only GRPO raise accuracy (43.44 → 51.99 → 54.44). This shows that answer-level RL does not by itself produce evidence-aligned citations.
- The analysis is transparent and self-critical in places. The authors report interventions (Table 10), citation breakdowns (Tables 6–8), often unfavorable transfer results (Table 9) and failure cases (Figs. 8–10).
Critical weaknesses
- No comparison with models that take the raw video as input. All comparisons, including Gemini-2.5-Pro, read the same precomputed textual catalogs (Section 5; Table 1 caption; Appendix F), and the paper reports no result for any model given the raw video. "Matching Gemini-2.5-Pro" therefore shows comparable reading of textual evidence, not comparable video reasoning.
- The Grounded metric is too lenient to support the headline grounding claim. Under standard localization metrics the advantage over Gemini largely disappears (Table 2): Union mIoU is 1.40 vs. 1.41 on Video-Holmes and lower than Gemini on VRBench (11.76 vs. 17.73), and Acc@GQA stays near zero on Video-Holmes (0.16 vs. 0.11) and CG-Bench (1.71 vs. 0.99). The "+6.7% Grounded" therefore mainly reflects overlap with coarse entries rather than localization.
Questions for authors
- Please report results for Gemini-2.5-Pro and Qwen3-VL with the raw video as input, to quantify the gain or loss introduced by catalog construction itself, and revise the comparison with Gemini in the abstract accordingly.
- Are all GRPO ablation rows in Table 1 evaluated with the same inference procedure as ASSEMBLE (full), i.e., the five-call Dynamic Skills pipeline? On VRBench, whose default catalog contains no narrative entries, "+GRPO (full reward; w/o narrative)" (75.96 / 30.10) and "ASSEMBLE (full)" (78.99 / 35.15) appear to describe the same configuration but differ substantially. Please clarify what distinguishes these rows.
- How are graph-oriented skills such as retrieve by relation ("query graph paths or relation edges") executed on the flat (text, interval) catalog? Please report per-skill invocation frequencies on each benchmark, and an ablation with a reduced library (e.g., only the skills that appear in the traces).
- What prompts and citation-parsing rules were used for the external baselines, and what fraction of their responses contain at least one valid citation? A fairer control would be a baseline fine-tuned on the same SFT data and format.
- Dynamic skill selection shows no clear advantage over a fixed skill sequence (Table 3). With the reader and catalog held fixed, Dynamic Skills and Fixed Skills reach nearly the same accuracy on all three benchmarks (54.06 vs. 53.78 on Video-Holmes, 78.99 vs. 76.57 on VRBench, 44.59 vs. 44.54 on CG-Bench). The Grounded differences on VRBench (35.15 vs. 31.31) and CG-Bench (24.50 vs. 23.90) are within the reported confidence intervals; only Video-Holmes shows a clear gap (16.49 vs. 12.63). Question-specific skill composition, the central idea of the method, is therefore not shown to matter beyond a fixed localize-then-assess template.
- The skill library does not match the evidence representation. The 23 atomic skills are presented as a main contribution, but several are described as graph operations: retrieve by relation "query[ies] graph paths or relation edges", and retrieve by event returns "event or evidence nodes" (Table 5). Nodes, edges and paths are knowledge-graph or scene-graph concepts; these operations only have something to act on once the video has been built into a graph (e.g., person A —talks to→ person B). The catalog defined in Section 4.1 and Appendix A, however, is a flat list of (text, interval) entries with no nodes, edges or paths, so "querying relation edges" has nothing to retrieve. In practice, the 9B reader presumably reads the text descriptions and improvises according to the skill name. In addition, per-skill invocation frequency is not reported, and the qualitative traces use almost only a handful of skills such as localize_clue and score_hypothesis_support. This suggests the library has not been carefully designed or validated, and that many skills may be unused.
- The skill chain confirms a pre-selected answer rather than reasoning toward one.
- The planning call already proposes a candidate answer ("The planner proposes C" and similar in the traces), and the subsequent skills assess this pre-determined candidate, which invites confirmation bias.
- The authors acknowledge that the support assessment is advisory and does not constrain the final answer: in Fig. 9 the claim is assessed as unsupported, yet the final answer is still D.
- The external baselines are not adapted to the evaluation format. The video-reasoning baselines (Video-R1, VideoChat-R1, GRPO-CARE) are all built on Qwen2.5-VL-7B, a generation behind the reader, and none is adapted to catalog reading or the "(clip i)" citation format. The ASSEMBLE reader, by contrast, is trained with SFT and a citation reward on exactly this format. The near-zero Grounded scores of these baselines (e.g., 0.00 for Video-R1 on VRBench) more likely reflect that they were never fine-tuned on this output format than poor localization ability.