- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SUTURE, a rollout-group-conditioned verifier for video temporal grounding with reinforcement learning with verifiable rewards. The verifier redistributes weight within the annotated target using temporal group coverage, while pairwise disagreement controls the redistribution amplitude subject to a floor. Normalization preserves total target mass and leaves the IoU denominator unchanged. The paper derives an exact covariance-based reward decomposition and a fixed-group local GRPO direction identity. Experiments report improvements over TaRO across five benchmarks, with a second backbone evaluated on the three primary benchmarks. Fixed-amplitude controls, a separate three-run comparison, sensitivity analyses, and temporal-behavior diagnostics further characterize the method.
Critical strengths
-
A simple and interpretable verifier modification. The separation between where to redistribute the target weight and how strongly to redistribute it is well motivated. The exact decomposition into ordinary IoU and a group-conditioned correction clarifies how this changes the learning signal. The matched-overlap ordering result and the local gradient analysis are appropriately qualified in the appendices: the latter permits a non-collinear correction but does not guarantee one for every group. This is more informative than an unexplained reward heuristic would be.
-
Evidence beyond headline comparisons. The shared-setting Naive IoU and fixed-amplitude controls provide useful evidence for structured target weighting. Appendix F.1/Table C reports three-run means and sample standard deviations on the primary benchmarks, with the reported means favoring SUTURE. Group size and floor sensitivity are also examined in Appendix G.1/G.3. Together with the reported gains across benchmarks and the second-backbone results, these establish a credible positive case. The explicit limitations on selected qualitative examples, local gradient diagnostics, and token-count measurements are also valuable.
Critical weaknesses
-
The specific benefit of disagreement-adaptive amplitude is not fully isolated. Table 3 and Appendix G.2 show that the adaptive rule outperforms the tested constants, but these ablations are explicitly single-run point estimates. A mean-matched constant does not match the adaptive amplitude distribution or its training-stage evolution. Appendix G.3 also reports that 82.8% of groups are floor-clamped at the analyzed late-training stage; this is not a whole-training statistic, but it makes the timing of the adaptive contribution particularly relevant. These results support the narrower empirical comparison while leaving uncertainty about whether the advantage specifically stems from assigning amplitude based on current group disagreement. This limits confidence in the claimed contribution of the adaptive component, rather than invalidating structured weighting itself.
-
Some evaluation details are insufficiently specified for independent reproduction. Appendix E provides splits and counts for the three primary datasets, but the exact Ego4D-NLQ/TACoS evaluation splits, counts, complete second-backbone configuration, and checkpoint-selection criterion remain unclear. The long-video improvements in Table 2 are modest (0.4–1.0 percentage points), and lack reported across-run uncertainty. Separately, Figure 2 does not define its shaded bands, binning, target-start category cutoff, or treatment of outputs without valid temporal mentions. Appendix G.5 gives first-mention and hit conventions for a different comparison, without explicitly linking them to Figure 2. These omissions limit how confidently the broader performance and behavioral conclusions can be assessed.
Questions for authors
-
Adaptive contribution. How stable is the fixed-versus-adaptive advantage under matched seeds and a common checkpoint-selection rule? Can you clarify when the amplitude is above its floor during training? A control preserving the stage-wise amplitude distribution while breaking its association with current group disagreement would help distinguish disagreement-specific allocation from scheduling effects; this is a suggested diagnostic, not a claim that the existing comparison is incorrect.
-
Evaluation protocol. Please specify the long-video split versions/counts, the complete Qwen3 configuration, and the checkpoint-selection data and criterion. Which backbone and seed-pairing protocol does Table C represent, and is there uncertainty information for the long-video gains?
-
Behavioral and implementation definitions. For Figure 2, please define the bins, central statistic, shaded bands, population counts, cutoff for classifying targets as “at video start,” and handling of missing/malformed mentions. Do the first-mention and positive-overlap hit conventions in G.5 also apply here? Separately, Appendix D.1 explains zero-length coverage; how do invalid parses enter the group, and how is pairwise IoU defined when the union has zero duration? These details could materially affect interpretation and reproducibility.
2 comments
W2 / Q2. Evaluation protocol
These details were missing from the submission. Appendix J.5 of the revision states each of them in full.
- Long-video splits. Ego4D-NLQ uses the v2 validation split (4,552 queries), since test annotations are not public, and TACoS uses the 25-video test split (4,001 queries).
- Qwen3-VL-8B-Instruct. It follows TaRO's Qwen3 recipe and the GRPO settings of Appendix E.1. Table Q in Appendix J.5 lists the complete training and evaluation configuration.
- Table C. All runs use Qwen2.5-VL-7B-Instruct with seeds 42, 1, and 2. The constant-amplitude runs in W1 / Q1 above use the same seeds.
- Checkpoint selection. See the checkpoint rule in W1 / Q1 above.
- Long-video uncertainty. We trained and evaluated TaRO and SUTURE with Qwen3-VL-8B-Instruct over three runs (seeds 42, 1, and 2) under the same checkpoint rule. The table reports mean ± standard deviation of R1 (%).
| Dataset | Method | [email protected] | [email protected] | [email protected] |
|---|---|---|---|---|
| Ego4D-NLQ | TaRO | 12.4 ± 0.7 | 7.2 ± 0.7 | 3.6 ± 0.3 |
| Ego4D-NLQ | SUTURE | 13.5 ± 0.7 | 8.0 ± 0.6 | 4.4 ± 0.1 |
| TACoS | TaRO | 27.9 ± 0.6 | 17.3 ± 0.6 | 7.7 ± 0.4 |
| TACoS | SUTURE | 28.3 ± 0.2 | 18.3 ± 0.7 | 8.4 ± 0.8 |
SUTURE has a higher mean at all three thresholds on both datasets. The revised Table 2 reports these across-run statistics.
W2 / Q3. Behavioral and implementation definitions
The revised Figure 2 caption and Appendix J.5 give the complete definitions, including the bin edges and the number of queries in every bin and split. Figure 2 follows the conventions of Appendix G.5:
- First temporal mention. The first "X s to Y s" span after
<think>; Appendix J.5 gives the full parsing rule. - Panel (a). ActivityNet queries are grouped into seven bins of normalized target start . Markers are medians and bands are interquartile ranges, over the same 14,005 queries for every method.
- Panel (b). A target is at the video start when . A hit is any positive overlap with an annotated window, as in Appendix G.5.
- Malformed responses. A query is kept only if all three methods produce a well-formed response with a valid first mention. Nearly all removed queries are Naive IoU responses without an
<answer>tag, in line with its rate of reaching the 1,024-token limit (Table H).
Invalid parses and zero-length unions. An invalid response (no <answer> tag, no parsable interval, or an end that does not exceed the start) stays in the group as an empty prediction that covers no part of the target. Its pairwise IoU with every other member is 0. Valid intervals have positive length, so a union of zero duration occurs only between two invalid responses and follows the same rule. Sections 2.1 and 3.1 now state this convention.
We thank the reviewer for the detailed reading, including the appendices. Changes in the revised PDF are marked in blue, and equation and appendix references follow the revised PDF.
W1 / Q1. Contribution of the disagreement-adaptive amplitude
We ran both checks the reviewer proposed: matched seeds under a common checkpoint rule, and a control that keeps the amplitude distribution while breaking its link to each group's disagreement.
Matched seeds. We trained the mean-matched constant a=0.4 with the seeds of Table C (42, 1, and 2) and compare it with the three SUTURE runs already reported in Table C (Appendix F.1) of the submission. All runs use Qwen2.5-VL-7B-Instruct and the training settings of Table 3. Values are mean ± sample standard deviation over the three runs.
SUTURE has the higher mean on all twelve metrics, with mIoU gaps of 1.4 to 2.4 points.
Checkpoint rule. For every method and run, we select the checkpoint with the highest mIoU on TVGBench (Wang et al., 2025), following TaRO's checkpoint-selection protocol. TVGBench is not used for evaluation. The same checkpoint is used on every benchmark, and Appendix J.5 states this rule.
Within-batch distribution-preserving control. A control that permutes amplitudes across groups while preserving their within-batch distribution falls 3.3 mIoU below SUTURE. We compute the usual floored amplitudes for groups with uP>0 and randomly permute these values among those groups before forming the weight fields. Each ϕP stays with its own group. The permutation exactly preserves the set of amplitudes within each batch while randomizing their assignment.
All three runs below use Qwen2.5-VL-7B-Instruct, the training settings of Table 3, and the checkpoint rule above. Values are single-run equal-weight averages over QVHighlights and Charades-STA, in percent. The fixed a=0.4 and SUTURE rows are already reported in Table 3 (Section 4.3) of the submission.
The shuffled control falls below SUTURE on every metric. On ActivityNet it reaches 40.9 mIoU, against 44.3 ± 1.8 across the three SUTURE runs of Table C. Appendices J.1 and J.2 report these results.
The shuffled control also falls below the constant. One explanation consistent with the training statistics is that random reassignment can give a large amplitude to a group whose rollouts agree and cover most of the target. There ϕˉP approaches −1, so the correction coefficient βP=aP/(1+aPϕˉP) grows large. Over the first 234 steps, 23.5% of groups in the shuffled run have βP>0.9, against 6.9% under the adaptive rule.
When the amplitude is above the floor. With ufloor=0.3, the amplitude exceeds the floor for 88% and 60% of groups in epochs 1 and 2, and for 17% in epoch 3, the late stage analyzed in Appendix G.3. The adaptive rule therefore differs from a constant mainly in the first two epochs, and the shuffled control above, which keeps each step's amplitudes but reassigns them across groups, tests whether this assignment matters.