acceptodds
Under review as a conference paper at ICLR 2027

Accuracy Is Not Evidence: Measuring and Training Evidence Dependence in Video Temporal Grounding

Abstract

In video temporal grounding, a correct interval can come from finding the event, answering blind or fitting one annotation style, and Recall at IoU scores them alike. We measure evidence dependence, how much the answer rests on the event's frames. Our fixed‑answer audit edits either the frames inside the annotated interval or, as a placebo, as many frames elsewhere in the same video, and compares the two drops in the annotated answer's likelihood. Supervised fine-tuning and three public checkpoints trained with IoU-reward reinforcement learning raise the accuracy of Qwen2.5-VL-7B by 9 to 20 points, partly by answering blind and by fitting one annotation. Yet their answers rest on the event no more than the base model's. This comparison is a difference of the model's own likelihoods, so we can train on it. A placebo contrast and a bound on the placebo drop rule out learning to react to any edit. With them, evidence dependence rises in all five base models we train, and the weight on clean supervision controls how much. Masking the event's frames makes the trained models decline or miss the event more often than their base; masking as many other frames does not. The dependence moves to another event when the query names it, and holds on a second dataset and under an unseen edit.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video temporal grounding, evidence dependence, video-language models, placebo control, likelihood audit, post-training, shortcut learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SudqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that high accuracy in Video Temporal Grounding (VTG) does not necessarily imply reliance on relevant visual evidence. The authors propose a fixed-answer audit that compares the ground-truth answer's log-likelihood when the annotated event frames are removed versus when an equal number of other frames are removed. This difference is defined as evidence dependence (ψ), and its change after training is measured by Γ.

The authors show that SFT and IoU-based RL improve VTG accuracy but do not meaningfully increase Γ. They then propose a training objective that explicitly increases evidence dependence and validate it across multiple base models. Additional experiments involving frame removal, query changes, different datasets, and unseen video perturbations demonstrate corresponding changes in model behavior.

Critical strengths
  1. A well-controlled evaluation of evidence dependence The fixed-answer audit measures changes in the likelihood of the same answer, reducing ambiguity associated with regenerating predictions after modifying the input. The matched placebo comparison also helps distinguish sensitivity to general video corruption from selective dependence on relevant event frames The finding that existing RL models improve their ability to follow relocated events without increasing Γ is particularly interesting, as it highlights the distinction between locating an event and depending on its visual evidence.

  2. Convincing training ablations and behavioral validation Figure 3 and Table 2 show that simply training a model to reduce its confidence when event frames are removed can make it overly sensitive to unrelated frame modifications. The proposed placebo contrast and bound effectively address this issue. Furthermore, the query-crossed experiment in Figure 5(a) shows that the frames a model depends on change when the query refers to a different event. Although this behavior is not consistent across all models, the results support the claim that the learned dependence is related to the query's meaning.

Critical weaknesses
  1. The practical benefit of further increasing Γ and its relationship to actual predictions remain unclear. Table 19 shows that existing models already experience substantial accuracy drops under evidence masking, while their performance remains largely unchanged under placebo masking. For example, TimeZero achieves 70.0% accuracy on clean videos, 21.8% under evidence masking, and 70.9% under placebo masking. This suggests that existing models already make substantial use of relevant event frames. While the authors show that Γ does not increase under accuracy-oriented training, the additional benefit of increasing Γ in models that already use visual evidence remains unclear. The proposed method changes model behavior when evidence is removed, but evidence of improved localization accuracy or generalization is limited. For example, on QVHighlights, the proposed Qwen-based model (α=0.5) increases Γ but improves accuracy by only 0.1 percentage points, compared with 6.8 points for SFT (Table 17). Additionally, ψ measures the likelihood of a single annotated answer, whereas actual predictions also depend on the probabilities of alternative intervals. Analyzing the relative likelihoods of the ground-truth answer and competing predictions would help clarify how ψ relates to evidence use in actual temporal localization decisions.

  2. Insufficient discussion of related work and the distinction from existing approaches. The paper shows that evidence-dependence training can lead models to abstain when relevant evidence is missing. However, related work on identifying and rejecting irrelevant queries, including RaTSG (NeurIPS 2024), NA-VMR (WACV 2025), and RA-RFT (CVPR 2026), is not sufficiently discussed. Admittedly, this paper focuses on removing event frames, whereas these studies mainly address queries that do not match the video. Nevertheless, the mismatched-query evaluation in Table 21 directly connects to this line of research. It would therefore be valuable to clarify whether training models to reject irrelevant queries also increases evidence dependence, and what distinguishes the proposed approach from these methods.

Questions for authors

Q1. What additional benefits does increasing Γ provide? Existing models already show substantial reliance on relevant event frames (Table 19). How does further increasing Γ improve VTG performance or prediction reliability, particularly when compared with models at similar accuracy or answer coverage?

Q2. Can the relationship between ψ and actual predictions be further examined? Could the authors compare the relative likelihoods of the ground-truth interval and competing predicted intervals under evidence and placebo masking, keeping candidate answers fixed across conditions? This would help clarify how well ψ reflects evidence dependence in actual temporal localization decisions.

Q3. Does refusal-aware training also increase evidence dependence? Please clarify the relationship to RaTSG, NA-VMR, and RA-RFT. If feasible, applying the same audit to a model trained with RA-RFT would help determine whether refusal-aware training also increases Γ and what distinguishes the proposed approach.

Q4. Do the main audit results generalize to other evaluation samples? The main audit uses the first 200 test questions, on which the base model achieves higher accuracy than on the full test set. Does the conclusion that accuracy-oriented training does not increase Γ hold on other randomly sampled subsets?

6 comments
AuthorsOpenReview ↗

Reply to Reviewer 3 (1/2): what a higher Γ adds; relative likelihood against competing intervals

Thank you for the review and the concrete suggestions. We ran the RA-RFT audit you suggested and also audited NA-VMR (part 2/2). All comparisons are paired within one stack, as in the general response.

W1 and Q1. What a higher Γ adds when models already use the event

We agree that base models use the event frames, and a higher Γ does not add accuracy by itself. The paper measures the change Γ, and the base already has ψ > 0 (p3 l.139–140). Your TimeZero example fits this reading. Masking the event removes about the same share of the accuracy TimeZero keeps under the placebo mask as for its Qwen2.5-VL base (69% for the base, difference +1.3 [−4.2, +6.9] points, Table 1), although TimeZero is about 20 points more accurate. Two comparisons at matched accuracy or coverage show what a higher Γ adds.

The first is on TimeZero, a model that already uses the event. On Charades-STA our objective leaves TimeZero's accuracy (+0.5 [−0.7, +1.5]) and coverage (0.999) unchanged, raises Γ to +1.40 (Table 3) and the share of correct answers lost to the evidence mask by 10.8 points, and lowers the rate of answering a fully black video from 0.968 to 0.303 (Tables 7, 10, 19). On QVHighlights it costs 3.8 points (Table 17). Added to IoU-GRPO and trained from TimeZero, it gives +1.0 [+0.0, +2.0] points of accuracy with Γ +1.04 [+0.96, +1.12], and masking the event lowers its [email protected] by 4 [1, 7] points more than the placebo mask does, relative to TimeZero (first 1,000 questions).

The second compares pairs of models whose accuracy gains lie within about one point of each other. "Correct under the mask" is, among questions that both the model and its base answer correctly, the share the model still answers correctly after the event frames are masked. Both behavioural columns use the first 1,000 test questions.

pairΔ[email protected]Γ (row 1: 600 new questions; row 2: first 200)correct under the mask (base → model)mismatched queries with no interval
IoU-GRPO / + our objective (two seeds)+15.9 / +15.3, +15.9+0.04 / +0.88, +0.920.29 → 0.32 / 0.29 → 0.23, 0.180.05 / 0.06 (seed 0)
SFT on 4,800 pairs, checkpoint after 800 / ours, 4,800 pairs+7.1 / +8.1−0.13 / +2.77 (+2.78 in the paper)0.27 → 0.39 / 0.27 → 0.200.09 / 0.37

At equal accuracy gain, the model with the higher Γ returns fewer correct answers once the event is removed, so more of its correct answers come from the event frames. Accuracy training alone moves this share the other way. Rejection of mismatched queries rises against the SFT checkpoint but not against IoU-GRPO, so we do not attribute it to Γ.

On the questions that a constant window fitted to the training labels cannot answer, our models keep Γ of +1.35 to +2.71 and SFT stays at +0.07, and our accuracy gain comes from these questions (reply to Reviewer 2, W1).

On QVHighlights your numbers are right. Our objective alone does not raise accuracy there, and the revision says so in the main text. In the re-run it gains +0.4 [−1.4, +2.3] points against +2.0 for SFT, where Table 17 gives +0.1 and +6.8 (base 0.319 in the paper, 0.407 in the re-run). Added to IoU-GRPO it keeps the QVHighlights gain (+11.2 against +12.2) and raises Γ there (+0.61 against +0.09).

Q2. Relative likelihood against fixed competing intervals

We added this readout, and its result is mixed. For each question the candidate texts are fixed across conditions. We measure how much more the annotated interval's log-likelihood advantage over a competitor shrinks under the evidence mask than under the placebo mask, relative to the base (first 200 questions). The values are summed over answer tokens, so they are not on the per-token scale of Γ.

competitorours, α = 0.5ours, 4,800 pairsIoU-GRPO + our objectiveIoU-GRPO, 2 fpsSFT, 4,800 pairs
the base model's own prediction+1.25 [+0.16, +2.26]+2.42 [+1.17, +3.69]+1.42 [+0.11, +2.64]+0.09+0.75 [+0.04, +1.37]
annotated interval shifted by its length+4.33 [+3.50, +5.14]+4.07+10.70 [+8.12, +12.92]+1.87−0.47 [−1.15, +0.19]

Against the base model's own prediction, the kind of competing prediction you suggested, the separation is small. Our 4,800-pair model is above SFT on the same base (+2.42 against +0.75), though their intervals overlap slightly, and the interval of our α = 0.5 model overlaps SFT's. Against the shifted interval our models separate clearly, but IoU-GRPO alone also reaches +1.87, so this row partly reflects accuracy training. We report both rows in the revision.

AuthorsOpenReview ↗

Reply to Reviewer 3 (2/2): refusal-aware training; other samples; presentation

W2 and Q3. RaTSG, NA-VMR and RA-RFT

Thank you for these references. The revision adds them to the related work. Of the two we could audit, RA-RFT leaves Γ unchanged and the released NA-VMR model raises it, so refusal-aware training need not raise Γ. The three methods train on queries that do not match the video and learn to reject them. Our queries always match the video. We change the video, compare the likelihood of the same answer when the event frames or as many other frames are removed, and our training uses no mismatched query and no rejection label.

RA-RFT is a generative MLLM like ours, so we audited it and its base Time-R1 with the same protocol. Its Γ is −0.003 [−0.022, +0.016] on 600 new questions. When every model is forced to give an interval, the evidence-minus-placebo difference in [email protected] is the same as its base's (difference 0.0 [−1.7, +1.7] points), while RA-RFT gives no interval for 84% of mismatched queries under its released prompt (Time-R1 under the same prompt: 72%; this counts any answer without an interval, unlike the verbatim refusals of Table 21). IoU-GRPO with our objective shows the reverse, with Γ +0.88 and no interval for 5.7% of mismatched queries (its base: 29%). Rejection and Γ can therefore come apart.

NA-VMR is a clip-feature model, so we ported the audit to feature space (replacing clip features with those of another video) and read the released UniVTG and UniVTG-NA checkpoints. There the negative-aware model does depend more on the event clips. Its Γ, computed on the mean log foreground probability of the event clips, is +0.55 [+0.47, +0.63] on the 1,883 questions both checkpoints answer correctly (+0.35 over the 3,625 questions whose event spans at most half the video), and its rejection head fires 27.8 points more often when the event clips are replaced than when other clips are (15.4 over all questions). The two released checkpoints also differ in learning rate and starting point, so we attribute this to the released model and make no claim about negative training alone. RaTSG is also feature-based. Trained on queries paired with other videos, it scores whether a query is unrelated to the video, next to a foreground score for each clip, and declines when that score is high. The score asks whether the queried event is in the video at all, and Γ asks how much the answer to a matching query rests on the event's frames, so a model can be strong on one and weak on the other, as RA-RFT is. The audits appear in a new appendix section.

Q4. Other samples

Reviewer 1 asked the same question, and the full table is in our reply to them (part 2/2). We drew 600 new questions at random from test videos outside the 79 of the first 200. They cover 484 videos, and the base reaches [email protected] 0.507 on them, as on the full split on this stack. The conclusion that accuracy-oriented training does not raise Γ holds on them. SFT gives −0.07 [−0.10, −0.04] and mDPO-style −0.01. IoU-GRPO gives +0.04 and −0.07 at 2 fps (two seeds) and 0.00 with the paper's 16 frames, the public TimeZero and Time-R1 checkpoints give −0.02 and +0.05, and the public TimeLens-8B gives −0.14 against its Qwen3-VL-8B base. Our objective gives +1.50 and +1.49 for two seeds, and +1.89 and +0.91 with α = 0.25 and 1.0. On TimeLens-8B it gives +1.42. On three disjoint random subsets of 200 of these questions, SFT gives −0.06, −0.08 and −0.08 and our model +1.45, +1.43 and +1.64.

Presentation

In the revision, Section 2 carries one question through ψ, the placebo and Γ as a running example. The notation table moves forward and keeps only the symbols the main text uses. Every abbreviation and unit (coverage, resolution, 16 frames and 2 fps, nats per token and points) is defined where it first appears, and each table caption names its sample and units. The training objective is stated in words before its equation.

If the RA-RFT and NA-VMR audits and the new sample address your concerns, we hope you will take them into account in your assessment. We are glad to run more checks during the discussion.

AuthorsOpenReview ↗

Reply to Reviewer 3, round 2: where your points are in the revised PDF

The revised PDF and the anonymous release are now up (see the general comment). Numbers below refer to the revised PDF.

  • W1 and Q1: a higher Γ is not a gain in accuracy, and the last paragraph of Section 4 now states the accuracy cost, including on QVHighlights. What it adds is that more of a model's correct answers rest on the event. In Table 38 (F.17), IoU-GRPO with our objective matches IoU-GRPO's accuracy gain, and its evidence share, the part of the placebo-masked accuracy lost to the evidence mask, rises over the base by 12.8 and 15.8 points for two seeds. Accuracy training alone changes it by +3.4 [−1.8, 8.7] for IoU-GRPO, an interval that includes zero, and by −7.5 [−11.9, −2.9] for the SFT checkpoint. The end of F.17 covers TimeZero, a base that already uses the event. There our objective leaves accuracy where it was (+0.5 [−0.7, 1.5]), raises Γ to +1.40 (Table 3) and lowers the rate of answering an all-black video from 0.968 to 0.303 (Table 10).
  • Q2: F.16 and Table 37. At the same accuracy gain, IoU-GRPO with our objective lies above IoU-GRPO alone against both fixed competitors, clearly against the shifted interval (+10.70 against +1.87) and with overlapping intervals against the base's own prediction (+1.42 against +0.09).
  • W2 and Q3: a paragraph in Section 6, and the audits of RA-RFT in F.14 (Table 35) and NA-VMR in F.15 (Table 36). RA-RFT gives no interval for 84% of mismatched queries against 72% for Time-R1, and also for more of the questions' own queries (19.8% against 11.3%), while its Γ stays at zero, so rejection training and evidence dependence come apart. F.15 adds the mask operator. On the questions both checkpoints answer correctly, the set on which Γ is defined, the released NA-VMR model depends more on the event clips under both edits (+0.55 under swap, +0.28 under the mask); over all audited questions it does so under swap only.
  • Q4: F.3 and Table 24, 600 questions from videos outside the first 200, with three disjoint subsets. Section 2 points to them through Appendix F.
  • Presentation: a running example and a notation table in Section 2, the objective in words before its equation in Section 4, and samples and units in the captions of Tables 1–3 and Figures 1, 2 and 4.

If anything is still unclear, please let us know.

Reviewer_JAVROpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Video temporal grounding is scored by Recall at IoU, which gives the same credit to a correct interval whether the model found the event, answered blind, or fit one annotation style. The paper measures evidence dependence — how much the answer rests on the event's frames — with a fixed-answer audit that removes either those frames or, as a placebo, as many frames elsewhere in the same video. On Charades-STA, supervised fine-tuning and several public checkpoints raise accuracy over the base model, but their answers rest on the event no more than the base's. A placebo-controlled objective raises that dependence, and the audit is repeated with reworded queries, unfamiliar corruptions, a second dataset, and a second backbone.

Critical strengths
  1. The question is well posed: the measure separates using the video from fitting the annotation, and the placebo control makes that separation testable.

  2. The paper tests its own measure rather than asserting it: it asks the same questions in different words, corrupts frames in ways the models never saw in training, and repeats the audit on a second dataset and a second backbone.

Critical weaknesses
  1. The audit credits only one route to a correct answer. The paper's fixed ten-second window cannot separate a model that exploits an annotation prior from one that reasons over the sequence, or from one that overfits the training split. The conclusions are then stated about evidence in general, while what is measured is dependence on the annotated frames alone.

  2. The method raises dependence while lowering accuracy. In domain it adds a few points, fewer than plain supervised fine-tuning on the same base, pairs and steps; on the second dataset the gain disappears; applied on top of the strongest public checkpoint it is negative in domain. The paper reports each of these results and weighs none of them.

  3. The paper states that declining a black video is not evidence dependence, and its appendix records that for the Qwen models the answers lost to the mask are refusals. The dependence rises, but the paper does not explain what the model came to rely on, so it cannot be said what the training taught.

Questions for authors
  1. After training with the loss, does the gain come from genuinely understanding the event, or merely from learning to answer with a refusal as soon as the input frames are blacked out? With the placebo control in place, how do you show the model has not learned a conditioned reflex to decline? For the questions that are still answered with an interval under the mask, does the score still hold?

  2. The audit's resolution comes from two pairs of seeds of one model under one prompt, and the placebo bound's constant from a single earlier experiment at one frame count and resolution. Do both survive genuine occlusion, and does a ten-percent jitter of the annotated boundaries change which values count?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer 2 (1/3): annotation prior, context and the training split

Thank you for the review. We tested each alternative explanation you raise with a new analysis. As in the general response, each new model is compared with its own base re-run on the same stack, and Γ is read under the mask on the 600 new test questions unless stated.

W1. What the audit credits

The fixed ten-second window is a model-free baseline that the introduction uses to show the annotation prior (p1 l.38–39; Table 7, last row). The audit itself edits each question's own annotated interval y⋆ (p3 l.126–128), whose length varies from question to question. The paper treats the other routes to a correct interval with separate readouts: answering black videos (Table 10), re-annotation (Tables 11–12) and the Charades-CD re-split (Table 18). We now test the three routes you name directly.

Annotation prior. We fitted the single window, as a fraction of video length, that best matches the 12,408 training labels. This window, [0, 0.47], uses neither video nor query and reaches [email protected] 0.27 on the test split. On the questions this window cannot answer (IoU below 0.3), our models keep Γ of +1.35 [+1.16, +1.53] (seed 0), +1.42 (seed 2) and +2.71 (4,800 pairs), while SFT stays at +0.07, mDPO-style at +0.14 and IoU-GRPO at 2 fps at +0.13. Our accuracy gain also comes from these questions (+5.1 points, against +0.4 on questions the window answers). Our predictions move slightly away from the window. Those of SFT, mDPO-style and the paper's 16-frame IoU-GRPO move slightly towards it, and those of IoU-GRPO at 2 fps do not move. The questions the window cannot answer differ from the others in where their events sit, so we read this comparison as directional only.

Context around the event. A model that reasons from the surrounding sequence would depend on the frames next to the event. We moved the edited window by one event length, onto the frames just before or just after the event, and kept the scored answer unchanged. The base does not depend on these frames (ψ +0.01 before, −0.07 after), and neither does our model after training. Its Γ is +0.03 [−0.01, +0.07] before and −0.01 [−0.08, +0.06] after, against +1.50 [+1.37, +1.64] on the event itself. IoU-GRPO with our objective added shows the same pattern (+0.88 on the event, 0.00 before, −0.05 after). Frames farther from the event are the ones the placebo mask edits, and the trained models keep their placebo drop within 0.10 of their base's (p7 l.359–360).

The training split. On the 742 Charades-CD test-ood questions that coincide with our test questions, every model we tested keeps a positive gain over its base (SFT +7.0, ours +2.8 points). On the 210 test-iid questions the gains are +19.5 and +8.1, so both keep about a third of their gain out of distribution. A higher Γ therefore does not come with more reliance on the training split, and our objective does not transfer better than SFT either (Table 18 gives each model's accuracy on both splits).

We agree that what is measured is dependence on the annotated frames. The revision says so in the definition in Section 2 and replaces the places where the paper used "evidence" more loosely.

AuthorsOpenReview ↗

Reply to Reviewer 2 (2/3): accuracy; refusal versus dependence

W2. Accuracy

We agree that the paper reported these costs without weighing them. Used alone, the objective buys dependence with accuracy. In domain it gains less than half of SFT's points on the same pairs (+3.9 against +9.1), and on QVHighlights it gains nothing (+0.1 against +6.8 for SFT in Table 17, +0.4 against +2.0 in our re-run). The revision adds a paragraph that says this, moves the QVHighlights accuracy and VideoChat-R1's in-domain loss into the main text, and states that the objective targets dependence on the event frames, with accuracy as a separate axis. The new results below weigh this cost.

  • Equal accuracy. The checkpoint of SFT on 4,800 pairs after its first 800 pairs comes closest in accuracy gain to our 4,800-pair model (+7.1 against +8.1 points). It has Γ −0.13, against +2.77 for ours (first 200 questions, re-run; +2.78 in the paper).
  • Combined with IoU-GRPO. Adding our objective to IoU-GRPO (weight 0.01, same pairs and schedule) keeps IoU-GRPO's accuracy in domain (+15.3 against +15.9) and on QVHighlights (+11.2 against +12.2), and raises Γ to +0.88 in domain and +0.61 on QVHighlights, where IoU-GRPO alone stays at +0.04 and +0.09. This Γ is lower than that of the objective used alone (+1.50). A second seed gives +15.9 points and Γ +0.92.
  • Public checkpoints. The in-domain loss you mention is on VideoChat-R1 (−2.7 [−4.2, −1.2], Table 3), and the revision reports it in the main text. On TimeZero, about as accurate under its own template ([email protected] 0.677 against 0.668), our objective alone changes accuracy by +0.5 [−0.7, +1.5]. Added to IoU-GRPO and trained from TimeZero, it gives +1.0 [+0.0, +2.0] points with Γ +1.04 [+0.96, +1.12].

W3 and Q1. Refusal versus dependence

Table 20 (p32) shows that the answers lost to the mask are not all refusals. Our Qwen model returns no interval 21.5 points more often than its base, and among the intervals it still returns, the share that is wrong rises by 10.4 [5.4, 15.4] points. Our TimeZero and Time-R1 models decline slightly more often (+0.8 and +3.2 points) and return more wrong intervals (+6.6 each), and VideoChat-R1's template cannot decline (+15.0 wrong intervals; Table 20 and p8 l.407–409).

Five results separate a learned refusal of blacked-out input from dependence on the event.

  • The placebo mask blacks out as many frames as the evidence mask, drawn at random outside the event. Under it our model keeps its clean accuracy (0.523 → 0.558, Table 19). A reflex to decline blacked-out frames would fire here too.
  • Swap and patch scrambling black out nothing, and Γ is +1.41 and +1.52 (Tables 1, 15).
  • With the masked frames fixed and only the query changed to name another event, the contrast reverses sign. Its query-specific part, Γspec in Table 16, is +2.86 for our model and −0.04 for SFT.
  • The mDPO-style model learned to decline black videos (it answers 2.5% of them, Table 10), and its Γ stays at −0.03 (Table 1; p5 l.245–246).
  • IoU-GRPO with our objective reaches Γ +0.88 and almost never declines. With the event masked it returns no interval on 1.4% of questions (3.0% for a second seed), against 11.0% for its base, and loses its answers to wrong intervals instead.

We also forced every model to answer. Decoding starts with the answer tag and is constrained to the interval grammar, so the model cannot decline. On the first 1,000 test questions, masking the event then lowers our model's [email protected] by 15 [12, 18] points more than the placebo mask does, relative to the base, against 5 for SFT.

As for what training taught, the model came to rely on the frames of the event the query names. The added dependence sits on those frames with none on their neighbours (part 1/3), and it moves when the query names another event (Γspec above). It also holds when the query is replaced by a second human-written description of the same event (Appendix D.2).

Q1, second part. Questions still answered under the mask

Yes. The dependence holds on the questions the model still answers, and its declines add to it. For generation, the rise in wrong intervals above shows this. In the likelihood audit, restricted to the questions it still answers with an interval under the evidence mask (72 of the 108 first-200 questions that enter its Γ in our re-run), our model keeps Γ +1.02 [+0.80, +1.26], against +1.62 on all 108 (+1.63 on 112 in the paper). The 4,800-pair model keeps +2.57 and IoU-GRPO with our objective +0.83, while IoU-GRPO stays at +0.01 and SFT on 4,800 pairs at −0.13.

AuthorsOpenReview ↗

Reply to Reviewer 2 (3/3): occlusion, jitter, seeds, prompts and bound constants

Q2. Occlusion, jitter, seeds and prompts

Under both occlusions and a 10% jitter of the boundaries, no value changes class. With the paper's resolution and bound constant held fixed, our models stay above 0.23 and SFT stays below zero. We did not re-measure the seed resolution under occlusion.

Occlusion and jitter (first 200 questions, Γ under the mask). The TimeZero value in the first row is the paper's, and all others are re-runs.

conditionours, Qwenours, TimeZeroSFT, Qwen
annotated boundaries+1.62+1.40−0.09
each boundary moved in / out by 10%+1.10 / +1.74+1.06 / +1.53−0.05 / −0.10
shifted left / right by 10%+1.52 / +1.28+1.27 / +1.27−0.12 / −0.04
event frozen on the frame before it+1.19+1.08−0.03
grey box, 60% of each event frame's height and width+0.60+0.57−0.03

Every new seed pair stays inside the paper's resolutions. Across prompts, SFT, IoU-GRPO and IoU-GRPO with our objective keep their class, and the standalone objective does not.

More seeds. A new seed of our Qwen model gives Γ +1.49 on the 600 new questions, against +1.50 for the original seed. Audited against each other, the two differ by −0.02 [−0.08, +0.03] under the mask and +0.22 [+0.16, +0.28] under the swap, where the paper's resolutions are 0.23 and 0.31 (p3 l.142–143). A new seed of our TimeZero model gives +1.23 against +1.21, and the two seeds differ by 0.04 [0.02, 0.06] against each other. Seed against seed under each unseen operator, the intervals reach 0.030 (noise), 0.049 (blur) and 0.065 (patch scrambling). Against a resolution this small, some accuracy-oriented models would show small effects of either sign, at most 0.16 in size, while the smallest mask Γ of a placebo-controlled model at 2 fps in the paper is +0.32 (Table 14).

Other prompts. We audited the models under two rewordings of the instruction (same answer format) and under VideoChat-R1's answer format (first 200 questions):

Γ (mask)trained instructionrewording 1rewording 2VideoChat-R1 format
SFT−0.09−0.04−0.08−0.04
IoU-GRPO, 2 fps+0.01+0.05−0.09+0.08
ours, seed 0+1.62+1.24+1.15+0.10
ours, seed 2+1.62+0.45+0.67 (n = 44)+0.20
IoU-GRPO + our objective, seed 0+0.85+0.82+0.81+0.31
IoU-GRPO + our objective, seed 1+0.96+0.92+0.91+0.31

SFT and IoU-GRPO stay within 0.1 of zero under every prompt, so the paper's negative result holds under the two rewordings and the second answer format we tried. The standalone objective is less stable. Under the two rewordings its Γ falls from +1.62 to between +0.45 and +1.24, and its two seeds, audited against each other, differ by up to 0.78 [0.65, 0.89] (rewording 1; 0.38 under rewording 2). Against that prompt's own seed resolution, seed 2's +0.45 does not count as an effect. The revision therefore gives Γ under each prompt against a seed pair measured under that prompt.

IoU-GRPO with our objective is steadier. Audited against each other, its two seeds differ by +0.05, +0.04 and −0.01 under the three new prompts, with every interval inside ±0.14, and both reach Γ +0.31 under VideoChat-R1's format (from +0.85 and +0.96). Under that format the standalone objective keeps little likelihood Γ, while the behavioural difference carries over. With answers regenerated in that format, masking the event lowers our model's [email protected] by 15 [10, 20] points (seed 0) and 17 (seed 2) more than the placebo mask does, relative to the base, against 0.6 for SFT.

Under rewording 2, our seed-2 model declines to give an interval on 65% of clean questions (seed 0: 18%; base: 0.5%), which is why only 44 questions enter its Γ. Forced to give an interval under rewording 2, it localises as well as SFT ([email protected] 0.59 against 0.60; base 0.50), and masking the event lowers its [email protected] by 22 [17, 28] points more than the placebo mask does, relative to the base, against 2 for SFT. Across eight instruction wordings, IoU-GRPO with our objective gives an interval on at least 99% of clean questions, and under rewording 2 it keeps both Γ (+0.81) and the behavioural difference (+20 [+14, +27] points).

Bound constants. The constants came from a setup with 16 frames at 224 px (Appendix C.2). Table 2 re-tests the bound at 2 fps and 3,584 × 28 × 28 pixels. Without it the placebo drop is 0.49, and with it 0.01 (p7 l.350–352). The five adapters of Table 3, one per base, keep their placebo drop within 0.10 of their base's (p7 l.359–360). Two further settings of the bound's ratio and slack, (1.0, 0.05) and (1.5, 0.20) against the paper's (1.25, 0.10), give Γ +1.57 [+1.42, +1.70] and +1.60 [+1.47, +1.72] on the 600 new questions, against +1.50 for the paper's setting, with accuracy gains of +3.8 and +4.0 points against +3.9.

Reviewer_up1ZOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks whether a temporal-grounding model's correct interval actually depends on the relevant video frames. Its audit edits frames inside the annotated interval and an equal number elsewhere, then compares the resulting likelihood changes before and after training. Several models trained for localization accuracy improve [email protected] without a comparable increase in this measure. The authors add an evidence margin, a placebo contrast, and a constraint on sensitivity to placebo edits during training. They test the resulting models with fixed-answer audits, regenerated answers, and query-switching experiments.

Critical strengths
  1. The matched evidence/placebo edit is a good way to test a claim that ordinary localization accuracy cannot settle. Holding the scored answer fixed avoids turning a change in wording into an apparent evidence effect. The ablation is especially useful: optimizing the evidence edit alone also increases sensitivity to irrelevant edits, whereas the full objective largely avoids that shortcut (Table 2).

  2. The paper checks whether dependence follows the question, not just whether masking some frames lowers likelihood. In the query-switching test, the video and edit locations stay fixed while the named event changes. The shift toward the newly relevant frames gives the audit more force than a single deletion score (Figure 5 and Table 16).

Critical weaknesses
  1. The claim about accuracy-oriented training is broader than the controlled IoU comparison supports. The paper's own IoU-GRPO model trains on 16 frames per video but is audited at two frames per second. The authors show that even their proposed objective has little measured effect under that same training/audit mismatch (Table 13). The public accuracy-trained checkpoints are interesting observations, but their data and training recipes differ, so they cannot isolate the effect of optimizing IoU.

  2. The new behavior transfers unevenly. For the Qwen2.5-VL variants, the gains under unseen noise and Gaussian blur are only 0.07–0.16 nats per token, within the paper's stated mask-audit resolution (Table 15). On QVHighlights, clean localization accuracy is flat or lower for the reported placebo-controlled models (Table 17). Table 20 also shows that, for models with answer tags, much of the increased dependence lies in the tags and first answer token rather than the timestamp tokens. That may be useful abstention behavior, but it is a narrower result than improved temporal boundary grounding.

Questions for authors
  1. Can you compare IoU-GRPO and the proposed objective with matched training frames, data, and compute? This would clarify how much of the near-zero IoU-GRPO result comes from the objective and how much from the frame-sampling mismatch.
  2. The likelihood audit uses the first 200 Charades-STA test questions, covering 79 videos. Do the main evidence-dependence comparisons hold on an independently sampled set of videos?
6 comments
AuthorsOpenReview ↗

Reply to Reviewer 1 (1/2): matched IoU-GRPO; noise and blur

Thank you for the careful reading. We ran new experiments for both of your questions and narrowed the claims you found too broad. As in the general response, each new model is compared with its own base re-run on the same stack, where the Qwen2.5-VL-7B base reaches [email protected] 0.507 (0.480 in the paper). Γ is read under the mask, on 600 new questions (Q2 in part 2/2) unless stated.

W1 and Q1. IoU-GRPO with matched frames, data and compute

Matched in frames and pairs, with one pass over the same 2,400 pairs, IoU-GRPO gains about 16 points and leaves Γ within 0.07 of zero. We trained IoU-GRPO at 2 frames per second, the audit's rate, on the same base and the same 2,400 pairs as our objective (in the same order for seed 0), with the same LoRA (rank 16) and software stack. With the 16-frame runs of Table 13 this gives a two-by-two comparison over training frames and objective. Only the 2 fps column is matched in data. The 16-frame IoU-GRPO of Table 13 used the full training split for 700 steps with LoRA rank 64 (Appendix E.1).

objectivetrained on 16 frames (Table 13, first 200 questions)trained at 2 fps (new, 600 new questions)
IoU-GRPOΓ 0.00Γ +0.04 [+0.02, +0.07] and −0.07 [−0.10, −0.05] for two seeds; Δ[email protected] +15.9 and +15.8
ours, α = 0.5Γ +0.06 (2,400 pairs; +0.23 on all 8,434)Γ +1.50 [+1.37, +1.64]; Δ[email protected] +3.9

Moving to 2 fps raises Γ only under our objective, so the frame-sampling mismatch does not explain the near-zero Γ of IoU-GRPO.

We also matched compute. On one A100, three runs of our objective on this schedule (variants in the bound or in the scored tokens) took 7.4–7.6 h each (300 steps of 8 pairs), and IoU-GRPO took 15.1 h (600 steps of 4 questions). We kept the step-300 checkpoint of the second IoU-GRPO seed. At the pace of the first seed's run it has used about 7.5 h, close to our time, and it has seen half of the 2,400 pairs. It has already gained 14.7 points, and its Γ is −0.06 [−0.08, −0.03].

A last run holds the IoU-GRPO setting fixed and changes only the loss. Adding our objective to the IoU-GRPO loss (weight 0.01, same pairs and schedule) keeps the accuracy gain (+15.3 and +15.9 points for two seeds) and raises Γ to +0.88 [+0.81, +0.94] and +0.92 [+0.85, +0.99].

We agree that the public checkpoints cannot isolate the IoU reward. The revision replaces the sentence in Section 3 (p4 l.204–207) saying they "show what IoU-reward training does to Γ", and its echo in Appendix E.1, with the controlled runs above, and keeps the public checkpoints as observations.

W2 (first part). Noise and blur

We agree with your reading of Table 15, and the revised abstract's "unseen edit" names patch scrambling. At the paper's settings (noise σ 40 in 8-bit pixel values, blur σ 3 pixels), noise and blur remove little even from the base (ψ 0.06 and 0.03 on the 600 new questions), so they leave little to depend on. Our Γ grows with the strength, and SFT stays at or below zero:

operator (σ)ours, α = 0.5SFT
noise 40 (paper) / 80 / 120+0.28 / +0.58 / +0.85 [+0.74, +0.97]−0.04 / −0.08 / −0.10
blur 3 (paper) / 6 / 10+0.25 / +0.59 / +0.97 [+0.85, +1.09]−0.01 / −0.02 / −0.02

Our model trained on 4,800 pairs reaches +1.47 (noise, σ 120) and +1.78 (blur, σ 10).

The 0.23 resolution was measured for the mask, as the largest interval end over two seed pairs. Measured for each unseen operator with one seed pair (seed 2 of our Qwen model audited against seed 0), the resolution is 0.030 for noise, 0.049 for blur and 0.065 for patch scrambling. By the paper's rule that an estimate beyond the resolution counts as an effect (p3 l.143), all six values of our Qwen models under noise and blur in Table 15 (first 200 questions, +0.07 to +0.16) count as effects, although one interval (α = 1, noise, +0.07 [−0.01, +0.14]) includes zero. At this finer scale some accuracy-oriented models also show small effects of either sign, at most 0.16 in size (reply to Reviewer 2, part 3). At the paper's settings on the 600 new questions, ours reaches +0.28 [+0.21, +0.35] and +0.25 [+0.18, +0.32], against −0.04 and −0.01 for SFT. The revision reports noise and blur at all three strengths, each against its own resolution.

AuthorsOpenReview ↗

Reply to Reviewer 1 (2/2): QVHighlights accuracy; tags and timestamps; an independent sample

W2 (second part). QVHighlights accuracy

You are right that our objective alone does not raise accuracy on QVHighlights. Table 17 gives +0.1 [−1.7, +1.8] points for our Qwen model against +6.8 for SFT, and −1.5 to −4.0 points for the other four placebo-controlled models. Re-run on our current stack with every model, where the base reaches [email protected] 0.407 rather than 0.319, our model gains +0.4 [−1.4, +2.3] against +2.0 for SFT, with Γ +1.19 [+1.00, +1.38] against +0.02. The revision reports this accuracy next to Γ in the main text.

The cost depends on what the objective is combined with. The model that adds our objective to IoU-GRPO (part 1/2) gains +11.2 [+9.0, +13.5] points on QVHighlights, against +12.2 for IoU-GRPO alone, and its Γ there is +0.61 [+0.52, +0.71] against +0.09. The revision presents this combination as the way to keep accuracy and transfer while adding the dependence.

W2 (third part). Tags and timestamps (Table 20)

This holds for tagged templates. For our Qwen model, Γ is +0.79 on the onset (the first token after the opening tag) and +0.86 on the tags, against +0.07 on the timestamps. The revision limits the framing-token paragraph of Section 5 to tagged templates, and the paper's behavioural claim is already the narrower one, that trained models "decline or miss the event" when its frames are masked (abstract, p1 l.25–26; contribution (iv), p3 l.110–112). For these models the dependence we measure sits mainly on the onset and the tags. The results below concern the timestamps themselves.

  • A variant that applies the margin to the timestamp tokens only reaches Γ +3.17 on the timestamp tokens and +1.52 [+1.35, +1.70] over the whole answer (first 200 questions; +1.52 [+1.43, +1.61] on the 600 new ones), with a similar accuracy gain (+3.8 against +3.9).
  • Against a competing interval written in the same template, so that the tags cancel, the evidence mask still costs the annotated interval more than the placebo mask does, relative to the base. Summed over the answer tokens, the effect for our α = 0.5 model is +4.33 [+3.50, +5.14] nats against the annotated interval shifted by its length, where SFT on 4,800 pairs gives −0.47 and IoU-GRPO alone +1.87. Against the base's own prediction it is +1.25 [+0.16, +2.26], where SFT gives +0.75 [+0.04, +1.37], so this comparison separates them only weakly (reply to Reviewer 3, Q2).
  • With tags, our TimeZero model has Γ +0.34 on the timestamps. VideoChat-R1's template fixes its tags and leaves only the timestamps free, and trained there our objective puts all of its Γ on the timestamps (+1.40; Table 20). With the event masked that model misses the event 15 points more often than its base.
  • Under VideoChat-R1's format, our standalone Qwen model trained on the tagged template keeps little likelihood Γ (+0.10 and +0.20 for two seeds), while masking the event still costs it more correct answers than the placebo mask does (reply to Reviewer 2, part 3).

Q2. An independent sample of videos

We drew 600 questions at random from the test questions whose video is not among the 79 videos of the first 200. They cover 484 videos, and the base reaches [email protected] 0.507 on them, as on the full split on this stack (0.600 on the first 200 here, 0.560 in the paper). No model's Γ changes sign, and the placebo-controlled models stay far above the accuracy-oriented ones:

modelΓ, first 200 (paper)Γ, 600 new questions
SFT−0.09−0.07 [−0.10, −0.04]
mDPO-style−0.03−0.01 [−0.04, +0.02]
IoU-GRPO, 16 frames (paper)0.000.00 [−0.02, +0.02]
TimeZero / Time-R1, public, against Qwen2.5-VL−0.08 / +0.02−0.02 / +0.05
TimeLens-8B, public, against Qwen3-VL-8B−0.16−0.14 [−0.18, −0.11]
ours, α = 0.5, Qwen2.5-VL+1.63 / +1.48 (seeds 0 / 1)+1.50 / +1.49 (seeds 0 / 2)
ours, α = 0.25 / 1.0, Qwen2.5-VL+2.03 / +1.03+1.89 / +0.91
ours on VideoChat-R1 / TimeZero / Time-R1+1.43 / +1.40 / +1.46+1.45 / +1.21 / +1.29
ours on TimeLens-8B, α = 0.5 / 0.25 / 1.0+1.36 / +1.72 / +0.92+1.42 / +1.80 / +0.97
ours, 4,800 pairs+2.78+2.64

Seed 2 is a new training run on the current stack. Across α, Γ keeps the order it has in the paper, on Qwen2.5-VL and on TimeLens-8B. We also split the 600 questions into three disjoint random subsets of 200. Every model in the table keeps its sign in each subset except two whose Γ stays within 0.02 of zero (mDPO-style and the 16-frame IoU-GRPO). For our Qwen model Γ is +1.45, +1.43 and +1.64, and for SFT −0.06, −0.08 and −0.08.

If these runs settle your two questions, we would be grateful if you took them into account in your rating. We are happy to run further checks during the discussion.

AuthorsOpenReview ↗

Reply to Reviewer 1, round 2: where your points are in the revised PDF

The revised PDF and the anonymous release are now up (see the general comment). Numbers below refer to the revised PDF.

  • W1 and Q1: Section 3 now rests the claim about IoU training on IoU-GRPO trained at 2 fps on our 2,400 pairs, and keeps the public checkpoints as observations. Appendix F.2 and Table 23 give both seeds, the checkpoint matched to our training time, the training settings, and IoU-GRPO with our objective added, which keeps the accuracy gain and raises Γ to about +0.9. Table 23 also crosses the training frames with the objective. IoU-GRPO stays near zero at 16 frames and at 2 fps, while our objective rises from +0.06 at 16 frames to +1.50 at 2 fps, so the frame mismatch does not account for IoU-GRPO's Γ.
  • W2, noise and blur: F.4 and Table 25, three strengths of each operator, with the Γ values and the seed resolutions measured on the same 600 questions (the first-200 values of Table 15 have wider intervals). The Γ of our α = 0.5 model grows from +0.25 and +0.28 at the paper's settings to +0.85 and +0.97 at the strongest, beyond each operator's seed resolution (at most 0.05, measured at the paper's setting), while SFT stays at or below zero. The 0.16 we quoted for accuracy-oriented models was a mask value, and the PDF does not use it.
  • W2, QVHighlights: the last paragraph of Section 4, with the re-run and the combined model in Table 23.
  • W2, tags and timestamps: the Section 5 paragraph is limited to tagged templates. F.13 applies the margins to the timestamps alone (Γ +3.17 on them), and F.16 and Table 37 score against fixed competing intervals.
  • Q2: F.3 and Table 24, 600 questions from videos outside the first 200, with three disjoint subsets. Section 2 points to them through Appendix F.

If anything is still unclear, please let us know.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response: new experiments and revisions

We thank the reviewers for careful reviews. We trained new models and ran new audits for the four questions they share. Does accuracy-oriented training leave Γ unchanged when frames, data and compute are matched (R1)? What does our objective cost in accuracy, and what does a higher Γ add (R1, R2, R3)? Is the rise in Γ a learned refusal (R2, R3)? Does the audit hold on other samples, seeds, boundaries and prompts (R1, R2, R3)?

Each new model is paired with its own base re-run on the same cluster and software stack, so absolute values can differ from the submission. Γ is the change in evidence dependence from a model's base, in nats per token. We read it under the mask on 600 new test questions covering 484 videos, drawn at random from videos outside the 79 of the paper's first 200 questions. Δ[email protected] is the paired change on the full test split, with 95% intervals from a video-level bootstrap. The paper's audit resolution is 0.23.

model (Qwen2.5-VL-7B base unless stated)Δ[email protected] (points)Γ
SFT, 2,400 pairs+9.1 [+7.5, +10.6]−0.07 [−0.10, −0.04]
IoU-GRPO at 2 fps, same 2,400 pairs, seeds 0 / 1+15.9 / +15.8+0.04 / −0.07
IoU-GRPO seed 1 at step 300 of 600 (about our training time)+14.7−0.06
ours, α = 0.5, seeds 0 / 2+3.9 / +4.5+1.50 / +1.49
IoU-GRPO + our objective (weight 0.01), seeds 0 / 1+15.3 / +15.9+0.88 / +0.92
TimeZero, then IoU-GRPO + our objective+1.0 [+0.0, +2.0]+1.04 [+0.96, +1.12]

With frames and pairs matched, IoU-GRPO adds about 16 points (14.7 at our training time) and moves Γ by at most 0.07 at either seed and at that time. Adding our objective to the same training raises Γ to about +0.9 and keeps the accuracy gain. On this base and these pairs, Γ therefore follows the training objective (R1).

Used alone, the objective gains less accuracy than SFT (+3.9 against +9.1) and none on QVHighlights. Added to IoU-GRPO it matches IoU-GRPO's accuracy in domain and comes within a point of it on QVHighlights (+11.2 against +12.2), at a lower Γ than alone (+0.88 against +1.50). On TimeZero it raises Γ without lowering in-domain accuracy. At matched accuracy, the model with the higher Γ returns fewer correct intervals once the event frames are masked, so more of its correct answers come from the event (R2, R3).

The rise in Γ does not depend on declining masked input. IoU-GRPO with our objective returns no interval on 1.4% of questions when the event is masked (its base: 11.0%) and loses its answers to wrong intervals instead. When decoding is forced to produce an interval, masking the event still costs our standalone model more accuracy, relative to the placebo mask, than it costs SFT (R2). RA-RFT, trained to reject mismatched queries, gives no interval for 84% of them (its base: 72%) and leaves Γ unchanged, so rejection and evidence dependence can come apart (R3).

On the 600 new questions no model's Γ changes sign from the first 200, and every placebo-controlled model stays far above every accuracy-oriented one. A 10% boundary jitter changes no classification, and under both occlusions we tried our models stay above 0.23 while SFT stays below zero (R1, R2).

The reviews also led us to a limit of the standalone objective. Under reworded instructions its likelihood Γ falls and its two seeds disagree more, and under one rewording one seed often declines to answer. Under another model's answer format its likelihood Γ nearly vanishes, although the behavioural difference remains. Added to IoU-GRPO, the objective keeps Γ under both rewordings and retains +0.31 under VideoChat-R1's format. Its two seeds agree under each new prompt, and it answers under all eight wordings we tried (reply to R2, part 3).

Revisions in the PDF (uploaded by 14 November AoE, with changes marked in colour)

  • Abstract: "unseen edit" names patch scrambling. The appendix reports noise and blur at three strengths, each against its own resolution.
  • Section 3 (p4 l.204–207) and Appendix E.1: the claim about IoU training rests on the controlled 2 fps runs, and the public checkpoints are reported as observations.
  • A paragraph weighing the accuracy cost, with QVHighlights accuracy and VideoChat-R1's in-domain loss in the main text, and IoU-GRPO plus our objective as the recipe that keeps accuracy.
  • Section 5 (p8 l.401–409): the framing-token paragraph is limited to tagged templates.
  • Section 2: "evidence" is defined as the annotated event's frames, and looser uses are replaced.
  • Presentation (R3): a running example, a shorter notation table moved forward, terms and units defined at first use, captions naming sample and units, and the objective in words before its equation.
  • Γ under each prompt, against a seed pair measured under that prompt.
  • Related work on RaTSG, NA-VMR and RA-RFT, with the RA-RFT and NA-VMR audits.
  • Appendix sections for every new experiment.
AuthorsOpenReview ↗

Revised PDF uploaded

Added and rewritten text is in blue; removals that bear on our replies are named below. Tables 1–22 and Figures 1–11 keep their numbers; page numbers below refer to the revised PDF, and those in our round-1 replies to the submitted one. The main text stays within ten pages. The new experiments are in Appendix F, Tables 23–38. Our replies number seeds from 0, so seed 2 of the placebo-controlled Qwen model is its "third seed" in Appendix F.

The code, the question and training-pair identifiers, and the LoRA adapters of the models we train are released anonymously at https://huggingface.co/iclropen (code: https://huggingface.co/iclropen/evidence-audit-code), in addition to the supplementary material.

Main text

  • Abstract: "unseen edit" now names patch scrambling; it adds that IoU-reward training on our data at the audit's frame rate does not raise Γ, and that our objective added to IoU-reward training keeps the accuracy gain and raises Γ.
  • Section 1: patch scrambling, IoU-GRPO with our objective, and the objective alone gaining fewer points than SFT. The masking sentence now gives the forced-decoding result (15 points of [email protected]) in place of the refusal rate, which stays in Table 20. It states the units of accuracy changes and which models we trained, and says the dependence "moves with the query".
  • Section 2 (p3–4): "evidence" is defined as the frames inside the annotated interval, with pointers to F.5–F.7 (annotation prior, frames next to the event, training split). One question is followed through ψ, the placebo and Γ, and a notation table is added on p4. A pointer to Appendix F covers the 600 questions from other videos, occlusion, new seeds and other prompts. The heading "Three signatures of evidence use" now reads "evidence dependence".
  • Section 3 (p4): the sentence saying the public checkpoints "show what IoU-reward training does to Γ" is replaced by the controlled 2 fps IoU-GRPO runs (F.2). Table 1 labels its IoU-GRPO row "16 frames".
  • Section 4 (p6–8): the objective is stated in words before its equation, and the source of the bound constants (16 frames at 224 px) is given. The accuracy range at α = 0.5 is given to one decimal (−2.7 to +20.9). A new paragraph reports VideoChat-R1's in-domain loss and the QVHighlights accuracy (no gain for our Qwen model against +6.8 for SFT in Table 17; −1.5 to −4.0 for the other placebo-controlled models). Added to IoU-GRPO, the objective keeps its accuracy and raises Γ; at the same accuracy gain it keeps fewer correct answers under the evidence mask (F.17); under reworded instructions it keeps its Γ, while two seeds of the objective alone differ by more than the resolution (F.11).
  • Section 5 (p8–9): the framing-token paragraph is limited to tagged templates and adds the timestamp-only variant (F.13). The masking paragraph adds the forced-decoding result (F.8). The paragraph on mismatched queries is now headed "The objective alone …", since IoU-GRPO with our objective declines fewer mismatched queries than its base. The noise and blur sentence ("mostly within the mask resolution") now says that their Γ is small and that on the 600 new questions the Γ of the Qwen model at α = 0.5 lies beyond each edit's seed resolution and grows with edit strength (F.4).
  • Corrections to the submitted text: the QVHighlights Γ range of the placebo-controlled models is +1.06 to +1.58, as in Table 17 (the submission said +0.92 to +1.58); VideoChat-R1's template fixes its answer tags and leaves only the timestamps free (the submission said it asks for a bare interval), in Section 5 and Appendix D.5.
  • Section 6 (p10): a paragraph on RaTSG, NA-VMR and RA-RFT (F.14, F.15).
  • Section 7: one sentence on adding the objective to IoU-reward training.
  • Captions of Tables 1–3 and Figures 1, 2 and 4 name their samples and units.
  • Reproducibility statement: the release above.

Appendix F (new)

  • F.1: the re-run software stacks and the 600 new test questions.
  • F.2: IoU-GRPO at 2 fps on the same 2,400 pairs with its training settings, the time-matched checkpoint, and IoU-GRPO with our objective, including its own placebo drop (R1, R2, R3).
  • F.3: the independent sample (R1, R3).
  • F.4: noise and blur at three strengths; the seed resolution of each operator is measured at the paper's setting on the 600 new questions (R1).
  • F.5–F.8: the annotation prior, frames next to the event, Charades-CD, and refusal against dependence (R2).
  • F.9–F.13: frozen and grey-boxed event frames and boundary jitter, more seeds, other prompts with two seeds each, the bound constants, and the timestamp-only margin (R1, R2).
  • F.14–F.15: the RA-RFT and NA-VMR audits (R3). F.15 adds the mask operator. On the questions both checkpoints answer correctly, UniVTG-NA's Γ is positive under both edits, and over all questions under swap only.
  • F.16–F.17: competing intervals, and the comparison at matched accuracy, with the change in evidence share (R1, R2, R3).
AuthorsOpenReview ↗

Summary of findings and contributions

We thank the reviewers and the Area Chair. As the discussion closes, we summarize what the paper finds and contributes. The paper asks whether a correct interval in video temporal grounding comes from the event's frames, a question Recall at IoU cannot answer.

What the paper finds

  • Accuracy and evidence come apart. Supervised fine-tuning and three public IoU-reward checkpoints raise the accuracy of Qwen2.5-VL-7B by 9 to 20 points, partly by answering blind and by fitting one annotation style, and their answers rest on the event's frames no more than the base model's. IoU-GRPO retrained on our data at the audit's frame rate shows the same.
  • Training on the evidence drop alone teaches a model to react to any edit of the video. A placebo contrast with a bound on the placebo drop removes this shortcut.

What the paper contributes

  • An audit of evidence dependence. It edits the event's frames or, as a placebo, as many frames elsewhere in the same video, and compares the drops in the likelihood of a fixed answer. It needs only an answer that can be scored and the location of its evidence.
  • A training objective built from the same quantity. It raises evidence dependence in all five base models we train, and added to IoU-reward training it keeps the accuracy gain.
  • Checks outside the likelihood. Masking the event's frames makes the trained models decline or miss the event more often than their base, and masking as many other frames does not. The dependence follows the event named in the query, holds on a second dataset, and holds under patch scrambling, an edit never used in training.

Where the discussion stands

  • Reviewer 3 raised the rating to 3 after the added experiments and asked that claims about the practical benefits of higher evidence dependence stay measured. The final version will follow this.
  • Reviewer 1 replied that they have no further concerns after the revision, which added the retrained IoU-GRPO baseline at matched frame rates (Table 23), graded noise and blur (Table 25) and the audit on 600 questions from new videos (Table 24).
  • On Reviewer 2's question of whether the gain reflects declining to answer, forced decoding separates evidence use from abstention (Section 5, Table 29).

Models and code are released anonymously. Thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.