- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2(+1)
Read review
Paper summary
The paper finds that the video helps video VLMs choose the start time more than the end time. It then identifies a small set of directions in a late layer that affects the end-time prediction, and shows this with causal interventions. For deployment, the method averages the projected differences over 256 unlabeled samples to get one direction for each model and dataset. At inference, it adds this direction once to the residual stream before the first end-time token. On TimeLens-Bench, the method improves performance for six backbones by about 2 mIoU on average.
Critical strengths
A careful causal analysis. The paper tests the endpoint subspace with various steering tricks and analysis methods. The finding that the video helps the start time more than the end time is interesting for the design of grounding models.
Critical weaknesses
- The paper is very difficult to read, which is my main concern. The text uses many long, coined noun phrases without definition, for example "Jacobian-guided low-rank projection framework", "canonical model-own-start prefix" and "readout-aligned temporal contrasts". Key notation is defined only once (P0 and P2 in Sec. 3), P1 is never defined, and abbreviations such as MAE are never expanded. Most paragraphs in Sec. 3 and 4 contain several undefined terms in a row. As a result, I could not follow the basic argument without the appendix.
- The Qwen3-VL baselines are much weaker than published results, and TimeLens is not compared. On the same benchmarks, TimeLens2 (Zhu et al., 2026, arXiv:2607.17423, Table 1) reports native mIoU of 47.2 / 48.0 / 59.4 for Qwen3-VL-8B on Charades- / ActivityNet- / QVHighlights-TimeLens, and 49.4 / 50.7 / 62.3 for Qwen3-VL-4B. Table 2 of this paper reports only 34.48 / 38.43 / 47.24 and 41.41 / 39.81 / 54.16. Even after steering, Qwen3-VL-8B (36.52 / 39.92 / 49.19) stays about 8 to 11 mIoU below the published native results. In contrast, the TimeLens2 baselines in Table 2 agree with the published numbers within about 0.5 mIoU, so the test split alone is unlikely to explain the gap.
- The test split is small, and the main results have no confidence intervals. The method uses 15% of the videos of each benchmark for testing (App. D.1), which gives a few hundred queries per dataset. The main gains are approximately 1 to 3 points when absolute performance is at 60~70 scale, i.e., minimal.
- The method needs in-domain calibration data. Each model and dataset needs its own unlabeled calibration set. When the calibration comes from a different dataset, most of the gain disappears (Table 17). This limits the practical value of the training-free claim, e.g, when a random video is provided.
- The deployed update does not use the evidence of the current video. The paper motivates the method as restoring temporal evidence from the video (Sec. 3.3). But at test time, it adds one fixed vector, averaged over the calibration set, to every query of a dataset. This vector does not depend on the current video; only its size scales with the hidden state. Thus, it acts more like a fixed correction than like evidence from the video. The paper does not show what this correction does to the predictions, for example whether it moves end times later or makes intervals longer.
- The method cannot fix a wrong start time. The method changes only the end time and keeps the generated start. The paper does not report the start-time error of the native predictions, so it does not show how often the start is wrong. Also, the gains do not seem to come only from better end times. Queries that the native model misses completely (IoU = 0) also gain (Table 14), although their wrong start stays. With the ground-truth start, the gains for the 8B models are generally smaller (Table 5) than with the model's own start (Table 2). This makes the usability of the proposed method doubtful.
Questions for authors
- Can you define all terms and notation at first use (for example P0, P1, P2, MAE and "canonical model-own-start prefix"), and describe the method in plain words before the formal construction ?
- Does the analysis in Sec. 3 hold for other benchmarks such as QVH-TimeLens, Activitynet-TimeLens , and Ego4D-NLQ?
- Why are your Qwen3-VL baselines about 8 to 13 mIoU below the results that TimeLens2 reports on the same benchmarks? Can you report native and steered results with the standard settings on the full TimeLens-Bench, and include TimeLens-8B?
- Can you give confidence intervals or paired significance tests for Table 2?
- What if the start timestamp is also wrong?
- How would you handle a random video during real-life scenarios with your method?
16 comments
Experimental coverage
Choice of TimeLens2. We prioritized TimeLens2 because its published evaluation reports stronger temporal grounding performance than TimeLens at the same 8B scale (TimeLens2, Table 1). This choice tests whether our training-free intervention can further improve a strong model already specialized through temporal-grounding post-training. Our evaluation includes both TimeLens2-4B and TimeLens2-8B. In Table 2, steering improves both mIoU and [email protected] for both sizes across all three datasets. On TimeLens2-8B, the mIoU gains are +1.83, +1.08, and +0.84 points on Charades-TimeLens, QVHighlights-TimeLens, and ActivityNet-TimeLens, respectively. These results support the complementarity of endpoint activation steering with temporal post-training.
Does the Sec. 3 analysis extend to other benchmarks? We extended the core diagnostics to QVHighlights-TimeLens (QVH) and ActivityNet-TimeLens (AN), using Qwen3-VL-4B, TimeLens2-4B, and InternVL3.5-4B. The key findings persist across all six settings.
Temporal information remains decodable. Separately fitted probes retain end-time information at , whereas transferring an unchanged probe increases normalized MAE in every setting. With Full/No-video/Roll probes fitted independently under the same native prefix, Full reduces normalized MAE by 0.04580–0.05576 relative to No-video and 0.00413–0.01969 relative to Roll. All 12 paired differences have 95% CIs above zero.
Start/end asymmetry persists. Within each model and comparison, we pair start and end on common eligible queries. Full uses mean numeric-token NLL; Visual uses the candidate-level cost difference . For each query and boundary, we compute Spearman correlation between cost and distance to the annotated boundary, then average the within-query difference . The 95% CIs use 10,000 video-level bootstrap replicates with query pairing preserved. Positive gaps indicate stronger start discrimination.
| Dataset | Model (4B) | Full start | Full end | Full [95% CI] | Visual [95% CI] |
|---|---|---|---|---|---|
| QVH | Qwen3-VL | 0.50533 | 0.04468 | +0.46065 [0.35498, 0.56632] | +0.107 [0.019, 0.195] |
| QVH | TimeLens2 | 0.10110 | 0.07150 | +0.02960 [0.025060, 0.19460] | +0.054 [0.005, 0.103] |
| QVH | InternVL3.5 | 0.33227 | 0.09678 | +0.23549 [0.15209, 0.31889] | +0.025 [0.002, 0.048] |
| AN | Qwen3-VL | 0.57601 | 0.43278 | +0.14323 [0.06342, 0.23042] | +0.408 [0.291, 0.526] |
| AN | TimeLens2 | 0.59325 | 0.58166 | +0.01159 [0.00040, 0.02280] | +0.160 [0.043, 0.277] |
| AN | InternVL3.5 | 0.39281 | 0.21414 | +0.17867 [0.05948, 0.30771] | +0.255 [0.101, 0.409] |
The visual start-minus-end gap is positive in all six settings, with every 95% CI excluding zero. Thus, weaker visual support for end-boundary discrimination extends beyond Charades-TimeLens.
Ego4D-NLQ. Due to time constraints, we were unable to complete the Ego4D-NLQ diagnostics within the rebuttal time window. The additional experiments above cover the other two benchmarks suggested by the reviewer, and we will consider including Ego4D-NLQ in future work.
Concurrent evidence. FinishStrong: Learning to End Well after Imperfect Start in Video Temporal Grounding, posted on arXiv on November 1, 2026, also reports systematically higher end MAE than start MAE across the evaluated model families (Fig. 1a; Appendix F). This concurrent observation supports the broader relevance of endpoint asymmetry. FinishStrong develops post-training for imperfect start prefixes; our work contributes a visual-evidence and activation-level diagnosis together with a training-free intervention in frozen models.
Changes in the revised manuscript. Section 3 (p. 3) now states the extended diagnostic scope. Section 3.1 (p. 3) summarizes the cross-dataset readout findings, and Figure 2(c) with Section 3.2 (p. 4) presents the visual-support gaps across all three datasets. Appendix A.1, Tables 7–8 (p. 14), gives the added readout and native-prefix video controls; Appendix A.2, Table 10 (p. 16), gives the full boundary comparisons and CIs, with the paired protocol on p. 15. Section 2 (p. 2) now cites FinishStrong. Revised text, tables, and figure captions are highlighted in red.
OMTG multi-interval boundary diagnostics
We extended the boundary-discrimination analysis in Sec. 3.2 to multi-interval queries from the RL subset (train split) of OMTG-56K, using frozen TimeLens2-4B. This adds a multi-interval setting to our cross-dataset diagnostics. We sampled 128 video sources across duration groups, selecting the first annotated event and one subsequent event per query. The analysis includes 206 eligible events from 125 sources. To compare the same event across conditions, we fixed preceding ground-truth intervals and the current interval's other ground-truth boundary.
We scored matched start/end candidates with and without video using the mean NLL of the moved boundary's numeric tokens. For each candidate, we computed the incremental visual cost , then its Spearman correlation with distance to the annotated boundary. Higher indicates stronger preference for candidates closer to that boundary. We computed start−end differences within events, averaged events within each source, and weighted sources equally. (Pointwise 95% CIs use 10,000 source-level paired bootstrap replicates.)
All three rows below use the same 206 events. The visual-contribution row measures the ranking of candidate-level cost differences.
| Scored information | Start ↑ | End ↑ | Start−end gap [95% CI] |
|---|---|---|---|
| Video + query | 0.751 | 0.708 | +0.043 [-0.008, 0.093] |
| Query only | 0.115 | 0.251 | -0.136 [-0.227, -0.041] |
| Visual contribution | 0.688 | 0.615 | +0.073 [0.016, 0.131] |
Adding video raises the start correlation from 0.115 to 0.751 (+0.636) and the end correlation from 0.251 to 0.708 (+0.457). Under target-boundary scoring, the visual start−end gap is +0.073 [0.016, 0.131], reproducing the direction of our Sec. 3.2 finding.
With the second predefined scoring rule, which includes both start and end digits of the current interval, the overall visual gap is +0.012 [-0.047, 0.070]. For first intervals, the visual gaps are +0.115 [0.019, 0.213] and +0.121 [0.025, 0.221] under the two scoring rules, respectively.
These results extend the endpoint-focused diagnosis to multi-interval examples under fixed ground-truth history. Video evidence supports both boundaries, with stronger start discrimination under boundary-specific numeric scoring; the first-interval results agree across both scoring rules. The positive target-boundary gap supports the relevance of endpoint asymmetry beyond single-interval queries.
Clarity, notation, and method overview
We sincerely thank the reviewer for carefully reading our manuscript and for the thoughtful, constructive feedback. We appreciate the time and effort devoted to reviewing our work.
We have revised the Introduction and Sections 3–4 to explain the purpose of each operation before its mathematical construction, define the notation at first use, and clarify the prefix protocols in Appendix A.1. Revised text and figure captions are highlighted in red.
Method overview before the equations. Section 3.3 now gives a concise five-step overview: compare original/rolled-video states under a common native prefix; filter each contrast through its own query's endpoint-sensitive directions; aggregate the filtered components into a model-specific subspace; calibrate a shared direction and strength on unlabeled target-dataset samples; and apply one update before end-time generation. The first four steps are offline. The following paragraphs explain the roles of endpoint-score gradients, query-specific projection, and singular value decomposition (SVD). Section 4 distinguishes the fixed subspace from the calibrated direction and presents direction calibration, strength calibration, and deployment separately. Algorithm 1 follows the same sequence.
Definitions at first use. Section 3 now uses Pans for the position before the first answer token and Pend for the position before the first end-time numeric token, replacing P0 and P2. These names directly identify the two positions. We also define the prefix, diagnostic probe, and mean absolute error (MAE), including normalization by video duration.
Canonical versus native prefixes. “Canonical” means standardized formatting of the model's own full-video-predicted start. For example, an illustrative native answer
The relevant interval is [[8.24, 14.6]].gives the canonical prefix[[8.2,, whereas the native prefix isThe relevant interval is [[8.24,. Neither the predicted end nor ground-truth timestamps are included in the canonical prefix. A diagnostic probe uses the final prefix token's hidden state to predict the annotated end. In the original Charades diagnostics, full-video, no-video, and rolled-video probe controls reuse the same canonical prefix, fixing the start and formatting across conditions. Subspace construction, causal interventions, and deployed steering retain the exact native token prefix, including the generated start and original formatting. Section 3.1 states this distinction; Appendix A.1 specifies the original protocol and the native-prefix controls used in the added cross-dataset experiments.Locations in the revised manuscript: Introduction (p. 2); definitions and probe protocol in Sections 3–3.1 (p. 3); method overview and construction in Section 3.3 (pp. 4–6); calibration and deployment in Section 4 (pp. 6–7); prefix protocols in Appendix A.1 (pp. 13–14); and Algorithm 1 (p. 19). These presentation revisions preserve the original results.