acceptodds
Under review as a conference paper at ICLR 2027

TEMPORAL RESOLUTION IS DURATION-DEPENDENT IN GENERATIVE VIDEO GROUNDING

Abstract

Temporal resolution is usually treated as uniformly beneficial in video temporal grounding. We show instead that, for a strong generative grounder, its marginal value is signed and duration-dependent. After averaging over sampling phase, increasing density from 4 to 8 fps improves one-second events, degrades medium-duration events, and crosses zero at 2.3 s, while the aggregate effect is null. Controlled interventions narrow the source: the short-event gain does not require new visual content, since duplicating byte-identical frames reproduces and exceeds the gain from real densification, while granularity-matched permutation of the model's interleaved time markers largely attenuates it, so readable temporal addressing drives the effect. Matched fine-tunes with different duration compositions also shift the density-response crossing, consistent with a learned duration-sensitive prior. We therefore introduce Zoom-VTG, an inference-time, duration-gated coarse-to-fine policy that applies a denser second pass only when the coarse prediction is short. On held-out Charades TEST-400, it improves one-second mIoU by +0.172, and the gain transfers without retuning to ActivityNet and QVHighlights. On the complete, descriptively-accounted 3,363-query population, one-second mIoU improves by +0.137 while aggregate mIoU changes by only +0.001. These results indicate that temporal resolution should be allocated and evaluated conditionally on event duration, rather than treated as a single global hyperparameter.

Primary area:
generative models
Keywords:
video temporal grounding, multimodal large language models, temporal resolution, temporal sampling, duration-aware inference, adaptive computation, coarse-to-fine decoding, short-event localization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eq1COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether increasing temporal sampling density helps video grounding equally across event durations. Using the 4B version of TimeLens2, the authors find that increasing the sampling rate from 4 to 8 fps improves localization of short events but hurts events of intermediate duration. This interaction remains after averaging over sampling phases, while the overall change is not statistically significant. The authors investigate the effect through frame duplication, timestamp perturbations, changes to predicted interval geometry, and training interventions. They then introduce a policy that runs a second grounding pass on a local crop only when the initial prediction is short. The policy improves short event accuracy and transfers across datasets with fixed settings.

Critical strengths
  1. A useful empirical finding supported by a meaningful sampling control. The paper goes beyond showing that short events are difficult. It shows that increasing density can have opposite effects on different duration groups within the same model. The phase analysis makes this result more credible.

  2. The experiments examine several plausible explanations and expose the performance tradeoff. Frame duplication, timestamp perturbations, crop budget controls, and routing comparisons at matched activation rates are useful attempts to understand where the gains come from. The paper also reports the costs to other duration groups. Improving the weakest duration group can be worthwhile even when average accuracy barely changes.

Critical weaknesses
  1. The proposed mechanism does not yet explain the full interaction. Duplicating frames shows that the short event gain does not require new raw visual content. It still changes the visual representation. When two frames are merged into a temporal patch, duplication changes which frames are fused, the number of representations, and the input sequence. Part of the gain could come from this change in visual encoding. Permuting timestamps shows that the model depends on readable time information. Since permutation severely damages localization, the smaller duplication gain could also reflect that general loss of performance. The quantization and offset tests partly address this issue, but do not separate token count from marker granularity. Replacing both the center and length of a prediction simply gives back the complete 8 fps prediction. That result describes the outputs without identifying the process inside the model. The unexplained negative effect is my main concern. Section 4.2 states that interval contraction does not explain the full response after phase averaging. The loss on events of 3 to 10 seconds is an important part of the finding. Do duplication and timestamp changes also reproduce this loss? If not, the positive and negative effects may have different causes. All positive model replications also use TimeLens supervision. The result on original ActivityNet annotations helps address the role of test labels, but leaves the training source unchanged. I am convinced more by the observed interaction than by the claim of a shared mechanism across generative grounding models.

  2. The policy gains need the same phase check used for the main finding. In Section 4.1, the default 4 fps phase is the weakest tested phase for short events. Its mIoU is 0.3622, compared with a phase average of 0.3946. Table 2 still measures the main policy gain of 0.172 against that default prediction. The released configuration is a fair baseline. What is missing is evidence that the gain holds when the phase changes. A different first pass phase can change the predicted interval, the routing decision, and the second crop. The whole policy needs to be run again for each phase. Subtracting the phase averaged baseline from the existing policy score would not test this. Cropping changes context, the time origin, and the sampling grid as well as density. The effect of raising the full video rate from 4 to 8 fps need not match the effect of local refinement. The crop controls on other datasets are useful, but the connection is not yet clear for the main Charades result.

Questions for authors
  1. Does the policy gain hold across sampling phases? Are there results from running the whole policy with different first pass phases and all selected settings fixed? Please give the paired gains by duration group and the routing activation rates. On Charades, is there a crop control that keeps the original density, or one that preserves the absolute sampling grid?

  2. Do the controls also reproduce the negative effect? After phase averaging, does frame duplication hurt events of 3 to 10 seconds? Please give the factorial interaction estimates and their uncertainty. Separate significance tests for each condition do not test the difference between conditions.

  3. Does the effect occur without TimeLens supervision? Do the authors have results from a model that can output short intervals but has not used TimeLens training data? If no such results are available, would they limit the claim to the tested conditions?

  4. Why do the text and response curve give different values? Line 164 gives a gain of 0.059 at 2.1 seconds. The curve in Figure 1a appears close to zero at that point. Do these refer to different estimates, or does one need correction? Please explain how the curve and its zero crossing were estimated.

Reviewer_gUJqOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper observes that the impact of increasing temporal sampling density on video event localization is related to the duration of the events.

Critical strengths

The paper raises an interesting question.

Critical weaknesses
  1. The experiments conducted are highly limited in scope; the conclusion regarding the 2.3s inflection point cannot be generalized to broader scenarios.
  2. The experimental design fails to provide compelling evidence to isolate the specific variables responsible for the observed gains.
  3. The methodological innovation is very limited, and the improvements in experimental results are marginal.
Questions for authors

Could you demonstrate that the conclusions are valid across a wider range of models and datasets?

Reviewer_ZNQBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how increased temporal sampling density affects generative video grounding across event durations. It reports duration-dependent gains and losses, investigates possible explanations through sampling-phase controls, frame duplication, timestamp perturbations, and training interventions, and proposes Zoom-VTG, a duration-gated coarse-to-fine refinement strategy. Validation includes multiple datasets and additional adapted models.

Critical strengths

The paper documents a useful duration-dependent failure pattern in the evaluated grounder. On Charades TEST-400, increasing sampling density from 4 to 8 fps improves one-second-event mIoU by +0.0563 but reduces 3–10 s performance by −0.0507 after phase averaging, with both reported confidence intervals excluding zero (main Table 1). This establishes opposing responses that the near-zero aggregate change conceals, providing concrete evidence for examining performance across event durations.

Critical weaknesses
  1. The proposed mechanism is not adequately supported by the experiments. The paper attributes the benefit of denser sampling to improved temporal addressing. However, duplicating frames also changes token counts and how frame pairs are encoded, so the improvement could arise from changes in visual processing. Likewise, scrambling timestamps shows that incorrect timing information damages localization; it does not establish why denser sampling helps under normal conditions. These experiments show that new visual content is unnecessary and that timestamps matter, but they do not establish the causal connection between those observations. The main text should narrow its mechanistic claims or provide controls that separate timestamp effects from changes in visual encoding.

  2. The practical value of Zoom-VTG is not sufficiently validated. Supplementary Table 16 states that the full gate/crop/refinement pipeline was not tested across sampling phases, although phase sensitivity is central to the paper’s argument. A single-phase method cannot be fairly compared with phase-averaged baselines as evidence of robustness. At matched image counts, Zoom-VTG gains +0.062 on the shortest events, loses −0.035 on 3–10 s events, and shows no significant aggregate improvement. Given the limited novelty of adding a duration gate to coarse-to-fine refinement, stronger evidence is needed that this trade-off is stable and practically useful.

  3. The manuscript is repetitive, poorly organized, and overstates the evidence. The main text repeatedly presents a unified causal explanation, while important qualifications—such as integer-only outputs and the missing phase evaluation of the complete policy—are buried in the appendix. Cross-references also hinder verification: main-text p. 5 cites “Table 2” for predicted lengths, although that table reports method performance; the intended reference is supplementary. Some statistical claims are too strong: a center-only transplant is described as harmful despite a confidence interval including zero, and a nonsignificant offset effect is called “free.” The paper needs substantial restructuring, clearer claim–evidence alignment, and more prominent discussion of its limitations.

Questions for authors
  1. Clarification of causal evidence: Which experiment distinguishes a benefit from denser temporal addressing from a benefit caused by changes in visual-token counts or frame grouping? Please explain what supports this specific mechanism beyond showing that timestamps matter.

  2. Additional validation of the method: Does Zoom-VTG retain its short-event gains when the entire gate/crop/refinement pipeline is evaluated across first-pass sampling phases? Supplementary Table 16 leaves this unresolved. Please keep the policy parameters fixed, compare against a baseline matched to the actual image budget at each phase, and report paired confidence intervals for each duration group and overall. This would establish whether the reported gains and medium-event losses represent a stable trade-off.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.