acceptodds
Under review as a conference paper at ICLR 2027

FinishStrong: Learning to End Well after Imperfect Start in Video Temporal Grounding

Abstract

A poor start lowers the ceiling, but does not negate the value of finishing well. In autoregressive video temporal grounding, models systematically exhibit an endpoint performance asymmetry, predicting ends significantly worse than starts. A committed flawed start acts as an immutable physical constraint, irreversibly capping the attainable overlap as a sunk loss. While models should minimize the remaining conditional regret, standard post-training paradigms fail to teach this adaptation. Specifically, teacher-forced SFT suffers from strict prefix exposure bias, and holistic trajectory evaluation in standard GRPO inadvertently penalizes conditionally optimal ends if they follow a poor start. To address this, we introduce \method, a novel two-stage post-training framework explicitly designed to teach optimal continuations. First, Online-start SFT exposes the end prediction to the model's own generated contexts. Second, Conditional-tree GRPO restructures the sampling topology, centering end rewards exclusively within the same-start branch to mathematically isolate the conditional end quality from the inherited start penalty. \method-9B achieves a state-of-the-art 61.7% overall IoU on TimeLens-Bench, remarkably outperforming flagship closed-source models like Gemini-3.8-Flash and Qwen3.8-Max. Crucially, when forced to continue from imperfect starts, it reduces the subsequent end prediction error by 36.5%, proving its exceptional ability to end well after flawed start. Code will be open.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Temporal Grounding, Video LLM

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_7ky3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Autoregressive video temporal grounding suffers from systematic endpoint accuracy asymmetry. The authors link this issue to error propagation from predicted start timestamps. They formulate start errors as sunk loss and remaining end errors as conditional regret. FinishStrong addresses this problem through two-stage post-training. Online-start SFT trains end predictions using model-generated start prefixes. Conditional-tree GRPO assigns end advantages through within-branch comparisons. FinishStrong-9B achieves 61.7% IoU on TimeLens-Bench and reduces conditional end MAE by 36.5% over standard SFT.

Critical strengths
  1. Clear motivation. Controlled prefix interventions reveal substantial error propagation from start to end predictions.

  2. Effective training framework. FinishStrong combines online prefix exposure with conditional credit assignment. FinishStrong achieves 61.7% overall IoU. Ablations isolate the contributions of both training stages and conditional credit assignment.

Critical weaknesses
  1. Limited generalization evaluation. Experiments focus on single-interval temporal grounding. Generalization to multi-moment grounding and general video understanding remains unexplored.
  2. Limited handling of severe start errors. Online-start SFT excludes starts beyond the GT end, while Conditional-tree GRPO provides zero end advantages in such cases. The impact of these errors remains unexplored.
  3. Insufficient empirical validation of the GRPO failure mechanism. The negative-advantage issue is theoretically motivated, but its prevalence during training is not quantified.
Questions for authors
  1. Can FinishStrong extend to multi-segment temporal grounding, such as OMTG-Bench (ICML26) ? Does it preserve general video understanding performance on benchmarks such as Video-MME and Video-MME-v2?

  2. How does FinishStrong handle predicted starts at or beyond the GT end, where positive IoU is unattainable? Can the authors analyze end prediction errors and interval duration bias across different start-error magnitudes and directions, compared with baseline methods?

5 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1

We sincerely thank you for the thoughtful review and constructive suggestions. We appreciate your recognition of the clear motivation established through controlled prefix interventions and the effectiveness of our training framework. Below, we address your comments point by point. All additions and revisions are included in the revised manuscript and its appendices.

W1 & Q1: Can FinishStrong generalize to multi-moment grounding and general video understanding?

Thank you for this insightful suggestion! Our additional experiments show that FinishStrong extends effectively to multi-moment grounding while largely preserving general video understanding capabilities.

1) Generalization to multi-moment grounding.

Following your suggestion, we reproduce OMTG-4B (ICML 2026) [1] on OMTG-Bench and apply FinishStrong to this model, extending our evaluation beyond single-interval temporal grounding.

Interestingly, we observe a similar endpoint asymmetry in multi-moment grounding as in the single-moment: our reproduced OMTG-4B has an End MAE of 4.36s, 27.5% higher than its Start MAE of 3.42s. Applying FinishStrong reduces End MAE by 0.82s (18.8%) and Start MAE by 0.25s (7.3%), while improving all six grounding metrics by 1.84-2.91 percent. These results demonstrate improved localization of both boundaries and support the applicability of FinishStrong to multi-moment grounding.

Table R-1. Results on OMTG-Bench.

MethodC-Acc ↑[email protected] ↑[email protected] ↑[email protected] ↑tIoU ↑EtF1 ↑Start MAE (s) ↓End MAE (s) ↓
OMTG-4B (ICML 2026)55.6373.4665.4048.9661.2443.65----
OMTG-4B (Our Reproduction)56.5674.2065.4247.4362.0443.843.424.36
OMTG-4B + FinishStrong58.7376.6368.3350.2364.3545.683.173.54
Improvement / MAE Reduction+2.17+2.43+2.91+2.80+2.31+1.840.250.82

2) Preserving general video understanding capabilities.

We further evaluate FinishStrong-9B on 15 video understanding benchmark settings, covering video perception, temporal reasoning, spatial understanding, and long-video understanding, under both non-thinking and thinking inference.

Overall, FinishStrong remains broadly comparable to its base model Qwen3.5-9B across these evaluations under both inference settings. It also improves several benchmarks, including VideoMMMU (+2.34 points), Video-MME v2 (+0.70 to +1.48 points), VSI-Bench (+1.15 points), TempCompass (MC) (+1.01 points), and MLVU (MCQ) (+0.57 points).

Table R-2. General video understanding performance of Base and FinishStrong.

BenchmarkNon-thinkingThinking
BaseFinishStrongΔ\DeltaBaseFinishStrongΔ\Delta
Video-MME (w/o subs.)74.6774.81+0.1474.7074.30-0.40
Video-MME (w/ subs.)80.2279.93-0.2981.1181.78+0.67
Video-MME v215.8416.54+0.7018.2719.75+1.48
VideoMMMU74.2276.56+2.3480.6780.89+0.22
LongVideoBench61.5661.63+0.0762.1561.41-0.74
MLVU (MCQ)71.1671.73+0.5783.9083.02-0.88
MLVU (Open-ended)4.024.18+0.163.993.97-0.02
TempCompass (MC)70.8971.90+1.0176.8477.34+0.50
TempCompass (Caption Matching)80.5780.17-0.4087.4386.49-0.94
TempCompass (Yes/No)74.5274.77+0.2581.6180.23-1.38
WorldSense43.8844.67+0.7945.5945.81+0.22
Video-Holmes43.6042.24-1.3648.2948.12-0.17
VSI-Bench55.4656.61+1.1554.6554.50-0.15
MMBench-Video1.471.44-0.031.701.69-0.01
MVBench70.2370.70+0.4773.2072.65-0.55

[1] Qi Xu et al. Towards One-to-Many Temporal Grounding. ICML, 2026. Paper

AuthorsOpenReview ↗

Response to Weakness 2 and Question 2

W2 & Q2-Part1: How does FinishStrong handle predicted starts at or beyond the GT end?

Thank you for highlighting this important case. We would like to clarify that both training stages explicitly account for these severe errors through complementary mechanisms: Online-start SFT retains GT start supervision, while Conditional-tree GRPO discourages such predictions through start-level credit assignment.

Online-start SFT. As formulated in Sec. 4.1 (Eq. (4)), when the predicted start is at or beyond the GT end, only the conditional end loss ℓˉe\bar{\ell}_e is masked. The GT start loss ℓˉs\bar{\ell}_s remains active for every sample, so these samples still provide supervision for learning the correct start.

Conditional-tree GRPO. Such starts remain in sampling and optimization. For a branch with si≥e∗s_i\geq e^*, no valid end can achieve positive IoU. All end continuations therefore receive zero rewards and non-positive end advantages. However, the start advantage is

Ais=Q^i−Qˉ=−Qˉ≤0.A_i^s=\hat{Q}_i-\bar{Q}=-\bar{Q}\leq0.

Thus, these zero-overlap starts are penalized relative to starts that achieve positive rewards (Sec. 4.2, Eqs. (5)-(6)).

To examine the practical impact, we track the frequency of these severe errors throughout both training stages.

Table R-3. Frequency of predicted starts at or beyond the GT end during training.

Training PhaseOnline-start SFTConditional-tree GRPO
Early4.991%4.357%
Middle5.935%2.073%
Final5.721%1.851%

The rate remains relatively stable during Online-start SFT (4.991% to 5.721%). A likely explanation is that GT start supervision promotes the annotated start while broadly suppressing incorrect alternatives. Since most incorrect starts fall before the GT end, this correction mainly addresses these more frequent errors, with less targeted pressure on starts at or beyond the GT end. In contrast, Conditional-tree GRPO directly discourages these zero-overlap starts through start-level credit assignment, reducing their frequency from 4.357% to 1.851% (57.5% relative reduction).

Q2-Part2: How do end prediction errors and interval duration bias vary across different start-error magnitudes and directions?

Thank you for this insightful question. Sec. 5.3 (Table 2 and Fig. 4) evaluates conditional end prediction under identical, controlled start prefixes. To complement these tests, we analyze each model's own predictions under greedy decoding, grouped by start-error magnitude and direction.

For a predicted interval [s^,e^][\hat{s},\hat{e}] and a ground-truth interval [s∗,e∗][s^{\ast},e^{\ast}], we define L∗=e∗−s∗L^{\ast}=e^{\ast}-s^{\ast} and the normalized start error d=(s^−s∗)/L∗d=(\hat{s}-s^{\ast})/L^{\ast}. We report End MAE and the mean of the per-example normalized duration bias BL=((e^−s^)−L∗)/L∗B_L=((\hat{e}-\hat{s})-L^{\ast})/L^{\ast}. Positive and negative bias indicate overestimation and underestimation of interval duration, respectively.

Table R-4. End prediction errors and interval duration bias across start-error regimes.

Start-error regimeModelN (%)End MAE (s) ↓Mean Duration Bias
Early (d<−0.5d<-0.5)TimeLens1904 (20.25%)12.5382.7700
Video-OPD1778 (18.91%)13.3312.7965
FinishStrong1365 (14.52%)11.9022.3836
Slightly Early (−0.5≤d<0-0.5\leq d<0)TimeLens1740 (18.50%)6.6700.4021
Video-OPD1678 (17.84%)5.2630.4699
FinishStrong1705 (18.13%)4.5250.3306
Near-GT / Late (0≤d<0.50\leq d<0.5)TimeLens5212 (55.42%)7.7320.4838
Video-OPD5200 (55.30%)5.2210.4433
FinishStrong5774 (61.40%)4.7140.2866
Severely Late (0.5≤d<10.5\leq d<1)TimeLens229 (2.44%)4.8340.4749
Video-OPD278 (2.96%)3.4080.1409
FinishStrong390 (4.15%)2.6050.0114
At or Beyond GT End (d≥1d\geq1)TimeLens319 (3.39%)39.2732.7503
Video-OPD470 (5.00%)44.9511.9653
FinishStrong170 (1.81%)41.1541.8254

FinishStrong achieves the lowest End MAE across all four recoverable start-error regimes (d<1d<1). Compared with the strongest baseline in each regime, it reduces End MAE by 5.1%, 14.0%, 9.7%, and 23.6% for early, slightly early, near-GT/late, and severely late starts, respectively. It also exhibits a smaller positive mean duration bias in every regime, indicating reduced average overestimation of interval duration. Under severely late starts, this mean bias is 0.0114.

For starts at or beyond the GT end (d≥1d\geq1), end errors remain large and no continuation can get positive IoU. FinishStrong produces such starts less frequently: 1.81%, compared with 3.39% for TimeLens and 5.00% for Video-OPD.

AuthorsOpenReview ↗

Response to Weakness 3

W3: Insufficient empirical validation of the GRPO failure mechanism. The negative-advantage issue is theoretically motivated, but its prevalence during training is not quantified.

Thank you for this insightful suggestion. We provide two complementary quantitative analyses of actual training rollouts to examine how end quality relates to the advantages assigned during training.

1) Shared trajectory-level advantages can penalize good ends following poor starts.

Standard GRPO assigns the same trajectory-level advantage to both start and end tokens. A poor start can lower the overall interval reward and lead to a negative advantage even when the subsequent end is accurate given that start. Applying this shared negative advantage to both decisions therefore also penalizes the conditionally good end.

We examine actual training rollouts across cIoU intervals. Higher cIoU indicates that the end realizes a larger fraction of the overlap attainable from its committed start. Table R-5 reports the rollout distribution and the negative-advantage frequency within each cIoU bin. GRPO uses joint trajectory advantages, while FinishStrong uses within-start end advantages.

Table R-5. Negative-advantage frequency across cIoU intervals.

cIoUGRPOFinishStrong
Rollouts (%)Negative Advantage (%)Rollouts (%)Negative Advantage (%)
=0=013.2197.0812.1455.75
(0,0.1)(0,0.1)0.53100.000.7888.00
[0.1,0.2)[0.1,0.2)1.4596.671.6188.46
[0.2,0.3)[0.2,0.3)1.9890.241.9679.37
[0.3,0.4)[0.3,0.4)3.3388.413.7268.33
[0.4,0.5)[0.4,0.5)4.0580.954.2265.44
[0.5,0.6)[0.5,0.6)8.7375.699.1667.12
[0.6,0.7)[0.6,0.7)9.4077.9510.7457.23
[0.7,0.8)[0.7,0.8)10.7070.2710.1843.90
[0.8,0.9)[0.8,0.9)10.7067.129.4736.07
[0.9,0.95)[0.9,0.95)5.8361.985.8427.66
[0.95,1][0.95,1]30.0951.7630.182.78
Total100.00--100.00--

Notably, even among nearly conditionally optimal end predictions (cIoU≥0.95\mathrm{cIoU}\geq0.95, i.e., good ends), 51.76% receive negative advantages under GRPO and are undesirably penalized along with their imperfect starts during training. Under FinishStrong, this proportion drops to only 2.78%. By comparing end predictions following the same start, Conditional-tree GRPO substantially reduces the conflict between joint trajectory credit and conditional end quality.

2) Advantages relative to end-localization error.

We also examine whether the assigned advantages encourage more accurate ends. Let Ei=∣e^i−e∗∣E_i=|\hat e_i-e^*| and let Eˉ\bar E be the mean error within the comparison set. We measure two types of disagreement:

  • L: the set contains an end with lower-than-average error but a negative advantage, discouraging a relatively accurate end.
  • H: the set contains an end with higher-than-average error but a positive advantage, encouraging a relatively inaccurate end.

Percentages are calculated over the comparison sets listed below. Both FinishStrong rows use its actual within-start end advantages.

Table R-6. Frequency of disagreement between end error and advantage sign (%).

MethodError-comparison scopeLHBoth
GRPOFlat rollout group76.0641.3332.43
FinishStrongFull rollout tree38.9524.4811.54
FinishStrongSame-start branch3.602.050.99

Under GRPO, 76.06% of groups contain a relatively accurate end that receives a negative advantage, while 41.33% contain a relatively inaccurate end that receives a positive advantage; both occur in 32.43% of groups. These opposing training signals are substantially less frequent in the analyzed FinishStrong trees, with corresponding frequencies of 38.95%, 24.48%, and 11.54%. Within its same-start branches—the comparison sets used to assign end advantages—the frequencies are 3.60%, 2.05%, and 0.99%.

The controlled G/H ablation in Table 3 further isolates the effect of end-advantage assignment: within-start end advantages improve Free IoU by 1.9 points and conditional cIoU by 2.45 points, while reducing conditional End MAE by 0.73s. These results complement the rollout analyses and support the benefit of conditional end-advantage assignment.

Thank you again for your constructive feedback. Please let us know if any concerns remain. We would be happy to provide further clarification or information.

Reviewer_1ZVCOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies an under-explored failure mode of autoregressive video temporal grounding (VTG): an "endpoint asymmetry" in which the end of an event is localized systematically worse than the start. The authors cast a committed (possibly wrong) start p as an immutable constraint that caps attainable temporal IoU at a conditional ceiling C_max(p), and decompose the IoU shortfall 1 - R(p, e_hat) into an irreducible "sunk loss" 1 - C_max(p) plus a recoverable conditional end regret C_max(p) - R(p, e_hat). Two claims are then made about existing paradigms: teacher-forced SFT rarely exposes the end to the model's own start errors, and holistic GRPO assigns a joint advantage R - R_bar, so that when a flawed start pushes the ceiling below the group mean (C_max(p) < R_bar) even a zero-regret end can receive a negative advantage.

The proposed remedy, FinishStrong, is a two-stage post-training recipe. (i) Online-start SFT supervises the end token conditioned on the model's own greedy start, which is rendered detached via stop-gradient (so start quality is still supervised by the ground-truth start cross-entropy), plus a gating term g(s) = 1[parseable and s < e*]. (ii) Conditional-tree GRPO samples K = 4 starts with M = 3 end continuations each; each start receives the advantage A_s = Q_i_hat - Q_bar computed from the mean downstream reward of its own branch, and each end receives the strictly within-branch advantage A_e = R_ij - Q_i_hat. Because the start-dependent ceiling cancels inside the within-branch contrast, the original IoU reward is used without ceiling normalization. The tree shape is chosen to match the token budget of standard GRPO with G = 8 rollouts (4 x 0.5 + 12 x 0.5 = 8 full sequences). Inference is unchanged: one greedy start followed by one greedy end.

Validation is on TimeLens-Bench (Charades / ActivityNet / QVHighlights, 9,404 queries), where FinishStrong-9B reports 61.7% ALL IoU, plus a "shared-prefix stress test" that injects an identical, possibly perturbed start and delimiter into every model and lets it emit only the end token. Under this intervention the method reports 75.80% global cIoU, +13.02 cIoU points over SFT for earlier starts, and a 36.5% reduction in conditional end MAE. A stage-wise ablation (Table 3, variants A-H) separates the contributions of context exposure (SFT stage) from credit structure (RL stage).

Critical strengths

S1. The problem framing and its formalization are genuinely useful, and I expect this decomposition to outlive the specific method proposed. Casting a committed start as a ceiling C_max(p) and showing that, for a legal fixed p, maximizing IoU is exactly equivalent to minimizing the conditional end regret is a clean and checkable statement (Eq. 2-3). Its most valuable consequence is diagnostic: it makes precise the regime in which standard GRPO misbehaves - not when the start is good or catastrophically bad, but when C_max(p) falls below the group mean, so that a conditionally optimal continuation (Reg_e = 0) still attracts a negative advantage (Fig. 2b). This reframing is consequential to my assessment because it is an explanation of why a widely used recipe fails, not merely an architecture that scores better.

S2. The credit-assignment argument is sound, and the ablation is designed in the right way to support it. Eq. (7) is an exact algebraic identity for balanced K x M trees (the branching factors 3/4 and 2/3 in Appendix E are ordinary finite-group centering factors, not extra reward shaping). More importantly, variant G (same 4 x 3 tree, but joint credit R_ij - Q_bar) performs comparably to standard GRPO (59.8 vs 59.6 free IoU), while variant H (branch-localized credit) gains another 1.9 points. This cleanly separates "sampling a tree" from "evaluating ends against their same-start siblings" and is the right evidence for the paper's central claim. The paper's own hedge in Sec. 4.2 - that the identity "does not by itself guarantee lower gradient variance or convergence" - is honest and correctly scoped.

Critical weaknesses

W1. The headline superiority claim over closed-source models rests on a 0.1-point difference with no reported uncertainty. Table 1 reports 61.7% ALL IoU for FinishStrong-9B versus 61.6% for Gemini-3.8-Flash, yet the seed study in Appendix H covers only the Qwen3.5-9B row and the open-source baselines (standard deviations strictly bounded between 0.13 and 0.34 points). The abstract and Sec. 5.2 nevertheless state that the method "remarkably outperforms flagship closed-source models", and Appendix H itself argues that the open-source gap of 3.1 points is meaningful precisely because it exceeds the maximum observed standard deviation - by that same logic, the 0.1-point gap is not established. This is consequential to my assessment because it is the one claim in the paper that the evidence does not support, and because the paper does not need it: the conditional-recovery results are the real contribution. I would recommend softening this claim to "matches or slightly exceeds" (noting that the proprietary numbers are compiled from technical reports under a potentially different decoding budget, so the comparison is not strictly controlled) and moving the "SOTA among all evaluated models" framing to the shared-prefix setting, where the margins are large.

W2. The relationship to existing decision-conditioned and tree-structured credit assignment is under-argued, and as written the RL contribution reads as a known variance-reduction device. The paper cites TreeGRPO (Ding & Ye, 2026) and T-GRPO in Related Work, but the distinction is dispatched in roughly two sentences. "GRPO with a two-level rollout tree and a per-branch baseline" is a standard hierarchical-advantage construction; what is genuinely task-specific here is that, for a legal fixed p the annotated end e* is exactly the IoU maximizer, so the end can be supervised and rewarded conditionally without any ceiling normalization, and that the resulting within-branch contrast has a zero-regret fixed point rather than merely lower variance. That argument is present implicitly in Appendix E but never stated as the novelty claim in the main text. A short, explicit paragraph contrasting this with per-step credit in tree RL (what is new: the task-specific identity that makes the end reward conditionally well-posed, not the tree itself) would materially strengthen the contribution. This is consequential because the current framing invites the reading that the gain is engineering.

W3. Several design choices are justified by budget parity or by assertion, but not ablated. (a) The tree shape K = 4, M = 3 is fixed only by the token-budget argument; no sensitivity over K, M, or budget-matched alternatives (e.g. K = 8/M = 1, K = 2/M = 6) is reported, yet the branch-mean estimate Q_i_hat is noisier for small M and the method's core quantity is that branch mean. (b) The gate g(s) = 1[parseable and s < e*] "holds for over 90% of generated starts in early updates" - no later-training statistic is given, and the paper does not say what fraction of starts becomes invalid late in training or how sensitive the result is to this masking rule. (c) The claim that artificial noise "completely fails" is demonstrated at two fixed second-scale noise levels (0.5 s, 1 s) on one model (Appendix G), while the shared-prefix grid used everywhere else operates on span-normalized offsets up to 0.9 L* (and up to 2 L* for video-span scans), i.e. far larger than 1 s for typical events. The two experiments are therefore not measuring the same perturbation regime, which weakens the argument that model-generated contexts are specifically what matters. These are all inexpensive experiments.

W4. Evaluation is confined to one benchmark family and one training distribution. All models are trained on TimeLens-100K with hyperparameters taken from TimeLens; the shared-prefix protocol is evaluated only on TimeLens-Bench variants. There is no cross-dataset transfer test (no experiment where the model is trained on TimeLens-100K and evaluated on a grounding benchmark outside this family, or the reverse). This is consequential because an alternative explanation for part of the gain - tighter fit to the TimeLens-Bench end-token distribution rather than a transferable ability to adapt after poor starts - cannot currently be excluded. Even a single transfer result would substantially raise my confidence in the generality of the claim.

W5. The controlled protocol that carries the paper's main argument is under-specified for re-implementation. The shared-prefix experiments require forcibly supplying a start and delimiter and letting the model generate only the end; the paper gives the offset formulas in Sec. 5.1, but the parsing/validity filter is described only as "validly parsed", the aggregation over offsets is deferred to Appendix A, and there is no statement of how many queries survive the filter per offset (the one number given, 9,357 matched predictions, appears only in the exploratory Appendix F). For a paper whose central claim is about a controlled conditional protocol, the protocol itself should be specified well enough to reproduce.

Questions for authors

Q1. In Table 2, the GT-start (d = 0) cIoU of FinishStrong is 78.25 versus 74.04 for SFT, while the free-decoding ALL IoU in Table 1 is 61.7. Please clarify the relationship between these two settings: does the shared-prefix evaluation restrict the query set to those for which the GT start is itself reliably predictable/parseable, or is the gap driven by the elimination of start error alone? Any restriction of the query set would change how the "conditional versus unconditional" gap should be interpreted.

Q2. Sec. 4.2 states that the 4 x 3 tree matches the token budget of G = 8 standard GRPO because 4 x 0.5 + 12 x 0.5 = 8 full sequences. This assumes that a start (and an end) each consume exactly half a sequence's tokens. Please report the measured mean token counts per rollout for both topologies, and confirm that the G versus H comparison in Table 3 is budget-matched rather than rollout-count-matched.

Q3. In Online-start SFT, the start used as conditional context is the greedy (mode) sample. Would sampling or top-k sampling of starts during training give better coverage of the start-error distribution encountered at inference? Relatedly, how does the method behave when the inference-time start distribution is shifted, e.g. under a different decoding temperature, or when the start is produced by a different backbone?

Q4. Nothing in the objective explicitly prevents the policy from preferring starts that are easy to end well from (high C_max(s)) over starts that are accurate, since A_s is computed from the mean downstream reward rather than from start accuracy. Sec. 5.3 reports that start MAE also improves, which is reassuring, but is there an explicit diagnostic for this, e.g. P(|s - s*| < tau) tracked separately across training?

Q5. Have you measured the fraction of evaluation queries for which the committed start is already beyond recovery, i.e. C_max(p) < 0.3 / 0.5 / 0.7? Reporting this would make the cIoU numbers easier to interpret, since a method can only be judged on the recoverable subset.

Q6. The "AI Documentation For Reviewing" field aside, the paper's AI Use Statement discloses LLM involvement in drafting initial sections and in the data-processing scripts. Given that a primary claim is about a precisely controlled evaluation protocol, could the authors confirm whether the parsing, validity filtering, and offset-aggregation code were human-specified and independently verified, and provide the anonymized code or at minimum the exact filter definition?

Minor Points

  • Sec. 3.1, Eq. (2): the piecewise ceiling for p < s* is L*/ (e* - p). This implicitly assumes that the end may be taken arbitrarily far beyond e*; if the end is capped at the video length, the ceiling for a start before s* is attained at the video end rather than "uniquely at e*". Please state the assumption (the worked example with [20, 40] and p = 32 is consistent with either reading, so the statement in the text is correct only under the unbounded-end convention).
  • Table 1: the comparison is easy to misread because the 8B and 9B rows are adjacent to different baseline blocks. Please state in one sentence that FinishStrong-9B improves over its own Qwen3.5-9B base by 9.5 points and over FinishStrong-8B by 3.6 points.
  • Table 1: the "ALL" column is described in the text as a macro-average over 9,404 queries, but the header reads "ALL IoU". Consider "Avg. IoU" or stating explicitly that it is the unweighted mean of the three per-dataset IoU values, so that it is not confused with a query-weighted average.
  • Appendix H: please report seed-to-seed variability for the conditional metrics (cIoU and conditional end MAE) as well as for free IoU, since those are the metrics on which the paper's central claim rests.
  • Sec. 5.3: "its cIoU advantage over SFT given a perfect GT start (d = 0) is 4.21 points. However, when forced to continue from earlier GT-span perturbations, this gap widens massively to 13.02 points" compares an absolute difference at d = 0 with an absolute difference under earlier starts. The related claim that this constitutes "an additional excess conditional advantage of 8.81 points" should be restated more carefully, since a larger absolute gap under a harder condition is expected for any method whose advantage grows with the size of the injected start error.
  • Figure 4's footer cards (+2.08, +4.98, +3.53 pp for cIoU; +4.34, +6.16, +5.25 pp for end accuracy) are not referenced in the text; please cross-reference them in Sec. 5.3.
  • I noted several LaTeX/rendering issues in the submitted PDF (e.g. the method-name macro appearing literally as "\method" in the abstract, and words being run together in the rendered text). These do not affect my assessment but should be fixed for the camera-ready version.
10 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Minor Point 4

We sincerely thank you for the thoughtful and detailed review. We appreciate your recognition of the broader value of our formulation for video grounding and the insights it provides into GRPO's limitations. We also appreciate your positive assessment of the sound theoretical foundation and controlled ablations. Below, we address your comments point by point. All additions and revisions are included in the revised manuscript and its appendices.

W1 & Minor 4: Comparison with proprietary models and seed variability of conditional metrics.

1) Clarifying the intended emphasis of the proprietary-model comparison.

Thank you for your reminder! As you noted, the central contribution of this work lies in conditional recovery. The comparison with proprietary models provides additional context for our overall grounding performance. Our intended emphasis was that a model with only 9B parameters achieves strong performance relative to huge flagship models. Table 1 reports 61.7% ALL IoU for FinishStrong-9B, with substantial score margins over several proprietary models:

Proprietary modelReported ALL IoU (%) ↑FinishStrong-9B gain (pp)
GPT-544.3+17.4
Qwen3.8-Max58.1+3.6
Gemini-3.1-Pro58.9+2.8
Gemini-3.8-Flash61.6+0.1

For Gemini-3.8-Flash, the closest comparator, we agree that these results indicate comparable performance given the 0.1-point margin. Our use of "remarkably" was intended to emphasize the performance achieved with a 9B model; it should not imply that this particular margin establishes a statistically reliable advantage. We have revised the wording in the abstract and Sec. 5.2 to make this distinction explicit.

2) Extending the seed study to conditional metrics.

Thank you for this suggestion. Table R-1 reports the mean and standard deviation of Free IoU, cIoU, and conditional End MAE across five training seeds. The conditional metrics are averaged over the twelve nonzero GT-span offsets.

Table R-1. Five-seed comparison.

MethodFree IoU (%) ↑Conditional cIoU (%) ↑Conditional End MAE (s) ↓
SFT56.4±0.2256.4\pm0.2266.22±0.3266.22\pm0.327.43±0.157.43\pm0.15
GRPO57.9±0.2557.9\pm0.2570.15±0.2870.15\pm0.285.30±0.125.30\pm0.12
TimeLens57.8±0.1857.8\pm0.1872.18±0.2172.18\pm0.216.32±0.116.32\pm0.11
Video-OPD58.5±0.2858.5\pm0.2868.23±0.3168.23\pm0.315.67±0.165.67\pm0.16
FinishStrong61.6±0.1461.6\pm0.1475.66±0.2275.66\pm0.224.68±0.104.68\pm0.10

Across all three metrics, FinishStrong's gains over the strongest baseline for each metric substantially exceed the observed seed-to-seed standard deviations, supporting the robustness of our improvements. We have added the five-seed results to Appendix H of the revised manuscript.

AuthorsOpenReview ↗

Response to Weakness 2 and Minor Point 1

W2: Clarifying the conditional-training rationale in the main text.

Thank you for suggesting that we make the connection between our formulation and training design more explicit in the main text. Building on the analysis in Appendix E, we have clarified how the conditional optimality of the annotated end informs supervision and credit assignment, including its relationship to existing tree-based methods.

We have clarified the relationship to TreeGRPO and T-GRPO in Sec. 2, explaining how our conditional-recovery formulation informs both the supervision target and the comparison used to assign end credit.

The formulation separates irreversible start loss from the remaining end regret. It explains why joint trajectory credit can penalize a conditionally good end following a poor start. Crucially, for any valid committed start p<e∗p<e^{\ast}, the annotated end e∗e^{\ast} uniquely maximizes temporal IoU. This provides an exact supervision target under model-generated starts, which Online-start SFT uses to train end prediction without an external teacher while retaining GT start supervision.

For RL, the connection to tree-based credit assignment becomes explicit through the role of the committed start. Each branch holds the start fixed, so its end continuations share the same attainable IoU ceiling. This shared ceiling cancels in the within-start reward contrast, making end credit a comparison of relative conditional end regret. The original IoU reward is retained without ceiling normalization, and an exactly conditionally optimal end cannot receive negative within-start credit because of an imperfect start. This gives the branch baseline a direct interpretation in terms of the remaining recoverable error.

As you highlighted, the G/H ablation in Table 3 holds the sampling tree and start-level advantages fixed and isolates the effect of end-credit assignment. Within-start credit improves Free IoU by 1.9 points and conditional cIoU by 2.45 points, while reducing End MAE by 0.73s. We have connected these results more directly to the conditional-regret analysis in the revised main text.

Minor 1: Does the optimal-end result hold when predicted ends are bounded by the video length?

Eq. (2) does not require unbounded end predictions. Since the GT end lies within the video (e∗≤Te^{\ast}\leq T), the unique optimum at e∗e^{\ast} remains feasible for any valid fixed start p<e∗p<e^{\ast}. Extending the end beyond e∗e^{\ast} only enlarges the union without increasing the overlap, so the video boundary cannot yield a higher IoU. We have explicitly stated the prediction domain alongside Eq. (2).

AuthorsOpenReview ↗

Response to Weakness 3 (a) and Question 2

W3(a) & Q2: Tree-shape sensitivity and measured generation budgets.

Thank you for these suggestions. We provide measured generation costs, a breakdown of timestamp lengths, and additional tree-shape comparisons below.

1) Measured generation budgets and the G/H comparison.

The calculation 4×0.5+12×0.5=84\times0.5+12\times0.5=8 assumes equal start and end lengths. Table R-2(a) reports measured costs, showing that G and H differ by only 0.03% in generated tokens per prompt.

Table R-2(a). Sampling topology and measured generation costs.

ConfigurationSampled starts / promptSampled ends / promptStart tokens / startEnd tokens / endGenerated tokens / prompt
E: Flat GRPO8810.98177.1728145.24
G: Tree / joint credit41210.98337.1706129.98
H: Conditional-tree GRPO41210.98607.1729130.02

To explain the start/end length difference, Table R-2(b) separately reports the token lengths of the timestamp fields themselves.

Table R-2(b). Timestamp token lengths excluding response formatting.

ConfigurationStart timestamp tokensEnd timestamp tokens
E: Flat GRPO3.98174.1716
G: Tree / joint credit3.98334.1705
H: Conditional-tree GRPO3.98604.1728

The timestamp fields have similar lengths. Response formatting adds approximately seven prefix/delimiter tokens to the start segment (e.g., “the event happens in”) and three suffix/EOS tokens to the end segment (e.g., “seconds”). This makes the full start segment approximately 3.81 tokens longer.

G and H share the same 4×34\times3 tree and start-level advantages. At nearly identical per-prompt generation costs, within-start credit improves Free IoU by 1.9 percentage points and conditional cIoU by 2.45 percentage points, while reducing End MAE by 0.73s. Moreover, our method uses 10.5% fewer generated tokens per prompt than flat GRPO (130.02 vs. 145.24), benefiting from shared start prefixes.

2) Sensitivity to the tree shape.

Table R-3 compares three allocations with the same nominal eight-sequence budget under the half-sequence approximation. We use 2×72\times7 as the alternative allocation because the suggested 2×62\times6 corresponds to seven nominal sequences.

Table R-3. Tree-shape sensitivity.

KKMMNominal full-sequence equivalentsFree IoU (%) ↑Conditional cIoU (%) ↑End MAE (s) ↓
27861.375.484.82
43861.775.804.65
81857.672.276.25

The two non-degenerate allocations perform similarly, with gaps of only 0.4 percentage points in Free IoU, 0.32 percentage points in conditional cIoU, and 0.17s in End MAE. The small gap may reflect limited start exploration in 2×72\times7, which samples only two starts despite allocating fourteen end continuations. The default 4×34\times3 provides more start diversity while retaining multiple continuations per start. For M=1M=1, the conditional end-credit signal vanishes. Its lower performance is consistent with the benefit of retaining this signal (GRPO). We have incorporated these results and clarified the budget accounting in the revised manuscript.

Reviewer_BZ8QOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses conditional recovery in video temporal grounding, focusing on predicting accurate ends after imperfect starts. FinishStrong combines Online-start SFT, which trains the model to predict ends conditioned on model-generated starts, with Conditional-tree GRPO, which compares rewards among end candidates sharing the same start. Experiments show improved video temporal grounding performance across three datasets and better end prediction under shared-prefix evaluations.

Critical strengths
  1. The paper presents a well-motivated formulation of video temporal grounding as a conditional recovery problem, focusing on predicting accurate ends after imperfect starts. It clearly explains how an imperfect start limits the achievable grounding accuracy and what can still be improved through better end prediction.

  2. The use of model-generated starts in Online-start SFT, combined with within-start comparisons of end predictions in Conditional-tree GRPO, is a sensible way to improve conditional recovery without requiring an external teacher or additional inference overhead.

Critical weaknesses
  1. Conditional-tree GRPO is closely related to existing tree-based credit assignment methods. Its novelty mainly lies in adapting these ideas to conditional recovery in video temporal grounding, which may still be seen as incremental.

  2. The experiments show improved end prediction, but do not clearly isolate conditional recovery from general end-prediction strength. The claim of excess conditional advantage relies mainly on comparisons with SFT and is not consistently supported against stronger baselines.

Questions for authors
  1. Does GRPO struggle with conditional recovery despite training on model-generated starts? A direct analysis would clarify the need for within-start comparisons.
  2. How frequently do start errors of different magnitudes occur under greedy decoding?
  3. How much does FinishStrong improve end prediction under naturally occurring start errors?
  4. What evidence from the TimeLens and GRPO comparisons in Section 5.3 distinguishes improved conditional resilience from generally stronger end prediction?
6 comments
AuthorsOpenReview ↗

Response to Weakness 1

We sincerely thank you for the thoughtful and encouraging evaluation. We appreciate your recognition of our well-motivated formulation, which clearly explains how imperfect starts limit grounding and what can still be improved. We also appreciate your acknowledgment of our approach to improve conditional recovery without an external teacher or additional inference overhead. Below, we address your comments point by point. All additions and revisions are included in the revised manuscript and its appendices.

W1: Conditional-tree GRPO is closely related to existing tree-based credit assignment methods. Its novelty mainly lies in adapting these ideas to conditional recovery in video temporal grounding, which may still be seen as incremental.

Thank you for raising this important concern. We agree that tree-structured sampling and hierarchical credit assignment have been explored in prior work, including TreeGRPO and T-GRPO discussed in Section 2. Our contribution begins with identifying a failure mode in autoregressive grounding: a conditionally good end can be penalized because of an earlier start error. We formalize the separation between irreversible start loss and recoverable end regret, and use this formulation to develop a two-stage training framework that teaches the model to make the best remaining end prediction.

Specifically, TreeGRPO propagates prompt-normalized leaf advantages through a denoising tree, while T-GRPO learns key-frame exploration using node- and tree-level rewards, with advantages normalized across sampled nodes. Our framework distinguishes the roles of start and end decisions through the attainable IoU and the remaining conditional end regret, as detailed below.

1) Identifying and formalizing conditional recovery after imperfect starts.

A committed start imposes a ceiling on the attainable IoU. Our formulation separates the irreversible loss caused by this start from the remaining conditional end regret (Sec. 3). Crucially, for any valid fixed start 0≤p<e∗≤T0\leq p<e^{\ast}\leq T, the GT end e∗e^{\ast} uniquely maximizes temporal IoU (Eq. (2)). Thus, the original end annotation remains an exact optimal supervision target even under an inaccurate but valid model-generated start. The same formulation explains why joint trajectory credit can penalize a conditionally good end: the overall reward still reflects the irreversible loss inherited from the start.

2) Aligning both supervision and credit assignment with conditional recovery.

Online-start SFT uses this exact target to train end prediction under valid model-generated starts without an external teacher, while retaining GT start supervision.

For RL, Conditional-tree GRPO assigns start credit from mean downstream IoU and computes end credit by comparing continuations sharing the same committed start. Because these continuations share the same attainable IoU ceiling, this ceiling cancels in the within-start reward contrast. End credit therefore reflects relative conditional end regret, using the original IoU reward without ceiling normalization. A conditionally optimal end consequently receives nonnegative within-start credit even when its start is inaccurate. The fixed-start analysis thus provides both an exact supervision target for SFT and a direct interpretation of the end-credit rule in RL.

3) Controlled ablations isolate the benefit of conditional end credit.

Variants G and H in Table 3 use the same Online-start SFT initialization, 4×34\times3 sampling-tree topology, and start-advantage rule, while changing only the end-credit rule.

Excerpt from Table 3 of the submitted manuscript. Effect of end-credit assignment under the same sampling-tree topology.

VariantEnd creditFree IoU (%) ↑cIoU (%) ↑End MAE (s) ↓
GJoint trajectory credit59.873.355.38
HWithin-start conditional credit61.775.804.65

Replacing joint end credit with within-start credit improves Free IoU by 1.9 percentage points and cIoU by 2.45 percentage points, while reducing conditional End MAE by 0.73s. These results support the benefit of within-start end credit beyond the shared sampling-tree topology.

We have clarified this task-specific contribution and its relationship to existing tree-based methods in the revised manuscript, with the detailed analysis retained in Appendix E.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 4

W2 & Q4: The experiments show improved end prediction, but do not clearly isolate conditional recovery from general end-prediction strength. The claim of excess conditional advantage relies mainly on comparisons with SFT and is not consistently supported against stronger baselines. What evidence from the TimeLens and GRPO comparisons in Section 5.3 distinguishes improved conditional resilience from generally stronger end prediction?

Thank you for this important question. Our goal is to teach the model to predict a better end after committing to an imperfect start. Improved conditional recovery and stronger general end prediction are compatible outcomes of this training. Our focus is whether the model predicts better ends when all methods face the same imperfect starts.

1) Shared-prefix comparisons directly evaluate end quality under the same imperfect starts.

In Sec. 5.3, every model receives the same start prefix for each query and perturbation condition. This controls start quality, so the resulting differences reflect how well each model predicts the end under the same constraint.

Excerpt from Table 2. End prediction under identical imperfect starts.

ModelEarlier startsLater starts
cIoU (%) ↑[email protected] (%) ↑End MAE (s) ↓cIoU (%) ↑[email protected] (%) ↑End MAE (s) ↓
GRPO71.6335.895.6168.8935.544.83
TimeLens75.3743.136.8269.1638.595.68
FinishStrong77.4547.475.0274.1444.764.29

FinishStrong improves cIoU over TimeLens by 2.08/4.98 points and over GRPO by 5.82/5.25 points for earlier/later starts, respectively. Its conditional End MAE is also lower by 26.4%/24.5% relative to TimeLens and 10.5%/11.2% relative to GRPO. Moreover, Fig. 4 shows higher End [email protected] at every evaluated GT-span offset, supporting consistent gains across start-error magnitudes and directions. These results directly demonstrate better end prediction following the same imperfect starts.

2) Controlled ablations connect these gains to conditional end credit.

The G/H comparison in Table 3 holds the Online-start SFT initialization, 4×34\times3 sampling tree, and start-level advantages fixed, changing only the end-credit assignment.

Excerpt from Table 3. Effect of end-credit assignment.

VariantEnd creditFree IoU (%) ↑cIoU (%) ↑End MAE (s) ↓
GJoint trajectory credit59.873.355.38
HWithin-start conditional credit61.775.804.65

Within-start credit improves conditional cIoU by 2.45 points and reduces End MAE by 0.73s. This isolates the benefit of evaluating end predictions against continuations sharing the same start. Together with the rollout analysis in our response to Q1, it supports the role of conditional credit in addressing the training conflict where good ends can be penalized along with poor starts.

3) Improvements under GT starts are compatible with our objective.

Training on imperfect prefixes can strengthen end localization in ways that also benefit predictions under GT prefixes. Online-start SFT uses the existing GT end annotations to supervise the model under its own generated starts, while Conditional-tree GRPO rewards better ends relative to other continuations from the same start. Such training can improve end prediction across prefix conditions; it need not produce larger gains under harder prefixes than under GT prefixes.

We agree that the larger gap over SFT does not establish an additional robustness advantage over every stronger baseline. We have revised Sec. 5.3 to avoid implying that the shared-prefix results fully disentangle general end strength from sensitivity to start errors. Our supported conclusion is that FinishStrong predicts better ends under identical imperfect starts, with controlled ablations demonstrating the benefit of the proposed conditional training.

AuthorsOpenReview ↗

Response to Question 1

Q1: Does GRPO struggle with conditional recovery despite training on model-generated starts? A direct analysis would clarify the need for within-start comparisons.

Thank you for this insightful question. Yes, GRPO can still struggle with conditional recovery despite training on model-generated starts. Our rollout analysis and ablation results provide direct evidence below.

1) Shared trajectory-level advantage of GRPO can penalize good ends following poor starts.

Standard GRPO assigns the same trajectory-level advantage to both start and end tokens. A poor start can lower the overall interval reward and lead to a negative advantage even when the subsequent end is accurate given that start. Applying this shared negative advantage to both decisions therefore also penalizes the conditionally good end. Thus, exposing the model to imperfect starts alone does not resolve this credit-assignment issue.

To quantify how often conditionally good continuations receive negative advantages, we analyze actual training rollouts across cIoU intervals. Higher cIoU indicates better end prediction relative to the maximum overlap attainable under the committed start.

Table R-1. Negative-advantage frequency across cIoU intervals.

cIoUGRPOFinishStrong
Rollouts (%)Negative Advantage (%)Rollouts (%)Negative Advantage (%)
=0=013.2197.0812.1455.75
(0,0.1)(0,0.1)0.53100.000.7888.00
[0.1,0.2)[0.1,0.2)1.4596.671.6188.46
[0.2,0.3)[0.2,0.3)1.9890.241.9679.37
[0.3,0.4)[0.3,0.4)3.3388.413.7268.33
[0.4,0.5)[0.4,0.5)4.0580.954.2265.44
[0.5,0.6)[0.5,0.6)8.7375.699.1667.12
[0.6,0.7)[0.6,0.7)9.4077.9510.7457.23
[0.7,0.8)[0.7,0.8)10.7070.2710.1843.90
[0.8,0.9)[0.8,0.9)10.7067.129.4736.07
[0.9,0.95)[0.9,0.95)5.8361.985.8427.66
[0.95,1][0.95,1]30.0951.7630.182.78
Total100.00--100.00--

Notably, even among nearly conditionally optimal end predictions (cIoU≥0.95\mathrm{cIoU}\geq0.95, i.e., good ends), 51.76% receive negative advantages under GRPO and are undesirably penalized along with their imperfect starts during training. Under FinishStrong, this proportion drops to only 2.78%. By comparing end predictions following the same start, Conditional-tree GRPO substantially reduces the conflict between joint trajectory credit and conditional end quality.

2) Advantages relative to end-localization error.

We also examine whether the assigned advantages encourage more accurate ends. Let Ei=∣e^i−e∗∣E_i=|\hat e_i-e^*| and let Eˉ\bar E be the mean error within the comparison set. We measure two types of disagreement:

  • L: the set contains an end with lower-than-average error but a negative advantage, discouraging a relatively accurate end.
  • H: the set contains an end with higher-than-average error but a positive advantage, encouraging a relatively inaccurate end.

Percentages are calculated over the comparison sets listed below. Both FinishStrong rows use its actual within-start end advantages.

Table R-2. Frequency of disagreement between end error and advantage sign (%).

MethodError-comparison scopeLHBoth
GRPOFlat rollout group76.0641.3332.43
FinishStrongFull rollout tree38.9524.4811.54
FinishStrongSame-start branch3.602.050.99

Under GRPO, 76.06% of groups contain a relatively accurate end that receives a negative advantage, while 41.33% contain a relatively inaccurate end that receives a positive advantage; both occur in 32.43% of groups. These opposing training signals are substantially less frequent in the analyzed FinishStrong trees, with corresponding frequencies of 38.95%, 24.48%, and 11.54%. Within its same-start branches—the comparison sets used to assign end advantages—the frequencies are 3.60%, 2.05%, and 0.99%.

The controlled G/H ablation in Table 3 further isolates the effect of end-advantage assignment: within-start end advantages improve Free IoU by 1.9 points and conditional cIoU by 2.45 points, while reducing conditional End MAE by 0.73s. These results complement the rollout analyses and support the benefit of conditional end-advantage assignment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.