acceptodds
Under review as a conference paper at ICLR 2027

Calibrated Answer-Stability Halting for Active Long-Video Understanding

Abstract

Long-video question answering enables efficient access to lengthy recordings, but confidence from partial observations can be misleading when later evidence changes the answer. We seek to reduce video processing while controlling premature commitment to answers that would change under a fixed, expanded view. We introduce Stability-Based Halting (SBH), which uses a lightweight linear predictor combining current confidence, observation progress, and coarse cues from remaining video content to estimate the current answer's advantage over alternatives under that reference view. Calibration across videos yields a lower bound that triggers stopping when positive and controls early-stop reference disagreement across steps and questions under exchangeable video sampling. On held-out Video-MME videos with Qwen2.5-VL-7B, SBH achieves a speedup over scoring all 64 budgeted frames once, with 68.02% versus 67.65% accuracy and 5.19% observed video-level early-stop reference disagreement at a 10% target.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long-video understanding, Video question answering, Vision-language models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WbzcOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Stability-Based Halting (SBH) for efficient long-video question answering. It uses a lightweight linear predictor to estimate whether the current answer will remain stable under an expanded video view, incorporating confidence, observation progress, and remaining-video information. A video-level conformal calibration strategy provides a statistical guarantee on early-stop disagreement with the reference answer. Experiments across multiple benchmarks and MLLM backbones demonstrate inference speedups with comparable accuracy.

Critical strengths
  1. The paper addresses an important limitation of confidence-based early stopping. A confident answer may change when additional video evidence becomes available. The proposed reference-consistency formulation is clearly defined, and the video-level conformal calibration provides a meaningful finite-sample guarantee while accounting for dependencies across observation steps and questions.

  2. The experiments include multiple backbones, benchmarks, controlled stopping baselines, component ablations, and end-to-end latency measurements. The authors also report paired bootstrap confidence intervals and distinguish reference consistency from answer correctness. The demonstrated inference speedups provide practical evidence for the proposed stopping framework.

Critical weaknesses
  1. The theoretical guarantee controls agreement with a fixed 128 frame reference view rather than correctness or consistency with the complete video. Although the authors acknowledge this distinction, the reliability of the reference itself is not sufficiently investigated. It remains unclear whether the reference answers and stopping decisions would remain stable under larger observation budgets. Since the proposed guarantee fundamentally depends on the reference definition, this limits its practical interpretation for long-video understanding.

  2. While the controlled experiments demonstrate meaningful latency reductions, the accuracy improvements are small and statistically uncertain. The contribution of the remaining-video cue is also relatively modest, removing it changes accuracy from 68.02% to 67.78% and latency from 7.74 to 8.05 seconds (Table 6). Moreover, the method requires substantial offline reference scoring and calibration, including 26.7 GPU-hours for Qwen2.5-VL on Video-MME. The paper does not sufficiently discuss how these costs are amortized or how well the calibrated stopping rule transfers across different deployment settings.

Questions for authors

Q1. The proposed guarantee is defined relative to a fixed 128 frame reference view. How sensitive are the reference answers and early-stopping decisions to this budget? Would the conclusions remain consistent if the reference budget were increased to 256 or 512 frames?

Q2. Table 6 shows that the remaining-video cue provides relatively small improvements in final accuracy (68.02% vs. 67.78%) and latency (7.74 vs. 8.05 seconds). Could the authors provide stronger evidence that these improvements are consistent and statistically meaningful, particularly compared with simpler calibrated stopping predictors?

Q3. Table 8 reports substantial offline scoring costs, including 26.7 GPU-hours for Qwen2.5-VL on Video-MME. How many online queries are required to amortize this overhead, and to what extent can the fitted predictor and calibration parameters be reused across datasets or deployment environments?

Q4. The qualitative examples in Figures 6-8 use AI-generated scenes with manually specified answers and stopping-bound signs. Could the authors provide actual inference trajectories from real benchmark videos, including the predicted margins, calibrated bounds, stopping decisions, and final answer correctness?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Wbzc (1/4): Reference-budget sensitivity

Thank you for your careful review and the time you devoted to assessing our theory and experiments. We address your constructive comments below with analyses of reference budgets, cue effectiveness, deployment costs, and inference trajectories.

W1 / Q1. Reliability of the 128-frame reference and sensitivity to 256/512 frames.

Response.{\color{green}\mathbf{Response.}} Theorem 1 formalizes preservation of a specified reference prediction. With the predictor and protocol fixed before calibration, exchangeability of calibration and test videos gives a marginal bound of α\alpha on any early-stop reference disagreement within a new video. To assess the reference itself, the tables below report ground-truth (GT) accuracy and sensitivity to larger observation budgets. The all-return metric includes budget-endpoint disagreements.

Preserving reference behavior is an established acceleration objective in CATs [1] and CALM [2]. SBH applies this objective to video evidence acquisition through simultaneous video-level calibration.

Reference-budget sensitivity: Qwen2.5-VL / Video-MME. Nested 128/256/512-frame references contain the same 64-frame online endpoint. Every frame is processed in full at the same 448×448 resolution. The first comparison freezes the original stopping rule and its stopping times. JJ is the percentage of videos with any early-stop disagreement. All-return disagreement is measured per question.

Reference framesReference GT accuracy (%)Answer flips versus 128Answer flips versus preceding budgetFrozen-rule JJ (%)All-return disagreement (%)
12868.400/810—5.195.93
25669.1430/810 (3.70%)30/810 (3.70%)6.306.67
51269.3834/810 (4.20%)14/810 (1.73%)6.677.16

Stopping sensitivity after refitting and recalibrating for each reference, α=0.10\alpha=0.10. JJ is measured per video. Stopping changes and accuracy are measured per question.

Reference framesρ\rhoJJ (%)Early (%)Changed stopping step (%)Accuracy (%)Latency (s)
1280.2145.1992.350.0068.027.74
2560.2295.9390.629.3868.158.01
5120.2446.3089.2613.3368.028.18

Expanding the reference changes 3.70–4.20% of answers and raises reference accuracy from 68.40% to 69.14–69.38%. The increase from 256 to 512 frames changes only 14/810 answers (1.73%). After refitting and recalibration, stopping steps change for 9.38–13.33% of questions, while accuracy stays within 68.02–68.15%. The larger-reference rules retain 1.35× and 1.33× speedups over the 10.84 s endpoint. This supports 128 frames as a practical reference in this setting. The frozen-rule rows measure sensitivity to a changed reference. Each refitted rule is calibrated against its own reference for the guarantee.

[1] Consistent Accelerated Inference via Confident Adaptive Transformers.

[2] Confident Adaptive Language Modeling.

AuthorsOpenReview ↗

Response to Reviewer Wbzc (2/4): Statistical evidence for the cue

W2(a) / Q2. Accuracy and statistical evidence for the remaining-video cue.

Response.{\color{green}\mathbf{Response.}} Table 9 shows a 3.10 s latency reduction with a paired confidence interval entirely below zero. The accuracy difference is +0.37 percentage points, with its confidence interval reported alongside latency:

Original Table 9: SBH minus endpointDifference [95% paired CI]
Accuracy (percentage points)+0.37 [−0.74, +1.48]
Latency (s)−3.10 [−3.48, −2.74]

For the remaining-video cue, the direct mechanism is better prediction of answer stability. Original Table 6 reports:

Original Table 6Margin MAEρ\rhoPositive bounds (%)Early (%)Accuracy (%)Latency (s)
SBH without cue0.1030.25135.9188.1567.788.05
SBH with cue0.0840.21442.6392.3568.027.74

The cue reduces prediction MAE by 18.4%. The positive-bound frequency rises by 6.72 percentage points and the early-stop rate rises by 4.20 percentage points. Positive-bound frequency uses all eligible prefixes, with question- and video-balanced averaging. It measures how often the predictor can certify a stop. The training-only donor cues in Table 6 give higher MAE (0.105–0.110) and latency (8.11–8.22 s), supporting the value of the remaining video's content.

Paired cue analysis with 95% confidence intervals. Each no-cue predictor is separately refitted and recalibrated. Differences are with-cue minus without-cue. The three seeds keep the evaluation videos fixed and vary the fitting/calibration split.

Dataset / backboneLatency difference (s) [95% CI]Accuracy difference (pp) [95% CI]Margin-MAE difference [95% CI]Latency-difference range across three seeds (s)
Video-MME / Qwen2.5-VL−0.31 [−0.50, −0.12]+0.24 [−0.49, +0.97]−0.019 [−0.025, −0.013][−0.39, −0.23]
Video-MME / InternVL2.5−0.36 [−0.59, −0.13]+0.24 [−0.49, +0.97]−0.018 [−0.024, −0.012][−0.44, −0.25]
LongVideoBench / Qwen2.5-VL−0.30 [−0.57, −0.03]+0.22 [−0.69, +1.13]−0.018 [−0.026, −0.010][−0.38, −0.18]

Simpler calibrated predictors. Qwen / Video-MME, identical video-level calibration at α=0.10\alpha=0.10. Margin and entropy use st=mts_t=m_t and st=1−hts_t=1-h_t, respectively. Each is calibrated against the binary reference-agreement target using the maximum residual across a video's questions and prefixes. The table reports their observed video-level risks JJ.

PredictorJJ (%)Accuracy (%)Latency (s)
SBH with cue5.1968.027.74
Calibrated classifier with cue5.5667.787.99
Calibrated margin5.9367.659.12
Calibrated entropy6.3067.289.38
Calibrated TV2.9667.5312.47

The cue reduces prediction error and latency in all three settings. The paired latency and MAE intervals are below zero in each setting, and all three fitting/calibration seeds give lower latency. The accuracy intervals include zero, so the evidence supports faster stopping at similar observed accuracy.

AuthorsOpenReview ↗

Response to Reviewer Wbzc (3/4): Amortization and cross-domain reuse

W2(b) / Q3. Amortization of offline scoring and reuse across deployments.

Response.{\color{green}\mathbf{Response.}} The offline scoring cost is amortized after approximately 31,000–37,400 queries under the Table 8 resource settings. The break-even query count is

NBE=⌈3600HofflineTendpoint−TSBH⌉.N_{\mathrm{BE}}=\left\lceil\frac{3600H_{\mathrm{offline}}}{T_{\mathrm{endpoint}}-T_{\mathrm{SBH}}}\right\rceil.

Amortization calculated from Table 8. The calculation uses comparable GPU resources and one GPU allocated throughout each online query. Fitting and calibration times are reported separately in seconds.

BackboneOffline scoring (GPU-h)Fit + calibration (s)Endpoint (s)SBH (s)Break-even queries
Qwen2.5-VL26.71.3610.847.7431,007
LLaVA-OneVision29.81.4411.308.4337,380
InternVL2.528.51.3912.138.7930,719

For Qwen, 50,000 and 100,000 queries yield calculated net savings of 16.36 and 59.41 GPU-h after the 26.7 GPU-h scoring cost. These GPU allocation-time savings make repeated deployment workloads the main use case.

Predictor reuse and calibration reuse have different conditions. A fixed source predictor β\beta can receive a target-domain guarantee by calibrating ρ\rho on independent target videos exchangeable with deployment videos. The source-only rows below evaluate direct reuse empirically, and the target-calibrated rows evaluate adaptation to the new domain.

Qwen2.5-VL transfer from Video-MME, α=0.10\alpha=0.10. Each dataset uses the same target evaluation videos across rows. Source-β\beta rows reuse the fitted predictor and change only calibration. Target-β\beta rows use target fitting and calibration. JJ measures any early-stop reference disagreement per video. CsimC_{\mathrm{sim}} measures coverage across all questions and eligible prefixes in a video, including prefixes after stopping. The GPU-h column reports additional calibration scoring for source-predictor reuse.

Targetβ\betaCalibration videosρ\rhoCsimC_{\mathrm{sim}} (%)JJ (%)Early (%)Latency (s)New scoring (GPU-h)
LongVideoBenchSourceSource 1800.21486.3411.4594.109.140.0
LongVideoBenchSourceTarget 500.25892.517.0586.409.582.8
LongVideoBenchSourceTarget 1500.23991.196.1788.609.268.3
LongVideoBenchTargetTarget 1500.22693.395.2990.728.89—
MLVUSourceSource 1800.21486.6910.3693.707.040.0
MLVUSourceTarget 500.23691.426.5187.607.522.4
MLVUSourceTarget 2240.23089.645.6289.107.3310.6
MLVUTargetTarget 2240.19894.084.7391.287.12—

Reusing β\beta with 50 target calibration videos reduces observed risk from 11.45% to 7.05% on LongVideoBench and from 10.36% to 6.51% on MLVU, at 2.8/2.4 GPU-h of additional scoring. Larger target calibration sets yield 6.17%/5.62% risk. Theorem 1 gives marginal validity over calibration and new videos. Calibration uses model-generated reference scores without GT labels. A hardware-only change permits rule reuse when predictions are unchanged. Changes to the model, input protocol, observation policy, reference, or deployment distribution call for calibration under the updated setting.

Reviewer_6hSiOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Stability-Based Halting (SBH) to determine when to stop expanding the observed video input and thereby reduce inference time. Specifically, SBH uses a lightweight linear predictor that combines current confidence, observation progress, and coarse cues from the remaining video to estimate the current answer's advantage over alternative answers under an expanded video view. SBH stops acquiring further video evidence when the predicted advantage exceeds a correction determined using an independent calibration set. The experiments show improved efficiency while maintaining comparable accuracy.

Critical strengths
  • The paper addresses an important problem: determining when to stop acquiring additional video evidence. Using a linear predictor to estimate the current answer's advantage under an expanded view is also a natural approach.
Critical weaknesses
  • The comparisons are incomplete. The Budget column in Table 1 should explicitly report the average cumulative number of processed visual tokens and frame instances, and latency should be compared against complete systems such as FOCUS under consistent evaluation settings. Table 2 supports the effectiveness of the proposed stopping strategy relative to the other stopping strategies
  • The method is built on a simple workflow. Its applicability to agent-based methods such as VideoChat-A1 has not been evaluated, nor has it been compared against such methods to establish the practical value of this approach.
Questions for authors
  • Regarding Weakness 1, could the authors provide the missing measurements in Table 1?
  • Regarding Weakness 2, could the authors extend the proposed halting predictor to other video-agent frameworks, or compare it against them, to demonstrate SBH's practical competitiveness relative to complete video-agent systems?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 6hSi (1/2): Cumulative resources and FOCUS

We appreciate the time you spent reviewing our work and your valuable suggestions on system-level evaluation. Below, we report cumulative resource use and comparisons with complete video-understanding systems.

W1 / Q1. Cumulative resources in Table 1 and complete FOCUS latency.

Response.{\color{green}\mathbf{Response.}} In Table 1, ≤64\leq64 frames is the cap on final unique fine frames. Cumulative accounting includes repeated processing across prefixes. Table 8 reports 52.4 fine-frame instances, 2.61 calls, and 7.74 s for Qwen SBH on the held-out split. Its endpoint uses 64 instances, one call, and 10.84 s. The table below follows Table 1's full-benchmark protocol, with each target dataset excluded from fitting, development, and calibration.

Full-benchmark cumulative resources. These rows correspond to the 12 SBH entries in Table 1. QA frame instances include repeated processing. The selector column records its own frame count. Video-MME Overall and Long share the same full-benchmark inference run.

BackboneDatasetCumulative QA frame instancesCumulative QA visual tokens†Selector frame instancesQA callsLatency (s)
Qwen2.5-VLVideo-MME54.36,950322.687.89
Qwen2.5-VLLongVideoBench59.17,565322.869.31
Qwen2.5-VLMLVU53.66,861322.637.37
Qwen2.5-VLLVBench63.78,154323.0110.29
InternVL2.5Video-MME56.08,960322.718.96
InternVL2.5LongVideoBench62.810,048322.9110.21
InternVL2.5MLVU57.79,232322.748.44
InternVL2.5LVBench66.210,592323.0810.89
LLaVA-OneVisionVideo-MME58.68,438322.758.52
LLaVA-OneVisionLongVideoBench63.59,144322.949.65
LLaVA-OneVisionMLVU56.18,079322.698.13
LLaVA-OneVisionLVBench67.99,778323.1411.12

†Measured mean cumulative visual-token positions newly processed by the QA language model. A shared candidate prefix is counted once per call. Each observation prefix is processed afresh, so repeated frames count again. The separate 32-frame SigLIP index processes 6,272 encoder patch positions per query. End-to-end latency includes all stages.

FOCUS [1] performs frame selection before QA. Both stages are included below. These comparisons use the held-out evaluation splits, LLaVA-OneVision-7B, visual-only input, and one A100. First-use timing starts from the compressed video and ends with the final answer.

Complete FOCUS and uniform-sampling pipelines. All cumulative counts are per question. Latency includes decoding, selection, QA, and other overhead.

DatasetSystemAccuracy (%)QA instancesSelector instancesLatency (s) [95% CI]
Video-MMEUniform-3256.5432.007.57 [7.27, 7.87]
Video-MMEUniform-6457.2864.0010.79 [10.49, 11.09]
Video-MMEFOCUS-3258.2732.0968.99 [8.69, 9.29]
Video-MMEFOCUS-6458.8964.012812.72 [12.42, 13.02]
Video-MMESBH58.6457.8328.43 [8.13, 8.73]
LongVideoBenchUniform-3259.7932.009.50 [9.07, 9.93]
LongVideoBenchUniform-6460.2464.0013.11 [12.68, 13.54]
LongVideoBenchFOCUS-3261.0132.011211.41 [10.98, 11.84]
LongVideoBenchFOCUS-6461.5464.015215.69 [15.26, 16.12]
LongVideoBenchSBH61.2061.4329.63 [9.20, 10.06]

On Video-MME and LongVideoBench, SBH’s accuracy is 0.37 and 0.19 percentage points higher than FOCUS-32, respectively. It also reduces latency by 0.56 and 1.78 s. Against FOCUS-64, SBH achieves 1.51× and 1.63× speedups, with accuracy differences of −0.25 and −0.34 percentage points. The latency advantage holds after including frame selection.

[1] FOCUS: Efficient Keyframe Selection for Long Video Understanding.

AuthorsOpenReview ↗

Response to Reviewer 6hSi (2/2): Complete video-agent comparison

W2 / Q2. Practical competitiveness with complete video agents.

Response.{\color{green}\mathbf{Response.}} SBH asks whether the current answer will survive additional video evidence. A confident answer can still be overturned by an unseen event. The remaining-video cue helps predict that possibility, and video-level calibration controls disagreement jointly across questions and prefixes. This lets a frozen QA model decide how much evidence to acquire using only five fitted coefficients.

To address your request for an agent comparison, VideoChat-A1 [1] retains its full sequence of shot selection, partitioning, and reflection. The table includes both its default and low-budget configurations.

Complete-agent comparison. Both systems use Qwen2.5-VL-7B, the same held-out evaluation videos, visual-only input, and one A100. Total time includes retrieval, agent iterations, and final answering.

DatasetSystemAccuracy (%)Cumulative QA instancesSelector instancesTotal latency (s) [95% CI]
Video-MMEVideoChat-A1 default68.40116.022419.80 [19.50, 20.10]
Video-MMEVideoChat-A1 low budget67.9080.016013.65 [13.35, 13.95]
Video-MMESBH68.0252.4327.74 [7.44, 8.04]
LongVideoBenchVideoChat-A1 default61.60130.025624.80 [24.37, 25.23]
LongVideoBenchVideoChat-A1 low budget60.7091.017617.90 [17.47, 18.33]
LongVideoBenchSBH60.3157.9328.89 [8.46, 9.32]

SBH achieves 2.56× and 2.79× speedups over the default agent on Video-MME and LongVideoBench, with accuracy differences of −0.38 and −1.29 percentage points. Speedups over the low-budget agent are 1.76× and 2.01×. SBH offers a faster option for these workloads, with the accuracy trade-offs shown above.

[1] VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoning.

Reviewer_m8xdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SBH for efficient long-video question answering. SBH combines current answer confidence, observation progress, and coarse remaining-video cues to predict whether the current answer will remain unchanged under a fixed expanded video view. Video-level conformal calibration provides a stopping criterion with a finite-sample guarantee on early-stop disagreement with the reference prediction, jointly across questions and observation steps. Experiments across multiple benchmarks and three frozen vision-language models show approximately 1.34–1.40× end-to-end speedups on Video-MME while maintaining comparable answer accuracy.

Critical strengths

The paper’s main contribution is an effective extension of existing calibrated stopping approaches to long-video question answering. By incorporating remaining-video cues and video-level calibration, it enables theoretically grounded stopping decisions and demonstrates practical end-to-end speedups while maintaining comparable accuracy. Although the method builds on established consistency-calibration frameworks, its design for video evidence acquisition and demonstrated empirical benefits constitute a valuable contribution.

Critical weaknesses
  1. The benefit of retained-margin regression over consistency classification is not yet clear. The classifier baseline uses the same features but does not receive comparable video-level risk calibration. This makes it hard to tell how much of the improvement comes from the prediction target and how much comes from calibration.

  2. The baselines choose stopping thresholds to maximize accuracy under a latency constraint, while SBH uses a preset risk target. The results show that SBH performs better at the reported settings, but it remains unclear whether it is faster when the methods are compared at the same risk level.

Questions for authors

Please see weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer m8xd (1/2): Regression and equal calibration

Thank you for your time and thoughtful comments on the stopping formulation. We address your questions below through equally calibrated comparisons and evaluation at comparable risk.

W1. Retained-margin regression versus equally calibrated consistency classification.

Response.{\color{green}\mathbf{Response.}} To isolate the effect of the prediction target, both predictors use identical features and video-level calibration. Retained-margin regression achieves lower latency on Video-MME. Its target

γt=pH(at)−max⁡a≠atpH(a)\gamma_t=p_H(a_t)-\max_{a\ne a_t}p_H(a)

measures the current answer's signed advantage under the expanded reference. It preserves how strongly that reference supports or rejects the answer. The binary target yt=1[at=aH]y_t=\mathbf{1}[a_t=a_H] records agreement, following the consistency-classification approach in CATs [1].

For the classifier score st∈[0,1]s_t\in[0,1], use the same calibration rule as Eq. (5):

R_i^{\mathrm{cls}}&=\max_{q\in\mathcal Q_i}\max_{t<T_{i,q}}(s_{i,q,t}-y_{i,q,t}),\\\\ k&=\lceil(n+1)(1-\alpha)\rceil,\qquad \rho_{\mathrm{cls}}=\max\\{0,R_{(k)}^{\mathrm{cls}}\\}. \end{aligned}$$ Here $n$ is the number of calibration videos, with $R_{(n+1)}^{\mathrm{cls}}=+\infty$. Stop when $s_t-\rho_{\mathrm{cls}}>0$. On the simultaneous coverage event, this implies $y_t>0$, hence reference agreement. Under the assumptions of Theorem 1, both predictors receive the same video-level disagreement guarantee. **Equally calibrated comparison.** Qwen2.5-VL / Video-MME, $\alpha=0.10$, with identical features and data splits. $J$ is the percentage of test videos containing any early-stop reference disagreement. The regression rows reproduce Table 6. The classifier in Table 2 uses a development-selected threshold. Here both classifier variants receive independent video-level calibration. | Predictor | Remaining-video cue | $J$ (%) | Early (%) | Accuracy (%) | Latency (s) | | --- | --- | --- | --- | --- | --- | | SBH regression | Yes | 5.19 | 92.35 | 68.02 | 7.74 | | Regression | No | 5.93 | 88.15 | 67.78 | 8.05 | | Calibrated classifier | Yes | 5.56 | 90.86 | 67.78 | 7.99 | | Calibrated classifier | No | 6.30 | 87.41 | 67.53 | 8.28 | **Paired latency differences with 95% confidence intervals.** Negative values favor regression. Both predictors include the remaining-video cue. | Dataset / backbone | SBH minus calibrated-classifier latency (s) [95% CI] | | --- | --- | | Video-MME / Qwen2.5-VL | −0.25 [−0.44, −0.04] | | Video-MME / InternVL2.5 | −0.24 [−0.45, −0.03] | | LongVideoBench / Qwen2.5-VL | −0.23 [−0.48, +0.02] | Regression reduces latency by 0.25 s and 0.24 s on Video-MME with Qwen and InternVL, respectively. Both paired intervals exclude zero. On LongVideoBench, the point estimate favors regression by 0.23 s, with a confidence interval of [−0.48, +0.02] s. [1] Consistent Accelerated Inference via Confident Adaptive Transformers.
AuthorsOpenReview ↗

Response to Reviewer m8xd (2/2): Efficiency at comparable risk

W2. Efficiency at comparable empirical risk.

Response.{\color{green}\mathbf{Response.}} To compare efficiency at similar risk, each method's α\alpha is selected against a common pilot risk target and frozen before independent calibration and testing. Pilot calibration and pilot evaluation each use 45 development videos. Model tuning uses video-level cross-validation within the fit split. JJ counts test videos with any early-stop reference disagreement, using all 270 test videos as the denominator.

Development-selected operating points: Qwen2.5-VL / Video-MME. Test results are evaluated only after selection is frozen.

Pilot target (%)MethodSelected α\alphaPilot JJ (%)Test JJ (%)Test latency (s)
5SBH0.104.445.197.74
5Calibrated classifier0.104.445.567.99
5Calibrated margin0.104.445.939.12
5Calibrated entropy0.0754.444.4410.40
5Calibrated TV0.154.444.4410.96
10SBH0.208.898.897.18
10Calibrated classifier0.208.899.267.44
10Calibrated margin0.2011.119.638.21
10Calibrated entropy0.2011.1110.008.41
10Calibrated TV — unmatched0.206.676.3010.05

At nearby test risks, SBH saves 0.25–0.26 s per query relative to the same-feature classifier. The pilot's risk resolution is 2.22 percentage points, so the test columns report the risk each rule actually attains. TV remains more conservative at the 10% target within the prespecified α≤0.20\alpha\leq0.20 grid.

Risk–latency comparison near 4% test risk. These rows come from the prespecified grid and provide a descriptive comparison. The pilot-selected operating points above are unchanged.

Methodα\alphaTest JJ (%)Latency (s)
SBH0.0754.078.07
Calibrated classifier0.0754.448.35
Calibrated margin0.0754.4410.11
Calibrated entropy0.0754.4410.40
Calibrated TV0.154.4410.96

SBH also has the lowest latency here, at slightly lower observed risk.

I appreciate the authors for the detailed response which has addressed most of my concerns. I will keep my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.