- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Adaptive Contrast with Token Interventions for Video (ACTIV), a training-free framework for mitigating spatial and temporal hallucinations in video-language models. ACTIV constructs spatial and temporal negative representations by averaging already-encoded visual tokens across spatial locations or time steps, respectively, avoiding the repeated vision-encoder passes required by approaches such as SEASON. During decoding, it combines the two contrastive signals using weights derived from Jensen–Shannon divergence between next-token prediction distributions. The paper also introduces the Video Evidence Margin (VEM), which compares the likelihood of a generated answer under original visual tokens with its average likelihood under random feature replacements, as a diagnostic proxy for visual-feature dependence.
Critical strengths
- Practical reduction in vision-encoder computation.
ACTIV constructs contrastive negatives directly from existing visual tokens, avoiding repeated vision-encoder passes. The reported latency breakdown supports this design choice: visual-processing costs are substantially lower than those of SEASON, while total latency is reduced by approximately 22–26% in the principal comparison. This is a meaningful systems contribution, although the method remains slower than standard decoding and retains considerable decoder-prefill overhead.
Critical weaknesses
-
Insufficiently established novelty relative to existing works.
ACTIV shares the central design of spatial–temporal contrastive decoding with SEASON. Its main changes are to construct negative representations by averaging existing visual tokens and to weight the contrastive branches using JSD. Although these changes reduce visual-encoder computation, the paper needs to establish more convincingly that they offer a substantive methodological advance beyond a computationally motivated replacement of existing components. -
Limited validation on newer model generations.
The main evaluation uses Qwen2.5-VL-7B, LLaVA-OneVision-7B, and LLaVA-Video-7B, the same established model families used in prior work in this area. Newer model generations, including Qwen3-VL and LLaVA-OneVision-2, were available before this review, yet ACTIV is not evaluated on them.
Questions for authors
-
Clarification of the methodological contribution.
Given the similarity to SEASON and the existence of other single-encoding interventions, what evidence demonstrates that ACTIV's token-space averaging and JSD-based weighting provide a substantive methodological advantage rather than primarily replacing existing components to reduce encoder cost? A convincing answer supported by existing ablations or additional controlled results could materially change my assessment of the paper's novelty. -
Evidence of generalization to newer model generations.
Can the authors provide results on at least one newer publicly available video-language backbone on a representative hallucination benchmark?
5 comments
Detailed answers
1. Clarification of the methodological contribution
You asked for controlled results. We ran a controlled comparison that changes one component at a time:
- the negatives: SEASON's (re-encoded video) or ACTIV's (token averages);
- the weights: SEASON's (JSD of attention over frames) or ACTIV's (JSD of next-token distributions, Eq. 5).
All rows use LLaVA-Video-7B, the full benchmarks, the same frames, the same decoder and α = 1. Rows 2–5 also use the same candidate filter (λ = 0.1, lines 267–269).
| # | Negatives | Weights | Encoder passes | VidHalluc | VideoHallucer | EventHallusion | Overall | Latency (s) |
|---|---|---|---|---|---|---|---|---|
| 0 | none (unmodified model) | – | 1 | 78.26 | 49.86 | 67.24 | 65.12 | 0.41 |
| 1 | SEASON | Attention JSD, no filter (SEASON as published) | 3 | 79.22 | 47.08 | 71.15 | 65.82 | 1.12 |
| 2 | SEASON | Attention JSD | 3 | 79.54 | 47.08 | 71.15 | 65.92 | 1.11 |
| 3 | SEASON | Output JSD | 3 | 79.94 | 47.52 | 72.13 | 66.53 | 1.11 |
| 4 | ACTIV | Attention JSD | 1 | 82.10 | 49.64 | 71.88 | 67.87 | 0.82 |
| 5 | ACTIV | Output JSD | 1 | 82.15 | 50.02 | 72.62 | 68.26 | 0.82 |
Overall is the mean of the three benchmark scores. Latency is the median on VidHalluc. Rows 1–3 use our implementation of SEASON (its code is not public), with a setting from the SEASON paper: α = 1 and β = 0.33. Rows 4–5 use ACTIV's setting from the paper (lines 300–301). We tuned no setting for this comparison. ACTIV's advantage does not depend on this α: at each tested α from 0.25 to 2, ACTIV is 1.17–3.14 points above the unmodified model. SEASON is 0.70 points above it (row 1).
The table shows three facts:
- The token negatives are more accurate, not only cheaper. With the weights fixed, ACTIV's negatives add 1.95 points (row 2 → 4) and 1.73 points (row 3 → 5) to the overall score. The score increases on all three benchmarks in both cases. Eq. 4 and Figure 3 give the reason: token averaging removes an exact fraction of the variation along time or space. A change before or inside the encoder does not give this control.
- The output-JSD weights add a further gain at no cost. They add 0.39 points with ACTIV's negatives (row 4 → 5) and 0.61 points with SEASON's own negatives (row 2 → 3). The gain is positive on all three benchmarks in both cases. These weights need no attention maps.
- Together, the two changes add 2.44 points over SEASON as published (row 1 → 5). At the same time, ACTIV uses one encoder pass instead of three and has 26% lower latency.
Thus, ACTIV is not SEASON with cheaper parts: the largest gain comes from the new negatives. The paper agrees with this on the other backbones. Table 4(b) isolates the temporal negative on LLaVA-OneVision: ACTIV's negative is 3.62 and 1.72 points above SEASON's. In Table 1, the full method is 2.73 and 2.83 points above SEASON overall on Qwen2.5-VL and LLaVA-OneVision.
Other single-encoding methods. STEAR needs attention maps, a selection of layers and uncertainty thresholds (lines 462–465). ACTIV needs only the encoded tokens and the next-token distributions. Thus, it has no attention layers or uncertainty thresholds to select.
2. Evidence of generalization to newer model generations
We evaluated Gemma 4 E4B on EventHallusion (all 409 questions). Gemma 4 is from a third model family (not Qwen and not LLaVA), so it tests a newer model and a different decoder at the same time. Both methods use transferred settings, with no tuning on Gemma 4. ACTIV uses its Qwen2.5-VL settings without change. SEASON uses the same setting as above (α = 1, β = 0.33), and its attention weights read the two global-attention layers that are nearest to the relative depth of the layers in the SEASON paper.
| Method | Encoder passes | Accuracy | Change from baseline [95% CI] | Latency (s) |
|---|---|---|---|---|
| Baseline | 1 | 53.06 | – | 0.27 |
| SEASON | 3 | 51.10 | −1.96 [−5.43, +1.47] | 0.67 |
| ACTIV | 1 | 55.26 | +2.20 [+0.24, +4.15] | 0.48 |
ACTIV is 4.16 points above SEASON (95% CI [+1.21, +7.43]) and 28% faster. The intervals use bootstrap resampling of the source videos. On the two subsets where each correct answer is "no", ACTIV does not add "yes" answers: it gives 78, the same number as the unmodified model. SEASON gives 95.
Changes in the revision. We will add the controlled comparison and the Gemma 4 result to Section 4.5.
You wrote that a convincing controlled result can change your assessment of the novelty. We hope that the comparison above is this result. If a concern remains, please tell us. We will answer it during the discussion period.
Thank you for your time and comments
Dear Reviewer xQ22,
We hope all is well and would like to thank you again for providing reviews to our submission.
We are wondering if you have a chance to read our rebuttal response. If so, do you have any more issue that you would like us to clarify and/or provide any more results, given the author-reviewer exchanges will be closing in less than a day?
Thank you in advance for your timely feedback.
best, The Authors
Controlled comparison with SEASON, and a newer backbone (Gemma 4)
Thank you for the clear review. We are glad that you find the reduction in vision-encoder computation "a meaningful systems contribution", and that you find the paper "clear and easy to follow". Thank you also for the statement of what can change your assessment. You raised two concerns. We answer each one with a new result.
Short answers