- Rating
- 2
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies whether future hallucinations in VLM responses can be predicted before the first unsupported token is emitted. It defines a response-level false-alarm protocol, constructs PREHALLU with onset-annotated trajectories and prefix-conditioned continuations, and introduces trained risk predictors under the RISKTRACE framework. Experiments report improvements over initial prediction across distinct models, together with warning lead times and alarm-triggered interventions.
Critical strengths
- Practically relevant task formulation. Early warnings enable targeted verification or correction, while response-level false-alarm accounting captures the cost of repeated monitoring.
- Useful experimental controls. Comparisons with initial prediction and optimized static baselines, together with lead-time analysis, frozen-threshold evaluation, and a future-information leakage audit, support the presence of additional predictive information during generation.
Critical weaknesses
- Unclear advantage over simpler dynamic predictors. Existing ablations support the value of historical information but do not establish the necessity of the full fusion design. Removing some gates does not reduce low-false-alarm recall, and a simple token-peak predictor outperforms the main dynamic implementation on this metric in some settings.
- Incomplete evaluation of practical warning value and cost. Predictors differ across backbones, and LLaVA additionally samples auxiliary continuations, introducing different evidence sources and computational budgets. Existing timing measurements do not cover the complete monitoring pipeline, while some Qwen3 warnings occur immediately before onset. Despite encouraging intervention results, warning coverage, available intervention time, and end-to-end cost still need to be assessed jointly.
Questions for authors
- Beyond existing ablations, could the authors compare against independently optimized per-prefix classifiers and fixed early-window baselines under matched information and compute budgets? Could they also shuffle or replace input histories before rescoring to test the value of historical content and order?
- Could the authors report end-to-end latency separately for passive and continuation-assisted monitoring and, with frozen thresholds, jointly report minimum-lead recall and the actual time required for scoring and intervention?
4 comments
Response to W2: Practical Value, Zero-Lead Interception, and Runtime
W2. Practical value, zero-lead interception, and runtime
We agree that practical value requires measuring coverage, executable interventions, answer quality, and total latency separately.
Quality and usable prefixes. In the reported intervention experiment, visual selection improves over the same-alarm reject-and-resample baseline, reducing Qwen3 CHAIRs from 10.175% to 9.775% on Fresh2 and from 9.875% to 9.475% on Fresh3, a 0.40 pp decrease in each case. On a separate fixed set of 200 inputs, original, alarm-time grounding, and post-hoc grounding give CHAIRs of 9.50%, 9.50%, and 10.00%; post-hoc grounding introduces one additional error. Alarm-time grounding generates 3.14 fewer tokens on average (paired 95% CI [-4.765,-1.850]). This supports prefix reuse and reduced generation, without establishing a significant quality or wall-clock improvement in that experiment.
At Development-frozen thresholds, coverage is:
| Model / split | Event recall | Actual FAR | Lead>=5 | Lead>=10 |
|---|---|---|---|---|
| InstructBLIP | 49.28% | 5.36% | 41.10% | 30.51% |
| Qwen3 Fresh | 17.05% | 2.64% | 5.53% | 5.07% |
| Qwen3 Fresh2 | 12.79% | 3.14% | 4.11% | 4.11% |
Lead coverage uses all event responses as its denominator. These are separate backbone-specific populations and thresholds. Longer token lead provides earlier warning; the wall-clock opportunity also depends on generation speed, action cost, and scheduling.
Zero lead can still permit interception. Qwen3's median lead among successful alarms is zero. State t has consumed t answer tokens; the next token has zero-based index t. If the first error is at k and the alarm occurs at state k, L=k-k=0, yet that token has not been committed. Our synchronous controller computes the state, waits for the risk decision, and pauses on an alarm; otherwise, it commits the next token. Pausing preserves the correct prefix for grounding, abstention, resampling, or candidate selection before the error is exposed.
We tested 48 retrospectively stratified COCO images with ordinary decoding, monitoring only, and interception before token commitment: 144 real greedy decodes, with the model, three-seed detector, and Selection threshold frozen. Because new answers can differ from archived trajectories, a supplementary analysis relabeled all new answers after decoding using the same frozen CHAIR evaluator:
- Ordinary and monitoring-only outputs were token-identical on 48/48 images.
- Of 24 historical zero-lead cases, 22 again triggered at zero lead. All 22 withheld the first erroneous token; monitoring-only controls emitted that same token in 22/22 cases.
- The other two cases did not alarm and were not intercepted.
The detector reads neither onset labels, candidate-token identity, nor future answers. The 22/22 result is conditional on triggered alarms, not population recall or corrected-answer accuracy. The original strict archive-matching analysis separately gives 9/9; we retain both analyses. Commitment was audited at the output callback.
Full runtime and acceleration. In this 48-image feasibility experiment, alarm computation averages 6.51 ms per state (P95 8.53 ms), and complete monitoring-only answers add 0.223 s on average. Each mode ran once per image on a shared server; shorter stopped outputs do not demonstrate faster complete answers.
A separate 200-input experiment used 20 warmups and three repetitions. Timing includes preprocessing, input transfer, generation, state capture, risk computation, updates, thresholding, and synchronization; it excludes disk reads and one-time model loading.
| Mode | Mean E2E | P90 |
|---|---|---|
| Natural generation | 0.8179 s | 1.0160 s |
| Full passive monitoring | 1.1523 s | 1.3451 s |
Overhead is 0.3344 s (40.9%); mean per-state monitoring is 7.789 ms. All 600 paired outputs are token-identical, and 3,452 legal states show no online-versus-cached alarm flips.
A separate 32-input evaluation includes corrective candidates, object checking, and selection:
| Mode | Mean E2E | P90 |
|---|---|---|
| Natural generation | 1.5673 s | 2.0062 s |
| Serial full controller | 2.9541 s | 3.0637 s |
| Accelerated full controller | 1.7504 s | 2.2741 s |
Our earlier work focused primarily on monitoring effectiveness, with limited efficiency optimization. This comparison demonstrates scope for acceleration: runtime falls 40.7% versus serial execution, with 183.1 ms (11.7%) overhead over natural generation. Only two inputs triggered correction, so this is engineering evidence, not a large-scale deployment estimate. In a separate small passive-pipeline experiment, token buffering, incremental scoring, CUDA Graphs, and detector heads on a second GPU reduce overhead from 22.1% to 4.35%. These timings use different settings and are reported separately.
Response to Q1: Prefix Classifiers, Early Windows, and History Interventions
Q1. Independent prefix classifiers, fixed early windows, and history interventions
Thank you for suggesting these comparisons. We have evaluated independently optimized Per-prefix MLP and Simple GRU predictors, fixed early-window models, and interventions on explicit history. These experiments address different aspects of dynamic observation. Here pp denotes percentage points, and R@5 is recall under a response-level FAR<=5% budget.
Independent classifiers and fixed windows. The original fixed-scorer observation-window analysis holds the alarm model unchanged and restricts the prefix range it can observe. Extending the range to K=20 gives 83.59% AUROC and 47.57% R@5. Thus, observing additional legal prefixes can improve prediction without retraining the scorer.
In the separate mixed Test comparison, Full-original, Per-prefix MLP, and Simple GRU achieve AUROCs of 59.58%, 53.98%, and 53.68%. Full's gains are 5.60 pp over MLP (95% CI [+3.95,+7.26]) and 5.91 pp over GRU ([+4.25,+7.56]). These AUROC comparisons do not imply significant R@5 gains.
We also trained fixed early-window models with the same future-event supervision:
| Model | K=5 R@5 | K=10 R@5 | K=20 R@5 |
|---|---|---|---|
| Per-prefix MLP | 9.98% | 9.52% | 6.76% |
| Full | 8.91% | 9.83% | 9.98% |
MLP performs better at K=5, while Full has higher point estimates at the longer windows; its K=20 advantage is 3.22 pp. We therefore distinguish independently trained dynamic predictors, retrained fixed-window models, and restricting the observation range of a fixed scorer. The 83.59%/47.57% fixed-scorer result and the mixed Test comparisons are different evaluations and should not be read as one experimental population.
History order and content. We kept the current and initial representations unchanged and altered only the explicit history sequence consumed by Full-control. R@5 is:
| History setting | Fresh | Fresh2 | Fresh3 |
|---|---|---|---|
| Full-control | 20.74% | 16.89% | 15.62% |
| Shuffled order | 23.96% | 21.00% | 21.88% |
| Replacement content | 22.58% | 17.81% | 18.27% |
For shuffled order on Fresh3, the simultaneous 95% CI for the difference from Full-control is [+1.57,+10.93] pp. This is evidence against a claim that the particular explicit history order is always necessary. Current states and other evidence paths already encode prefix information, which may limit the need for an additional ordered history sequence.
The replacement-content difference intervals cross zero. These results do not establish that replacing history degrades performance or that explicit history content is indispensable. They show sensitivity to history interventions, while leaving its necessity unsupported in these settings.
Our conclusion is therefore limited: continued observation of legal prefixes can be useful, but that does not establish that a particular explicit history structure is necessary. The full model has an overall ranking advantage in the independent-predictor comparison and higher recall point estimates at longer trained windows, while simple models remain competitive. We will make this distinction explicit rather than infer the necessity of every temporal component from the value of dynamic monitoring.
Response to Q2: LLaVA Rollout Costs, Distillation, and the Core Finding
Q2. Auxiliary rollouts, distillation, and the core finding
Thank you for asking about the cost of continuous auxiliary monitoring. We distinguish the ability to identify future risk while the visible answer is still correct from the computational budget used to obtain that evidence. Passive monitoring and auxiliary rollouts have different costs. The passive coverage, complete Qwen3 runtime, zero-lead interception, and serial-versus-accelerated controller measurements are reported in our response to W2.
LLaVA auxiliary evidence and cost. The auxiliary path samples possible continuations from the current prefix and evaluates their risk. It does not read the original answer's actual future suffix or its future error labels. It can therefore identify risk before the original answer reaches its first error, although obtaining this estimate takes additional time.
On LLaVA Fresh, this path gives 40.38% event recall at 6.34% actual FAR, with Lead>=5 and Lead>=10 coverage of 20.88% and 7.97%. Separately, a complete synchronous timing study on 200 inputs using LLaVA with a local Qwen judge gives 1.431 s for natural generation and 32.699 s for auxiliary monitoring. This is substantial overhead. The local-judge timing variant differs from the historical effectiveness configuration; these numbers are not a joint quality-latency estimate for one identical system.
Existing results without additional rollouts. We also recovered and recomputed earlier LLaVA results:
| Setting | Own-t0 R@5 | Dynamic R@5 | Evaluation scope |
|---|---|---|---|
| Sampled original answers | 4.46% (10/224) | 8.48% (19/224) | 600 independent Test images |
| v101: suppress clean false peaks | 10.89% (441/4049) | 13.51% (547/4049) | 12,000 training answers |
| v103: causal global deadzone | 10.89% (441/4049) | 14.10% (571/4049) | Same training population |
R@5 uses empirically matched response-level FAR<=5%; pp denotes percentage points. Lead coverage uses all event responses as its denominator. In the first experiment, sampling is the original answer's decoding strategy; the monitor generates no additional continuation. Dynamic recall improves by 4.02 pp, capturing nine more events (paired image-bootstrap 95% CI [+0.41,+8.50] pp). Actual FAR is 3.72% for t0 and 4.52% for dynamic. AUROC does not improve, so the positive result concerns low-FAR recall.
That early monitor uses the probability of the currently selected candidate token. It is a candidate-available interface without additional rollouts; online commitment timing for this historical model has not been audited. It is not evidence of forecasting before candidate selection using only the visible prefix.
v101/v103 use legal prefixes and observed history, with no inference-time rollouts. Their training-set gains capture 106 and 130 additional events, supporting the ability to fit dynamic signals. Generalization is not established: on independent Confirmation, v103 R@5 is 8.22%, versus 11.33% for its own t0. We report training diagnostics separately from held-out evidence.
Distillation as a concrete cost-reduction path. Rollouts can be moved to training: a teacher estimates future hallucination risk from multiple continuations, and a student learns this soft target from current hidden states, visual information, and observed history. At inference, the student predicts risk directly, without continuation generation or judging.
We have implemented and evaluated such a student. On a retrospective evaluation of historical disjoint images, the full-hidden-state student achieves 12.09% dynamic R@5 versus 11.26% at its own t0. The +0.82 pp gain has a 95% CI [-1.70,+4.75] pp. This validates an executable interface without online rollouts, but does not establish a dynamic gain or preservation of the strong teacher's performance. The student also requires an additional substituted-image forward pass, and its full online cost remains to be measured. Distillation offers a specific optimization route; lossless removal of rollout cost has not been demonstrated.
Core finding. Even without these supplementary no-rollout results and student experiments, the original pre-onset evaluations support the central finding: risk can be identified while the visible answer prefix is still correct. Auxiliary samples estimate possible futures from that prefix; they do not wait for the original answer to become erroneous. Computational cost affects practical applicability and intervention benefit and must be measured explicitly. Distillation, incremental computation, and parallel execution offer ways to reduce it, with concrete Qwen3 acceleration already measured. Improving efficiency and preserving LLaVA teacher performance remain separate goals; the core finding does not depend on claiming that every implementation is already inexpensive.
Response to W1: Simple Dynamic Predictors and Component Ablations
W1. Comparison with simpler dynamic predictors and component ablations
Thank you for the careful review and for recognizing the value of pre-onset hallucination forecasting and the response-level false-alarm rate (FAR) protocol. We distinguish comparisons with independently trained predictors from ablations within the full model.
Comparison with simpler predictors. On the mixed Test set, Full-original, Per-prefix MLP, and Simple GRU achieve AUROCs of 59.58%, 53.98%, and 53.68%. With the same onset target and matched update budgets, Full improves over MLP and GRU by 5.60 and 5.91 percentage points (pp), with 95% CIs [+3.95,+7.26] and [+4.25,+7.56]. This supports better overall risk ranking in this setting; it does not imply superiority on every metric.
The Token-Peak result concerns a particular Qwen3 setting. Token-Peak retains the full network's visual, cross-layer, and GRU representations, but uses only the temporal maximum of the token-risk head. Full also incorporates the initial risk anchor and full risk aggregation. Their tradeoff is:
Here R@5 is recall under an empirically matched response-level FAR<=5% budget; Lead>=5 is coverage over all event responses. Token-Peak captures more events near onset, whereas Full has higher point estimates for earlier coverage. We do not claim significance for these lead-coverage differences or extrapolate this Qwen3 comparison to InstructBLIP or LLaVA.
With the same upstream signals and matched retraining, Full-control also improves R@5 over simple layer averaging by 6.91, 7.08, and 4.81 pp across the three Fresh batches. This supports learning how to organize cross-layer evidence, beyond merely supplying additional inputs.
Component roles. In the reported InstructBLIP ablation, Full achieves 83.58% AUROC. Removing Critical Attention, Champion-tail Protection, or all tail protection gives 81.34%, 81.37%, and 81.70%. The existing out-of-fold refit using only the actual-current component gives 80.42%. These results support complementary ranking information; they do not establish that every layer or submodule is individually indispensable.
Gate serves a different purpose: protecting the response-level false-alarm budget when auxiliary branches produce anomalously high scores. Under a stress test corrupting all four auxiliary branches, the same frozen threshold gives:
Gate reduces FAR by 11.58 pp, with a simultaneous 95% CI [+9.88,+13.28]. The higher recall without Gate accompanies more false alarms and cannot alone establish better prediction. After separately matching FAR<=5% under the same corruption:
This supports false-alarm protection and ranking robustness under auxiliary-branch failures. It is a controlled stress test, not evidence of universal gains on naturally shifted data. The full design therefore offers benefits in ranking, evidence organization, and earlier coverage, while simpler predictors remain competitive on some metrics.