- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work investigate a generation-discrimination gap issue, i.e., VLM can describe counterfactual image without stating that it is unreliable. The paper first anaylsizes hidden states, attention and output probabilities and then propose BRIDGE to train a small directional projector to select verification prompts of different strengths. Empirically, BRIDGE improves recognition of unrealiable input while increasing inference cost.
Critical strengths
-
This paper focuses on a practically meaningful yet under studied problem. A description can faithfully describe an unreliable image without acknowledging it is fake or counterfactual. So this study could be helpful for high stack system.
-
The paper presents deliberate ablation, showcasing meaningful improvement on several benchmarks.
-
The crafted method itself is simple and intuitive.
Critical weaknesses
-
Different model or weights will require retraining of proposed projector, making the method less portable and generalizable. It also require white-box model, making it unusable for blackbox API which is the most powerful and suitable model for high stack scenarios.
-
Benefit of BRIDGE’s routing design is insufficiently isolated.PrivGate [1] already uses latent risk signals to trigger reasoning selectively, and ReasoningGuard [2] already inserts reflection prompts after attention sinks. The existing ablations do not replace a simple probe using the same prefix features and intervention policy.
-
BRIDGE induces substantial inference cost, and can up to 2x higher latency. The paper reports this overhead, but lacks an end-to-end reliability/cost comparison with simpler prompting and routing alternatives.
[1] https://icml.cc/virtual/2026/poster/61631 [2] https://aclanthology.org/2026.acl-long.1453/
Questions for authors
-
Does Figure 3d’s conclusion survive equal-sized within/between-class comparisons, a label-permutation null, and analysis of full output vectors? Which datasets, class counts, and scene-group splits underlie each diagnostic plot?
-
How do linear and standard MLP probes perform with the same layer, prefix features, and prompt-insertion policy as BRIDGE? Can binary versus hierarchical routing be compared while holding the detector fixed?
8 comments
Response 2
Comment 2:
"W2: Benefit of BRIDGE’s routing design is insufficiently isolated...The existing ablations do not replace a simple probe using the same prefix features and intervention policy."
Re: We appreciate the request for a simpler probe under matched experimental conditions. To isolate the benefit of BRIDGE's router design, we replaced our router with Logistic Regression (Linear) and MLP classifiers, using the same cosine-center prefix features, data splits, and multi-level intervention policy. Both baselines were trained using only BCE loss, without our specialized losses in Section 3.3.
On VLind, Linear and MLP achieve higher detection PR-AUC (0.988 and 0.984) than BRIDGE (0.979). However, their performance under the same multi-level routing policy is lower:
| Method | P | R | F1 | Acc |
|---|---|---|---|---|
| MLP (cosine) | 0.804 | 0.780 | 0.792 | 0.795 |
| Linear (cosine) | 0.857 | 0.780 | 0.817 | 0.825 |
| BRIDGE | 0.867 | 0.780 | 0.821 | 0.830 |
To further examine whether a simpler single-threshold decision would suffice, we additionally evaluated binary routing with these classifiers:
| Method | P | R | F1 | Acc |
|---|---|---|---|---|
| BRIDGE | 0.867 | 0.780 | 0.821 | 0.830 |
| Linear (single threshold) | 0.796 | 0.860 | 0.827 | 0.820 |
| MLP (single threshold) | 0.721 | 0.880 | 0.793 | 0.770 |
Linear (single threshold) achieves slightly higher F1 (0.827 vs. 0.821), but its precision decreases from 0.867 to 0.796, and accuracy drops from 0.830 to 0.820. MLP (single threshold) exhibits a larger precision decrease (0.721) and lower F1 (0.793). Thus, single-threshold routing favors recall, whereas BRIDGE achieves higher precision and accuracy.
We also observed that Linear and MLP scores concentrate within narrow ranges, increasing sensitivity to threshold selection. We will include the corresponding score distributions in the revised manuscript.
Together, these experiments show that higher binary separability does not guarantee better routing performance. While a single-threshold Linear router remains competitive in F1, BRIDGE offers a more favorable precision–accuracy trade-off. The controlled multi-level comparisons further support the effectiveness of our router design and intervention policy.
Response 3
Comment 3:
"W3: BRIDGE induces substantial inference cost, and can up to 2x higher latency. The paper reports this overhead, but lacks an end-to-end reliability/cost comparison with simpler prompting and routing alternatives."
Re: We appreciate the reviewer’s attention to the practical cost of BRIDGE. We acknowledge that BRIDGE introduces additional inference overhead. However, this cost is justified by the reliability improvements and the prevention of potentially more costly downstream errors. Moreover, selective routing avoids unnecessary interventions on reliable inputs, further improving the cost–reliability trade-off.
1. Intervention frequency. We report injection rates across four datasets:
| Dataset | Positive Injection Rate | Negative Injection Rate |
|---|---|---|
| VLind | 96.0% | 22.0% |
| IMD2020 | 95.0% | 77.0% |
| Columbia | 95.0% | 25.0% |
| WHOOPS | 87.0% | — |
WHOOPS contains only positive test samples.
BRIDGE intervenes on 96% and 95% of unreliable inputs in VLind and Columbia, while avoiding intervention on 78% and 75% of reliable inputs, respectively. IMD2020 has a higher negative injection rate (77%) because its anomalies are more subtle in both semantic inconsistencies and splicing artifacts (Appendix B, Table 6), requiring more conservative verification.
2. Inference latency. We additionally measured inference time per sample:
| Dataset | None (s) | Low (s) | Medium (s) | High (s) |
|---|---|---|---|---|
| VLind | 61.8 | 92.6 | 91.1 | 92.5 |
| IMD2020 | 31.2 | 58.4 | 49.4 | 55.4 |
| Columbia | 34.2 | 48.8 | 52.0 | 45.7 |
| WHOOPS | 40.7 | 66.6 | 69.9 | 66.4 |
The maximum observed latency increase is 1.87× (IMD2020, Low). VLind was evaluated on an RTX 4080, while other datasets used an RTX 4090. Latency does not consistently increase with intervention strength because routing and attention-sink operations are shared across levels, while generation time varies.
Importantly, as illustrated in Figure 1 and Section 4.3, unreliable outputs can cause more costly downstream errors, including incorrect recommendations and contaminated records. Thus, although BRIDGE incurs additional computation, its selective verification is intended to balance inference cost against the risk of unreliable information propagation.
As discussed above, BRIDGE introduces additional inference overhead to prevent potentially more costly downstream errors. Nevertheless, we also considered a simpler, lower-cost alternative: external routing, as discussed in Section 4.7.
BRIDGE's overhead mainly comes from hidden-state detection, attention-sink computation, and additional generation. External routing reduces this cost by using an independent image classifier to select verification prompts before VLM inference. We compare their performance in Figure 5:
| Method | IMD2020 F1 | VLind F1 | Columbia F1 | WHOOPS Recall |
|---|---|---|---|---|
| CNN Router | 0.684 | 0.831 | 0.835 | 0.670 |
| ResNet Router | 0.700 | 0.836 | 0.723 | 0.800 |
| ViT Router | 0.713 | 0.807 | 0.776 | 0.750 |
| BRIDGE | 0.758 | 0.821 | 0.878 | 0.630 |
We additionally measured inference latency:
| Method | BRIDGE | CNN Router | ResNet Router | ViT Router |
|---|---|---|---|---|
| Time (s/sample) | 84.5 | 36.3 | 36.7 | 38.8 |
External routing significantly reduces inference time. CNN and ViT achieve 57.0% and 54.1% lower latency than BRIDGE, respectively. ResNet also achieves competitive performance, leading on VLind and WHOOPS. However, its F1 drops to 0.723 on Columbia, compared with 0.878 for BRIDGE. BRIDGE also performs best on IMD2020.
These results demonstrate an efficiency–reliability trade-off. External routing is a practical lower-cost alternative, while BRIDGE offers stronger performance on IMD2020 and Columbia by leveraging the target VLM's internal representations. More importantly, our primary objective is to investigate the internal G–D Gap mechanism, rather than exclusively minimize inference latency.
Response 1
Comment 1:
We thank the reviewer for raising this question. We would like to emphasize that the primary goal of this work is to investigate the internal mechanism of the G–D Gap and develop targeted corrections using the VLM's own representations. As described in Section 3.3, BRIDGE extracts reliability-related signals from hidden states H through a lightweight Projector, producing a risk score s=fθ(H). This score determines the strength of verification intervention during generation. White-box access is therefore central to our mechanism analysis, allowing us to directly study and utilize the target VLM's internal information.
For black-box API models, although hidden states and internal injection positions are inaccessible, the core idea of BRIDGE can be adapted through external detection and selective re-prompting. Specifically, we can train an independent visual reliability detector on reliable and unreliable images to estimate the risk score s=fθ∗(I). Based on this score, the system can selectively add discrimination instructions and request the black-box VLM to verify or regenerate its response.
This approach preserves BRIDGE's risk-guided selective correction without requiring internal model access. However, external visual features may be less aligned with the target VLM's internal reliability signals. Therefore, our focus remains on understanding the G–D Gap under white-box access, while external detection combined with re-prompting offers a feasible extension to black-box APIs that warrants further evaluation.