acceptodds
Under review as a conference paper at ICLR 2027

OmniConfess: Eliciting Token-Level Confessions to Mitigate Omni-Modal Hallucination

Abstract

Omni-modal large language models (OmniLLMs) unify text, images, audio, and video, yet hallucinate when generation relies on the wrong evidence. Existing inference-time methods can reduce hallucinations, but rarely reveal which evidence sustains a generated commitment. We introduce **OmniConfess**, a training-free method for mitigating omni-modal hallucinations. It fixes a candidate response and re-scores it at token resolution under controlled channel-wise evidence interventions, producing a structured token-by-channel confession that reveals the response's evidential dependence. OmniConfess uses this confession to preserve grounded content and correct commitments driven by irrelevant or contradictory evidence. To evaluate OmniConfess, we construct **OmniHalluBench**, a 3,540-example benchmark built from six datasets spanning text, image, audio, and video settings and both judgment and free-form generation. Experiments show that OmniConfess mitigates hallucinations across heterogeneous modality and task settings. Our code and benchmark are publicly available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Omni-Modal, token-level, confess, hallucination mitigation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_bZg4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OmniConfess, a training-free method for reducing hallucinations in omni-modal models. It first generates and freezes a candidate response, then removes evidence channels one at a time and rescores the same tokens under fixed prefixes. These changes are used to estimate which channels support each prediction and to guide either a revised judgment or local rewriting of suspect spans. The authors evaluate the method on OmniHalluBench, a 3,540-example collection drawn from six benchmarks, using three backbones. They report improvements over several inference-time baselines and analyse the method's components, generation quality, and computational cost.

Critical strengths

S1. Fixing the candidate and its prefixes provides a clear way to separate the effect of changing evidence from the effect of generating a different continuation. The token-by-channel measurements also give the method a concrete basis for deciding which parts of an answer to revise. The same framework handles both judgment and free-form tasks without additional training, which makes this approach practically useful.

S2. The evaluation covers text, image-text, and audio-visual tasks across three backbones, with comparisons against several families of inference-time methods. The ablations examine anchoring, channel-wise scoring, and correction separately, and the main results show substantial improvements in several settings. The paper also reports inference costs. Overall, the breadth of this approach is impressive.

Critical weaknesses

W1. Important evaluation and implementation details are missing.

The paper names GPT-4o and human experts for the multidimensional quality assessment in Section 5.4, but it does not identify the LLM judge used for the GAV scores in Table 1.

The decoding settings, revision threshold τ\tau, thresholds for assigning diagnostic states, and baseline hyperparameters are also not specified in enough detail. Appendix E largely describes the methods rather than giving their actual configurations, such as the number of Self-Consistency samples or the VCD contrast coefficient. These choices can affect both quality and cost. Without them, it is difficult to reproduce the results or determine whether the baselines were appropriately adapted to the tasks and backbones.

W2. The reported Nemotron improvement does not agree with Table 1.

Figure 6(b) and Section 5.5 report an average F1 improvement of +7.50 for OmniConfess on Nemotron. However, Table 1 gives 61.57 for OmniConfess and 53.06 for Base, a difference of 8.51. Other entries agree with the table. Thus, the authors should clarify the value.

W3. The headline F1 improvement is not tied to a clear comparison.

The conclusion states that F1 improves by up to 13.3 points, but the source of this figure is unclear. I could not identify the dataset, backbone, and comparator that produce this value.

Questions for authors

Q1. Are the prompting conditions identical across methods? Section 4.1 introduces task-specific evidence constraints prior to the OmniConfess pipeline. Do Base and all baselines share identical generation prompts? Additionally, Appendix A.1 omits generation prompts for PhD and RAGTruth, and Section 4.1 references correction prompts for Revθ\mathrm{Rev}_\theta that are missing. Please provide these missing prompts and clarify how prompt consistency was enforced across models.

Q2. How are evidence interventions and correction rules implemented? Please specify key hyperparameters and definitions: the revision threshold τ\tau, the relevance/contribution thresholds for assigning ℓi(m)\ell_i^{(m)}, and task-specific definitions of rm(Q,T)r_m(Q,T). How is X−mX^{-m} constructed (token deletion, empty input, feature masking, or noise replacement)? For text-only benchmarks (PubMedQA, RAGTruth), how are evidence channels defined and isolated from questions and instructions? Providing baseline decoding settings and full hyperparameter specifications is critical to interpreting the measured probability shifts.

Q3. Why do certain baselines perform significantly worse than Base? On Qwen2.5-Omni, PhD F1 drops from 71.68 (Base) to 21.16 (GD) and 32.12 (PD). On RAGTruth, ToT achieves only 12.08 Token-F1 versus 38.25 (Base). Could you analyze representative failures to determine if these drops stem from intrinsic method flaws, formatting mismatches, or prompt adaptations? Furthermore, since MAD is evaluated only on CMM and AVHBench, its average is computed over a non-identical benchmark set; please include a matched-subset comparison or explicitly flag this discrepancy.

Q4. How should the efficiency metrics in Table 2 be interpreted? OmniConfess reports 0.82s latency on HaloQuest versus 0.93s for Base, despite adding channel-wise rescoring after candidate generation. On AVHBench, it reports 164.1 output tokens versus 25 for Base. Please clarify these timing measurements (e.g., per-example averages, caching, batching, hardware handling) and specify what the reported token counts include (candidate generation, revisions, or final response only). Computational costs for teacher-forced rescoring should be distinguished and reported separately to properly contextualize the efficiency claims.

Q5. Correct the appendix examples and explanations. In Figure 12 (PhD cat example), the Ground Truth field incorrectly contains the motorbike text from Figure 11. In Appendix G, the text claims Figure 11 contrasts "person" vs. "man," but neither word appears in the displayed reference, and the OmniConfess answer already starts with "A man adjusts..." Please update these figures with the intended predictions and ground-truth references to ensure the qualitative analysis can be verified.

Q6. Positioning this work with preference-trained hallucination mitigation. A discussion comparing OmniConfess to preference learning approaches such as HA-DPO, SENTINEL (ICCV 2025), and TPR (NeurIPS 2025) would help contextualize the contribution. For instance, SENTINEL highlights early object hallucination propagation, complementing OmniConfess's finding that evidence dependence is front-loaded; similarly, TPR's topic-level rewriting relates closely to local span repair. Please elaborate on what OmniConfess adds to diagnosing and correcting these phenomena.

I would be happy to revise my assessment if the authors can clarify these points and provide further details.

10 comments
AuthorsOpenReview ↗

Response to Reviewer bZg4 W1 & W2 & W3 & Q1

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your feedback. Below, we respectfully provide our detailed responses to address your concerns.


W1. Important evaluation and implementation details are missing.

The paper names GPT-4o and human experts for the multidimensional quality assessment in Section 5.4, but it does not identify the LLM judge used for the GAV scores in Table 1.

The decoding settings, revision threshold τ\tau, thresholds for assigning diagnostic states, and baseline hyperparameters are also not specified in enough detail. Appendix E largely describes the methods rather than giving their actual configurations, such as the number of Self-Consistency samples or the VCD contrast coefficient. These choices can affect both quality and cost. Without them, it is difficult to reproduce the results or determine whether the baselines were appropriately adapted to the tasks and backbones.

Response. We thank the reviewer for raising these points about the reproducibility of our evaluation and implementation. In Evaluation Metrics under Experimental Setup, we now specify that GPT-4o assigns the GAV scores in Table 1 on a 0–10 scale using the task-specific evaluation prompt.

We revised the text following Eq. (9) to specify the three diagnostic states: aligned (0), under-reliant (−), and over-reliant (+). The new “OmniConfess Implementation Details” appendix documents the rules and thresholds for assigning these states, the decoding setup (greedy decoding with at most 512 new tokens), and the correction thresholds and how we selected them. It also reports a threshold sensitivity analysis on Qwen2.5-Omni-7B.

In Appendix E, “Backbone and Baseline Details,” we report the baseline configurations used in our experiments. For example, Self-Consistency samples four reasoning paths at temperature 0.6, and VCD uses a contrast coefficient of 0.1 and a noise standard deviation of 0.1.


W2. The reported Nemotron improvement does not agree with Table 1.

Figure 6(b) and Section 5.5 report an average F1 improvement of +7.50 for OmniConfess on Nemotron. However, Table 1 gives 61.57 for OmniConfess and 53.06 for Base, a difference of 8.51. Other entries agree with the table. Thus, the authors should clarify the value.

Response. Thank you for the suggestion. The correct average F1 improvement on Nemotron is 8.51 points (61.57 − 53.06 in Table 1). We corrected the value from 7.50 to 8.51 in Figure 6(b) and Section 5.5.


W3. The headline F1 improvement is not tied to a clear comparison.

The conclusion states that F1 improves by up to 13.3 points, but the source of this figure is unclear. I could not identify the dataset, backbone, and comparator that produce this value.

Response. Thank you for the suggestion. We checked the result against Table 1 and revised Discussion and Conclusion to specify that OmniConfess improves Token-F1 by 17.3 points over the base model on RAGTruth with Qwen2.5-Omni-7B.


Q1. Are the prompting conditions identical across methods?

Section 4.1 introduces task-specific evidence constraints prior to the OmniConfess pipeline. Do Base and all baselines share identical generation prompts? Additionally, Appendix A.1 omits generation prompts for PhD and RAGTruth, and Section 4.1 references correction prompts for Revθ\mathrm{Rev}_{\theta} that are missing. Please provide these missing prompts and clarify how prompt consistency was enforced across models.

Response. Thank you for pointing out the ambiguity in our description of the prompts. Base and all baselines share the same task-level initial generation templates. OmniConfess uses its own templates, each containing instructions for both generation and correction. We revised the “Prompting Setup” subsection in Section 4.1 to clarify this and added the generation prompts for PhD and RAGTruth to Appendix A.1.

AuthorsOpenReview ↗

Response to Reviewer bZg4 Q2

Q2. How are evidence interventions and correction rules implemented?

Please specify key hyperparameters and definitions: the revision threshold τ\tau, the relevance/contribution thresholds for assigning ℓi(m)\ell_i^{(m)}, and task-specific definitions of rm(Q,T)r_m(Q,T). How is X−mX^{-m} constructed (token deletion, empty input, feature masking, or noise replacement)? For text-only benchmarks (PubMedQA, RAGTruth), how are evidence channels defined and isolated from questions and instructions? Providing baseline decoding settings and full hyperparameter specifications is critical to interpreting the measured probability shifts.

Response. We thank the reviewer for raising these points about the reproducibility of our evidence interventions, correction rules, and baseline decoding settings.

Channel relevance and evidence interventions. The rule rm(Q,T)r_m(Q,T) indicates whether evidence from channel mm should be used to answer question QQ in task TT. To clarify how this rule is applied, we added the table below to the Evidence Interrogation paragraph of the “OmniConfess Implementation Details” appendix. For each dataset, it shows how channel relevance is determined and which inputs are modified or retained during the intervention.

DatasetHow channel relevance is determinedHow the input evidence is modified
PhDThe task designates the image as relevant and the misleading textual context as irrelevant.We remove the misleading context before tokenization, retaining the image, question, and task instructions.
CMMA keyword rule assigns audio or video as the relevant channel from the question text.We remove audio and video in separate passes. Each pass omits the corresponding media tensor and placeholder tokens, retaining the other modality, question, and task instructions.
PubMedQAThe PubMed abstract is the relevant evidence channel.We delete the entire Passage: line containing the abstract before tokenization, retaining the question, task instructions, and output-format constraints.
RAGTruthThe retrieved passage is the relevant evidence channel.We pass an empty passage field to the prompt constructor, retaining the question, task instructions, and output-format constraints. Both views score the same candidate tokens under the same answer prefixes.
HaloQuestThe image is the relevant evidence channel.We omit the image tensor and its placeholder tokens, then rescore the fixed candidate with the same question and task instructions.
AVHBenchBoth audio and video are task-relevant. Audio and visual lexicons associate content words in the candidate captions with their corresponding channels.We vary video frame sampling and resolution while keeping audio and text fixed, then compare agreement on content words across the resulting captions.

Diagnostic states and correction thresholds. We revised the text following Eq. (9) to define three diagnostic states and added their assignment rules to the “OmniConfess Implementation Details” appendix. For channel-removal interventions, we subtract the ablated score from the full-input score. For judgment tasks, the score is the candidate’s log-probability margin over its fixed alternative. For token-level diagnosis on RAGTruth and HaloQuest, it is the selected token’s log probability. A relevant channel is marked under-reliant when the signed change in judgment margin is below 1.0. For token-level diagnosis on RAGTruth and HaloQuest, the corresponding threshold on the signed change in log probability is 0.5. An irrelevant channel is marked over-reliant when the absolute difference exceeds 2.0. The remaining cases are marked aligned.

These discrete states support post-hoc interpretation. Correction uses continuous evidence scores with separate thresholds. The main configuration uses τ=−1.6\tau=-1.6 to determine when to revise a judgment in Eq. (14). For free-form generation, ϵ=0.1\epsilon=0.1 identifies weak evidence support, and τedit=−0.5\tau_{\mathrm{edit}}=-0.5 governs whether to initiate local revision. The same appendix reports the threshold selection protocol and sensitivity analysis.

Baseline decoding settings. Appendix E, “Backbone and Baseline Details,” reports the decoding configurations and key hyperparameters used for the baselines in our experiments.

AuthorsOpenReview ↗

Response to Reviewer bZg4 Q3 & Q4 & Q5 & Q6

Q3. Why do certain baselines perform significantly worse than Base?

On Qwen2.5-Omni, PhD F1 drops from 71.68 (Base) to 21.16 (GD) and 32.12 (PD). On RAGTruth, ToT achieves only 12.08 Token-F1 versus 38.25 (Base). Could you analyze representative failures to determine if these drops stem from intrinsic method flaws, formatting mismatches, or prompt adaptations? Furthermore, since MAD is evaluated only on CMM and AVHBench, its average is computed over a non-identical benchmark set; please include a matched-subset comparison or explicitly flag this discrepancy.

Response. We thank the reviewer for suggesting a closer analysis of why these baselines underperform Base. To investigate what may account for the performance gaps on PhD and RAGTruth, we examined representative outputs from GD, PD, and ToT and added a “Baseline Failure Analysis” subsection to the “Results by Task Family” appendix. We also revised the table caption to clarify that MAD was evaluated only on CMM and AVHBench, so its average is computed over those two subsets.


Q4. How should the efficiency metrics in Table 2 be interpreted?

OmniConfess reports 0.82s latency on HaloQuest versus 0.93s for Base, despite adding channel-wise rescoring after candidate generation. On AVHBench, it reports 164.1 output tokens versus 25 for Base. Please clarify these timing measurements (e.g., per-example averages, caching, batching, hardware handling) and specify what the reported token counts include (candidate generation, revisions, or final response only). Computational costs for teacher-forced rescoring should be distinguished and reported separately to properly contextualize the efficiency claims.

Response. Thank you for the suggestion. We reviewed and revised Table 2 and added an analysis of the efficiency results in the main text. We clarified the measurement protocol in the Inference Efficiency (RQ5) subsection. We measure end-to-end wall-clock latency per example at a batch size of one, synchronize CUDA at both timing boundaries, and average the measurements after excluding warm-up runs. Token counts include all tokens generated during inference, including those from discarded candidates and revisions. Teacher-forced rescoring generates no new tokens. Its runtime is included in end-to-end latency and was not measured separately.


Q5. Correct the appendix examples and explanations.

In Figure 12 (PhD cat example), the Ground Truth field incorrectly contains the motorbike text from Figure 11. In Appendix G, the text claims Figure 11 contrasts "person" vs. "man," but neither word appears in the displayed reference, and the OmniConfess answer already starts with "A man adjusts..." Please update these figures with the intended predictions and ground-truth references to ensure the qualitative analysis can be verified.

Response. We thank the reviewer for pointing out the inconsistencies in the appendix examples. We corrected the fields in Figures 11 and 12, including the Ground Truth field in the PhD cat example, and revised the “AVHBench Case Study” discussion in Appendix G to match the content shown in Figure 11.


Q6. Positioning this work with preference-trained hallucination mitigation.

A discussion comparing OmniConfess to preference learning approaches such as HA-DPO, SENTINEL (ICCV 2025), and TPR (NeurIPS 2025) would help contextualize the contribution. For instance, SENTINEL highlights early object hallucination propagation, complementing OmniConfess's finding that evidence dependence is front-loaded; similarly, TPR's topic-level rewriting relates closely to local span repair. Please elaborate on what OmniConfess adds to diagnosing and correcting these phenomena.

Response. We thank the reviewer for highlighting the connections between early hallucination propagation, targeted rewriting, and our work. We expanded the “Omni-Modal Hallucination and Preference Training” paragraph in Related Work to discuss HA-DPO, SENTINEL, and TPR.

SENTINEL targets sentences where object hallucinations first appear. Our analysis tracks dependence on input evidence during generation and shows that it is stronger at earlier positions. TPR rewrites selected semantic topics to construct preference pairs for training. OmniConfess uses channel-specific evidence dependence to guide local correction at inference time. It rescores the same fixed candidate after removing each evidence channel in turn and checks the resulting token-level score changes against the task’s evidence requirements to locate suspect spans. It then regenerates those spans while preserving the rest of the response, without additional model training.

Reviewer_72LAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a training-free method for mitigating hallucinations in omni-modal LLMs. It measures token-level likelihood changes when individual evidence channels are removed to guide answer recalibration or selective span regeneration. Experiments on a self-constructed benchmark show improvements in hallucination-related metrics.

Critical strengths
  1. Fixing the candidate response during evidence interventions represents an interesting design.

  2. The evaluation covers several modalities and model architectures and shows improvement over several baselines.

Critical weaknesses
  1. The proposed channel-wise attribution does not adequately account for interactions among evidence modalities. The contribution of each channel is analyzed independently and then combined through an additive surrogate, which ignores the simultaneous effects of multiple channels, which provide complementary or contradictory evidence among each other. A justification on this is better.

  2. The self-constructed benchmark, which is the unique benchmark for evaluation and is claimed to be a major contribution, needs more details about its characteristics, such as the modality distribution. Are PubMedQA and RAGTruth used in the paper single-modality datasets? The sample distribution across different sub-datasets is uneven, but the average (whether macro or micro) of the metrics is not clear. The evaluation baselines omit several recent, closely related studies discussed in the paper without clear justification, e.g., [1][2].

[1] Zhu et al., Token-guard: Towards token-level hallucination control via self-checking decoding. 2026. [2] Ouyang et al. Taming the phantom: Token-asymmetric filtering for hallucination mitigation in large vision-language models

Questions for authors

See weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 72LA W1

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your feedback. Below, we respectfully provide our detailed responses to address your concerns.


W1

The proposed channel-wise attribution does not adequately account for interactions among evidence modalities. The contribution of each channel is analyzed independently and then combined through an additive surrogate, which ignores the simultaneous effects of multiple channels, which provide complementary or contradictory evidence among each other. A justification on this is better.

Response. We thank the reviewer for raising this important point about the limits of additive attribution when evidence from different modalities is complementary or conflicting. We designed OmniConfess’s attribution to guide channel-wise correction. We measure how removing audio or video changes the judgment margin while retaining the other modality, yielding the two channel-specific scores used for correction.

In a 2 × 2 factorial analysis on 721 CMM examples, we varied the presence of audio and video while holding the text, candidate answers, and scoring prefix fixed. The sum of the two single-modality removal effects and the measured joint removal effect agree on whether the margin rises or falls relative to the full-input margin in 80.7% of examples, with a mean absolute error of 1.447 margin units. The reconstructed and measured text-only margins fall on opposite sides of the Yes/No decision threshold in 162 examples (22.5%).

The two single-modality rescoring passes provide the scores needed for channel-wise correction, and the sum of their effects captures the direction of the measured joint effect in most examples. Explicitly measuring the interaction while retaining both channel-specific scores requires an extra text-only pass, increasing counterfactual rescoring time by 24.3%, from 1.0719 to 1.3324 seconds per example. This trade-off supports the two-pass design for channel-wise correction on CMM. We report the full analysis and rescoring costs in Table 8 of the ‘Audio–Visual Interaction Analysis’ appendix.

Table 8. Additive approximation error and counterfactual rescoring cost on 721 CMM examples.

Additive Approximation

MetricValue95% CI
Mean interaction residual+0.223[0.089, 0.362]
Mean absolute error1.447[1.366, 1.536]
Directional agreement80.7%[77.8%, 83.2%]
Decision disagreement22.5%[19.6%, 25.4%]

Counterfactual Rescoring Cost

Rescoring configurations/exampleTotal (s)
Two single-channel passes1.0719772.8
Single-channel passes + joint pass1.3324960.7
Additional cost (+24.3%)+0.2605+187.8

AuthorsOpenReview ↗

Response to Reviewer 72LA W2 & Details Of Ethics Concerns

W2

The self-constructed benchmark, which is the unique benchmark for evaluation and is claimed to be a major contribution, needs more details about its characteristics, such as the modality distribution. Are PubMedQA and RAGTruth used in the paper single-modality datasets? The sample distribution across different sub-datasets is uneven, but the average (whether macro or micro) of the metrics is not clear. The evaluation baselines omit several recent, closely related studies discussed in the paper without clear justification, e.g., [1][2].

[1] Zhu et al., Token-guard: Towards token-level hallucination control via self-checking decoding. 2026.
[2] Ouyang et al. Taming the phantom: Token-asymmetric filtering for hallucination mitigation in large vision-language models

Response. We sincerely thank the reviewer for these specific suggestions. Your feedback helped us clarify the benchmark composition and metric aggregation, motivated a direct comparison with Token-Guard [1], and helped us explain how our PhD ablation relates to the question studied by TAF [2].

Benchmark composition. OmniGroundBench comprises six subsets drawn from public benchmarks. In our evaluation, PubMedQA and RAGTruth use text-only inputs, PhD and HaloQuest use images and text, and CMM and AVHBench use audio, video, and text. In the “OmniGroundBench Details” appendix, the “Benchmark Composition” subsection describes the six subsets and their tasks, reports each subset’s sample count and the Yes/No counts for binary judgment subsets, and explains the PubMedQA label conversion. The “Benchmark Construction” subsection documents the candidate sources and explains how we used difficulty-based selection and manual verification to construct the fixed evaluation set.

Metric aggregation. In “Main Results,” we clarify that Table 1’s Avg. F1 is a subset-level macro-average. Each subset with a reported F1 score receives equal weight regardless of its sample count. The “Results by Task Family” appendix separately reports mean Yes-F1 for binary judgment and mean Token-F1 for free-form generation.

Baseline comparison and related ablation.

  1. Token-Guard [1]. Following the reviewer’s suggestion, we adapted Token-Guard’s decoding procedure to Qwen2.5-Omni-7B to support audio and video inputs. In the new “Comparison with Token-Guard” appendix, we evaluate the adapted Token-Guard and OmniConfess on the same 3,540 examples across all six OmniGroundBench subsets using the same scoring protocol. OmniConfess achieves higher F1 on all six subsets, with an Avg. F1 of 65.10 versus 57.85 for Token-Guard, a difference of 7.25 points. OmniConfess also generates fewer tokens on all six subsets and has lower end-to-end latency on five. The appendix reports and discusses the results by subset, including the slightly higher latency of OmniConfess on CMM.

  2. TAF [2]. We could not locate a publicly available official implementation of TAF at the time of evaluation. TAF studies how influential text tokens can interfere with visual grounding. Our PhD ablation examines this question at the input-channel level. As described in the “Evidence Interrogation” paragraph of the implementation appendix, we rescore the same candidate response with and without the accompanying text. The resulting score difference tells OmniConfess how the text affects the model’s preference for the candidate on each example. OmniConfess uses this signal together with the task’s requirement to answer from the image to guide correction. On PhD, the full OmniConfess method improves Yes-F1 from 71.68 for Base to 89.60, a gain of 17.92 points.


Details Of Ethics Concerns:

While the paper attaches an anonymous git link to its code and dataset for reproducibility, this git project has no content in it.

Response. Thank you for pointing this out. We checked and updated the anonymous repository link.

AuthorsOpenReview ↗

Response to Reviewer 72LA Summary of Revisions and Clarifications

Weakness or QuestionLocation in the revised manuscriptRevision
W1Audio–Visual Interaction Analysis, Table 7Added a 2 × 2 factorial analysis on CMM to quantify the discrepancy between additive and measured joint removal effects. Reported the additional rescoring cost of measuring the joint effect while retaining both channel-specific scores.
W2OmniGroundBench Details, Benchmark Composition (Table 3) and Benchmark ConstructionClarified the modalities used by the six subsets, including the text-only settings for PubMedQA and RAGTruth. Reported subset sizes and Yes/No counts, and documented PubMedQA label conversion, candidate sources, difficulty-based selection, manual verification, and the fixed evaluation protocol.
W2Main Results, opening paragraph, and Results by Task Family, Table 4Defined Avg. F1 as an equal-weight macro-average over subsets with reported scores. Reported mean Yes-F1 for binary judgment and mean Token-F1 for free-form generation separately.
W2Comparison with Token-Guard appendixAdapted Token-Guard to Qwen2.5-Omni-7B to support audio and video inputs. Compared F1, generated token counts, and end-to-end latency on all 3,540 examples across the six OmniGroundBench subsets.
W2OmniConfess Implementation Details, Evidence InterrogationEvaluated the idea behind TAF that text can interfere with visual grounding through an input-channel ablation on PhD.
Details Of Ethics ConcernsAnonymous repository linkChecked and updated the anonymous repository link.

At last, we sincerely appreciate your valuable feedback, and we will carefully consider all your suggestions to further improve our paper. We would be deeply grateful if you could kindly reconsider raising the score to 3 or above. Thank you very much!

Reviewer_Q1hbOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OmniConfess to mitigate hallucinations caused by reliance on inappropriate evidence in omni-modal large language models that process text, images, audio, and video. The method first generates and fixes a candidate response, then measures how the probabilities of the generated tokens and their alternatives change when individual evidence channels are removed. These measurements are combined with question- and task-specific modality-relevance rules to analyze the response’s evidential dependence and either recalibrate judgment outputs or locally revise selected spans of free-form responses.

The preliminary analysis examines properties of evidential dependence and uses these observations to motivate the method’s design. Experiments across multiple benchmarks and backbone models report overall performance improvements over existing methods.

Critical strengths

1. The preliminary analysis helps motivate the method.

The paper presents observations about evidential dependence before introducing the method and explicitly connects them to its design choices, making the rationale behind the approach easier to follow.

2. The appendix provides useful supporting material.

The theoretical proofs, algorithmic pseudocode, prompting examples, and evaluation details help clarify the proposed framework and its experimental setup.

3.The targeted free-form revision design is a useful aspect of the method.

It uses token- and channel-level evidence-dependence signals to identify potentially problematic spans and revise them while preserving the rest of the candidate response. This connects the evidence-dependence analysis to localized correction rather than regenerating the entire response.

Critical weaknesses

1. Insufficient Specification of Key Method Components

1.1 The source and construction of the channel-relevance rules rm(Q,T)r_m(Q,T) are unclear. Section 4.3 invokes task rules, and Algorithm 1 takes these rules as input, but it is not specified whether they are derived solely from the question and task instructions, benchmark metadata, or additional sample-level annotations. Since these rules explicitly specify modality relevance, please clarify their availability at inference time and whether comparable relevance information is available to the baselines. The applicability of the method when such rules are not predefined also requires discussion.

1,2 Section 4.2 defines the free-form alternative as the “top nonselected token.” Please specify whether the candidate response is generated using greedy decoding or sampling, and confirm whether the alternative is the highest-probability token other than the selected token under the original full-evidence distribution.

1.3 Equation (9) defines four diagnostic states ℓi(m)\ell_i^{(m)}, but the assignment criteria and contribution thresholds are not specified. Although these states are included in Cθ(Q,T)C_\theta(Q,T), which is passed to the revision operator in Eq. (15), their operational role remains unclear. Please explain how the states are computed and how they affect correction beyond the continuous dependence scores.

1.4 The revision threshold τ\tau in Eq. (14) is introduced as an input to the algorithm, but its numerical value and selection procedure are not reported. Is the same threshold used across datasets and backbones, or is it tuned separately? Please specify the selection protocol, including whether a held-out validation set is used, and report the method’s sensitivity to this threshold.

2. Potential Confounding from Task-Specific Prompting

Appendix A.1 contains task-specific anti-hallucination instructions that may themselves contribute to performance. Please clarify whether the Base model and all baselines use the same task instructions and output constraints, and whether the “w/o Correction” condition retains these prompts. A prompt-controlled comparison, keeping the multimodal evidence and required answer formats fixed, would help distinguish gains from prompting from those attributable to the proposed method.

3. Insufficient Rationale for the Specific Correction Rules

The propositions establish several algebraic properties, but the rationale for the specific correction rules is not sufficiently developed. For example, why should the residual preference be removed entirely in judgment correction, and why are the three free-form risk components combined as an unweighted sum? A clearer explanation, a worked example, and component-level ablations would help distinguish the properties that follow from the definitions from the reasons for choosing these particular rules.

4. Limited Coverage of Free-Form Module Ablations

Figure 5(a) reports the free-form module ablations only on HaloQuest. Please explain this choice and clarify whether the component-level findings also hold on AVHBench and RAGTruth, which cover different modality settings.

Questions for authors

1. Clarification of Methodological Details

Could the authors provide more detailed definitions and implementation procedures for the functions and symbols discussed above? I understand that page limits may have constrained the presentation, but these details are important for understanding and reproducing the proposed method.

2. Potential Information Leakage from Channel-Relevance Rules

I am concerned that rm(Q,T)r_m(Q,T) may provide additional information about which modalities are relevant to a given question. Could the authors clarify how these rules are obtained and whether they rely solely on information available in the original question and task instructions?

If the rules instead rely on additional relevance annotations that are not available from the original inputs, they could provide privileged information that guides the final prediction. I would appreciate the authors’ perspective on whether this introduces a risk of information leakage, as well as clarification on whether the baselines have access to equivalent modality-relevance information.

8 comments
AuthorsOpenReview ↗

Response to Reviewer Q1hb W1 1.1-1.3

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your feedback. Below, we respectfully provide our detailed responses to address your concerns.


W1. Insufficient Specification of Key Method Components

1.1 The source and construction of the channel-relevance rules are unclear. Section 4.3 invokes task rules, and Algorithm 1 takes these rules as input, but it is not specified whether they are derived solely from the question and task instructions, benchmark metadata, or additional sample-level annotations. Since these rules explicitly specify modality relevance, please clarify their availability at inference time and whether comparable relevance information is available to the baselines. The applicability of the method when such rules are not predefined also requires discussion.

Response. We thank the reviewer for raising this point about the source of channel-relevance rules and their availability at inference time. On CMM, OmniConfess uses a manually written keyword rule to assign audio or video as the relevant channel for each example internally, based on the original question text, and uses this assignment to guide correction. PhD and HaloQuest designate the image as evidence, PubMedQA uses the abstract, and RAGTruth uses the retrieved passages. AVHBench treats both audio and video as task-relevant and uses audio and visual lexicons to associate content words in the candidate caption with their corresponding channels for channel-specific scoring.

We revised the “Relevance Mapping Sensitivity” appendix to compare OmniConfess’s automatic channel assignment with annotation-based assignment under the same correction gate. The generation prompt specifies which evidence to consider at the task level, and OmniConfess derives each example’s channel assignment internally. Both settings achieve 82.15 Yes-F1 on CMM. Swapping the audio and video assignments lowers Yes-F1 to 71.79, a drop of 10.36 points. For a new task without a predefined rule, the current correction procedure requires a way to assign relevant channels from information available at inference time.

1,2 Section 4.2 defines the free-form alternative as the “top nonselected token.” Please specify whether the candidate response is generated using greedy decoding or sampling, and confirm whether the alternative is the highest-probability token other than the selected token under the original full-evidence distribution.

Response. We thank the reviewer for raising this point about candidate generation and alternative-token selection. As specified in the “OmniConfess Implementation Details” appendix, we generate the initial candidate using greedy decoding with at most 512 new tokens. At each position in free-form generation, the alternative is the highest-probability token other than the selected token under the original full-evidence distribution. We hold the candidate and its alternative tokens fixed during channel-wise rescoring.

1.3 Equation (9) defines four diagnostic states , but the assignment criteria and contribution thresholds are not specified. Although these states are included in , which is passed to the revision operator in Eq. (15), their operational role remains unclear. Please explain how the states are computed and how they affect correction beyond the continuous dependence scores.

Response. We thank the reviewer for raising this point about the definition and operational role of the diagnostic states. The discrete states summarize evidence dependence for post-hoc interpretation. Correction uses continuous evidence scores and separate correction thresholds to decide whether to intervene. We revised the text following Eq. (9) to specify three states: aligned (0), under-reliant (−), and over-reliant (+).

The “OmniConfess Implementation Details” appendix specifies the assignment rule. For a relevant channel, judgment tasks assign the under-reliant state when the margin change is below 1.0. For token-level diagnosis on RAGTruth and HaloQuest, the corresponding threshold is 0.5 on the selected token’s log-probability change. Under this diagnostic rule, an irrelevant channel is marked over-reliant when the absolute value of its score exceeds 2.0. The remaining cases are marked aligned.

AuthorsOpenReview ↗

Response to Reviewer Q1hb W1 1.4 & W2 & W3

W1. Insufficient Specification of Key Method Components

1.4 The revision threshold in Eq. (14) is introduced as an input to the algorithm, but its numerical value and selection procedure are not reported. Is the same threshold used across datasets and backbones, or is it tuned separately? Please specify the selection protocol, including whether a held-out validation set is used, and report the method’s sensitivity to this threshold.

Response. We thank the reviewer for raising these questions about threshold selection and sensitivity. We added the settings and selection protocol to the “OmniConfess Implementation Details” appendix. We use the same thresholds across all three backbones and all datasets within each task family: τ=−1.6\tau=-1.6 for judgment tasks, and ϵ=0.1\epsilon=0.1 and τedit=−0.5\tau_{\mathrm{edit}}=-0.5 for free-form generation.

We selected the thresholds using stratified subsets comprising approximately 20% of the corresponding datasets, preserving Yes/No label proportions for judgment tasks. The main results are evaluated on the full datasets, including the tuning subsets.

The appendix also reports threshold sensitivity on Qwen2.5-Omni-7B. Mean Yes-F1 peaks at 89.67 for τ=−1.6\tau=-1.6 and drops by 1.21 and 1.49 points at the two adjacent settings. For free-form generation, at ϵ=0.1\epsilon=0.1, increasing τedit\tau_{\mathrm{edit}} from −0.5-0.5 to −0.3-0.3 increases mean Token-F1 by 0.16 points and decreases mean logical coherence by 0.19 points on a 0–10 scale.

W2. Potential Confounding from Task-Specific Prompting

Appendix A.1 contains task-specific anti-hallucination instructions that may themselves contribute to performance. Please clarify whether the Base model and all baselines use the same task instructions and output constraints, and whether the “w/o Correction” condition retains these prompts. A prompt-controlled comparison, keeping the multimodal evidence and required answer formats fixed, would help distinguish gains from prompting from those attributable to the proposed method.

Response. We thank the reviewer for raising this concern about the effect of task-specific prompting on the evaluation results. Base and all baselines share the same task-level initial generation templates. OmniConfess’s generation templates include task-level instructions on how to use the evidence. To assess their contribution, we added “w/o Evidence-Use Instructions” to Figure 5(a). This setting removes those instructions from the generation prompt while retaining automatic channel assignment and the correction pipeline. The “w/o Correction” condition keeps the full OmniConfess initial generation prompt fixed, allowing us to measure the contribution of correction under the same prompt.

W3. Insufficient Rationale for the Specific Correction Rules

The propositions establish several algebraic properties, but the rationale for the specific correction rules is not sufficiently developed. For example, why should the residual preference be removed entirely in judgment correction, and why are the three free-form risk components combined as an unweighted sum? A clearer explanation, a worked example, and component-level ablations would help distinguish the properties that follow from the definitions from the reasons for choosing these particular rules.

Response. We thank the reviewer for highlighting the distinction between the algebraic properties of the correction rules and the rationale for choosing them. We added two worked examples in the “Correction Mechanisms” subsection of the “Case Studies” appendix.

Judgment correction. Under the additive surrogate, full subtraction of the estimated residual preference and the task-irrelevant channel contribution leaves the corrected margin determined solely by the estimated contribution of task-relevant evidence. In the CMM example, applying Eqs. (10)–(12) offline changes the Yes–No margin from +0.0625 to −1.8125, which favors the reference answer No.

Free-form risk aggregation. The three terms in Eq. (13) share a log-probability margin scale. We use unit coefficients as a fixed default that requires no additional weight fitting. In the HaloQuest example, we remove each term in turn while holding the content-word filter and span selector fixed. The full score selects “coin”. Removing either of the first two terms preserves this selection, while removing the question-text contribution shifts it to “image”. The appendix reports all four conditions in the single-example offline localization analysis.

AuthorsOpenReview ↗

Response to Reviewer Q1hb W4 & Q1 & Q2

W4. Limited Coverage of Free-Form Module Ablations

Figure 5(a) reports the free-form module ablations only on HaloQuest. Please explain this choice and clarify whether the component-level findings also hold on AVHBench and RAGTruth, which cover different modality settings.

Response. Thank you for the suggestion. We updated the ablation table in Figure 5(a) to include AVHBench and RAGTruth in the free-form module ablation.

Questions For Authors:

Q1. Clarification of Methodological Details

Could the authors provide more detailed definitions and implementation procedures for the functions and symbols discussed above? I understand that page limits may have constrained the presentation, but these details are important for understanding and reproducing the proposed method.

Response. We thank the reviewer for emphasizing the importance of these details for reproducibility. We revised the text following Eq. (9) to define the three diagnostic states and expanded the “OmniConfess Implementation Details” appendix. The appendix now documents candidate generation, channel-wise evidence interventions and rescoring, diagnostic state assignment, and the correction thresholds, including how we selected them and tested their sensitivity. Our responses to W1.1–W1.4 address the specific definitions and implementation questions in detail.

Q2. Potential Information Leakage from Channel-Relevance Rules

I am concerned that may provide additional information about which modalities are relevant to a given question. Could the authors clarify how these rules are obtained and whether they rely solely on information available in the original question and task instructions?

If the rules instead rely on additional relevance annotations that are not available from the original inputs, they could provide privileged information that guides the final prediction. I would appreciate the authors’ perspective on whether this introduces a risk of information leakage, as well as clarification on whether the baselines have access to equivalent modality-relevance information.

Response. We thank the reviewer for raising this concern. The channel-relevance rule did not introduce information leakage in our CMM evaluation. The generation prompt states which evidence to consider at the task level. OmniConfess uses a keyword rule to assign the relevant channel internally from the original question text. On all 721 CMM examples, the rule used only the question text, and the annotation fallback was never triggered. All baselines received the same question text and could access the same modality cues. The dataset annotations were used only as a reference in the separate comparison reported in the “Relevance Mapping Sensitivity” appendix.

Reviewer_EQKhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes OmniConfess. It is a training-free, inference-time method for omni-modal hallucination mitigation. It freezes the model’s first-pass candidate answer and its prefix. This prevents later comparisons from being confounded by re-decoding or re-scoring. It then re-scores each token of the frozen candidate. Each re-scoring removes one evidence channel at a time: text, image, audio, or video. This produces a token-by-channel-by-question “confession” of evidential dependence. It then uses task-specific relevance rules and token-level risk scores. These either re-scale a binary judgment or regenerate only the risky spans of a free-form output. The authors also assemble OmniHalluBench, and evaluate on three omni-modal backbones. Baselines include search-based, evidence-contrast, and fine-grained methods.

Critical strengths
  1. Freezing the candidate before perturbing evidence is the right fix for the trajectory-variation confound the authors identify, and it is a cleaner design than most decoding-time interventions, which cannot separate "the evidence mattered" from "the decode path changed"; exposing a structured per-token, per-channel dependency record rather than only a corrected string is a useful interface if it is properly validated.
  2. The evaluation surface is broad for this genre: three backbones spanning dense, MoE, and hybrid architectures, six datasets, and both judgment and free-form tasks, with a cross-backbone summary that makes the generalization claim concrete rather than rhetorical.
  3. Two controls go beyond what comparable papers report: the anchoring comparison against a shared-prefix alternative and the comparison of correction strategies under an equal correction budget, plus honest cost accounting in Table 2 that includes the ~27×/22× token reduction over search-based methods and the residual latency the authors concede remains substantial.
Critical weaknesses
  1. The confession is evaluated only through downstream CMM F1, Idx, and Recall Balance. Because the confession is computed from the same ablation deltas that drive the correction, a downstream accuracy gain cannot serve as evidence that the confession is faithful.

  2. Removing the correction stage produces by far the largest drop, so most of the benefit comes from moving decision scores rather than from the interrogation that motivates the paper. CMM Recall Balance shows the largest movement of any metric on a benchmark whose base F1 is already high. What is missing is a label-prior-only recalibration baseline and a random-channel ablation at equal rescoring budget. Without them the paper cannot exclude the failure mode documented for contrastive decoding, where apparent hallucination gains come from unidirectional distribution adjustment rather than a hallucination-specific signal.

  3. The signal is a leave-one-modality-out, token-level log-probability contrast on a frozen candidate, thresholded into spans and repaired, and two very close pieces of work are absent from the related-work discussion: SAFE (Zhao et al., 2026), which contrasts grounded and vision-ablated paths into a token-level contrastive grounding score and reports the same observation that dependency decays over generation, and CaRe (Mi et al., 2026), a training-free modality-probe method evaluated on exactly CMM and AVHBench. The shared-prefix anchoring used as a baseline is also doing very similar work to commitment anchoring. The related-work discussion itself is two short paragraphs with no attribution or explainability literature at all. Relatedly, the relevance assignment in the confession construction is a hand-specified per-task rule, so the “training-free and general” claim rests on dataset-specific knowledge, and no automatic variant or sensitivity analysis is offered.

  4. The main text gives no per-dataset counts, sampling criteria, description of how PubMedQA’s three-way labels were collapsed into binary judgment, or class balance.

  5. A benchmark named OmniHalluBench already exists (Yang et al., EMNLP 2026 Findings, 10,000 samples, six cross-modal tasks) and is neither cited nor compared against, so reusing the name will confuse readers.

  6. Mixing binary judgment with token-F1 generation across text-only and audio-visual sources makes the reported average F1 an average of quantities with different chance levels.

  7. The text claims consistent superiority across all datasets and metric, while several supporting margins are under one F1 point, and on Nemotron the method is not best on PHD, CMM, or AVHBench.

Questions for authors

Please see weakness

9 comments
AuthorsOpenReview ↗

Response to Reviewer EQKh W1 & W2

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your feedback. Below, we respectfully provide our detailed responses to address your concerns.


W1

The confession is evaluated only through downstream CMM F1, Idx, and Recall Balance. Because the confession is computed from the same ablation deltas that drive the correction, a downstream accuracy gain cannot serve as evidence that the confession is faithful.

Response. We thank the reviewer for raising this important point regarding the evidence for Confession’s faithfulness. We agree that downstream performance gains alone cannot establish diagnostic faithfulness. We designed Confession to identify which evidence channels influence the model’s preference between fixed answer candidates. Correction uses these diagnoses to guide answer revision, and the original CMM results evaluate the downstream performance of Confession-guided Correction.

To test these diagnoses directly, we added a controlled content replacement experiment on CMM examples. We first obtain Confession’s channel diagnosis from its standard ablations of the original input. We then replace audio and video one channel at a time with content from three same-group donors per example, keeping the question, the other modality, the Yes/No candidates, and the scoring context fixed. We rescore each modified input before applying Correction and construct a reference for channel dependence from the resulting changes in the Yes/No margin.

Channel diagnosisAgreement with replacement reference (%) ↑Judgment flip-rate gap (pp) ↑
Random49.96−0.05
Raw ablation scores (Raw-Δ)56.03−1.48
Group-wise47.77−2.42
OmniConfess72.26+18.49

Agreement measures how often each diagnosis identifies the same channel as the replacement-based reference. The flip-rate gap is the judgment flip rate when replacing the identified channel minus the rate when replacing the other channel.

On CMM, Confession agrees with the replacement-based reference 16.23 percentage points more often than Raw-Δ. Replacing the channel identified by Confession also flips the model’s judgment more often, with a gap of +18.49 pp, compared with −1.48 pp for Raw-Δ. These results support Confession’s ability to identify which channel the model’s judgment is more sensitive to under the tested interventions. We have added the full protocol and results to the Faithfulness Evaluation appendix (Table 9).


W2

Removing the correction stage produces by far the largest drop, so most of the benefit comes from moving decision scores rather than from the interrogation that motivates the paper. CMM Recall Balance shows the largest movement of any metric on a benchmark whose base F1 is already high. What is missing is a label-prior-only recalibration baseline and a random-channel ablation at equal rescoring budget. Without them the paper cannot exclude the failure mode documented for contrastive decoding, where apparent hallucination gains come from unidirectional distribution adjustment rather than a hallucination-specific signal.

Response. We thank the reviewer for raising the concern that a global shift in Yes/No scores could account for the observed correction gains. Removing Correction causes the largest drop because it disables all revisions and returns the initial predictions. The other module ablations still allow revisions. We designed Correction to use each example’s channel diagnosis to guide revision. To assess the contribution of these diagnoses, we followed the reviewer’s suggestion and added two controls.

  • Global label-bias calibration tests how much a uniform shift in the Yes/No score margin can improve performance. We estimate the offset using five-fold cross-validation.
  • Equal-budget random-channel correction uses the same fixed candidates and the same two modality rescoring passes as OmniConfess. It then selects one channel score at random to guide revision.
MethodCMM Yes-F1CMM Recall Balance
Base model70.0748.84
Global label-bias calibration78.8379.16
Equal-budget random-channel correction76.3684.98
OmniConfess82.1586.49

Global calibration improves both metrics on CMM. Because it applies the same offset to every example, it can flip initially correct judgments and cannot tailor corrections to the evidence in each example. OmniConfess exceeds global calibration by 3.32 Yes-F1 points and 7.33 Recall Balance points. At the same rescoring budget, the reported Yes-F1 scores are 82.15 for OmniConfess and 76.36 for random-channel correction. This 5.79-point difference supports the value of example-specific channel selection on CMM. We have added both controls to the Correction Controls appendix (Table 6).

AuthorsOpenReview ↗

Response to Reviewer EQKh W3 & W4

W3

The signal is a leave-one-modality-out, token-level log-probability contrast on a frozen candidate, thresholded into spans and repaired, and two very close pieces of work are absent from the related-work discussion: SAFE (Zhao et al., 2026), which contrasts grounded and vision-ablated paths into a token-level contrastive grounding score and reports the same observation that dependency decays over generation, and CaRe (Mi et al., 2026), a training-free modality-probe method evaluated on exactly CMM and AVHBench. The shared-prefix anchoring used as a baseline is also doing very similar work to commitment anchoring. The related-work discussion itself is two short paragraphs with no attribution or explainability literature at all. Relatedly, the relevance assignment in the confession construction is a hand-specified per-task rule, so the “training-free and general” claim rests on dataset-specific knowledge, and no automatic variant or sensitivity analysis is offered.

Response. Thank you for pointing out the need to clarify how OmniConfess relates to prior work and how task-level relevance rules affect its applicability. We expanded Related Work to discuss SAFE, CaRe, and prior work on attribution and explainability. We revised Anchoring Analysis (RQ2) to clarify what Shared Prefix and Commitment Anchoring each hold fixed. We also added a channel-assignment comparison and sensitivity analysis in Appendix H and a prompt control in Appendix F.2.

Relationship to SAFE and CaRe. SAFE uses the score difference before and after visual ablation to guide online decoding and observes that visual dependence decreases over generation. CaRe uses a text-only probe to select the unimodal branch relevant to the question and adjusts the joint representation when that branch disagrees with the joint prediction. OmniConfess is distinguished by holding the same candidate response fixed, using channel-wise ablations to measure its dependence on each evidence source, and combining these measurements with the task’s evidence requirements to guide correction. SAFE’s public implementation targets other backbones and does not support Qwen2.5-Omni-7B, and we did not find a public implementation of CaRe. We therefore added a comparison of the two mechanisms with ours in Related Work.

Commitment Anchoring and Shared Prefix. Shared Prefix uses the same scoring prefix across evidence conditions. Commitment Anchoring additionally fixes the entire candidate response and rescores each candidate token and its fixed alternative under each channel ablation. Compared with Shared Prefix, Commitment Anchoring improves Avg. F1 from 59.00 to 65.10 and hallucination localization AUPRC from 58.20 to 66.38 (Figure 5(b)). The revised Anchoring Analysis (RQ2) describes both settings.

Task-level relevance rules and sensitivity. On CMM, a keyword rule identifies the relevant channel from the original question (Table 10). With the correction gate held fixed, question-based and annotation-based assignment both achieve 82.15 Yes-F1 on 721 examples. Swapping the audio and video assignments reduces Yes-F1 to 71.79 (Table 7). Removing task-level evidence-use instructions while retaining automatic channel assignment reduces Avg. F1 from 65.10 to 62.01 (Figure 5(a)).

Giving Base the OmniConfess prompt did not improve Avg. F1. On all 3,540 examples, Base’s unweighted mean F1 across the six subsets decreased by 0.315 points (Appendix F.2, Table 5). On PhD, for example, both methods receive the same instructions. Base generates an answer from the full input. OmniConfess rescores the same candidate with and without the accompanying text and uses this per-example score difference together with the instructions to guide correction. This signal comes from rescoring the original input under ablation, without externally supplied sample-level channel annotations.


W4

The main text gives no per-dataset counts, sampling criteria, description of how PubMedQA’s three-way labels were collapsed into binary judgment, or class balance.

Response. We thank the reviewer for pointing out that these evaluation details were difficult to locate. The “Datasets” paragraph in Experimental Setup now introduces OmniGroundBench and directs readers to the “OmniGroundBench Details” appendix. The “Benchmark Construction” subsection explains how the examples were selected.

The “Benchmark Composition” subsection of the “OmniGroundBench Details” appendix reports the sample counts for all six subsets in Table 3 and explains the PubMedQA label conversion. We retained all 468 examples. We relabeled the eight examples marked Maybe based on the main conclusion of the corresponding abstract, yielding five Yes and three No labels. Table 3 reports the final Yes/No counts.

AuthorsOpenReview ↗

Response to Reviewer EQKh W5 & W6 & W7

W5

A benchmark named OmniHalluBench already exists (Yang et al., EMNLP 2026 Findings, 10,000 samples, six cross-modal tasks) and is neither cited nor compared against, so reusing the name will confuse readers.

Response. We thank the reviewer for pointing out the potential confusion caused by the overlapping benchmark names. We have renamed our benchmark OmniGroundBench throughout the revised manuscript to distinguish it from Yang et al.’s OmniHalluBench. We have also cited Yang et al. (2026) in the “Omni-Modal Hallucination and Preference Training” paragraph of Related Work.


W6

Mixing binary judgment with token-F1 generation across text-only and audio-visual sources makes the reported average F1 an average of quantities with different chance levels.

Response. We thank the reviewer for raising this point about how to interpret the overall Avg. F1 when judgment and generation metrics have different chance levels. We included Avg. F1 in Table 1 to summarize the overall performance trend across the six subsets. In Table 4 of the new “Results by Task Family” appendix, we separately report mean Yes-F1 for binary judgment and mean Token-F1 for free-form generation, giving equal weight to each dataset with a reported score within its task family. OmniConfess achieves the highest reported mean in both task families on all three backbones. The appendix table provides the full breakdown and notes the dataset coverage for MAD.


W7

The text claims consistent superiority across all datasets and metric, while several supporting margins are under one F1 point, and on Nemotron the method is not best on PHD, CMM, or AVHBench.

Response. We thank the reviewer for pointing out the need to clarify the scope of our conclusions and explain the three exceptions on Nemotron. We revised Main Results to clarify that, with Qwen2.5-Omni-7B, OmniConfess outperforms the baselines on all reported metrics across the six datasets. The smallest F1 lead is on HaloQuest, where OmniConfess outperforms the strongest baseline, ToT, by 1.05 Token-F1 points. Among methods evaluated on all six datasets, OmniConfess achieves the highest Avg. F1 on each of the three backbones (Table 1).

In Cross-Backbone Portability (RQ4), we quantified and analyzed the three exceptions on Nemotron. Compared with Base, OmniConfess improves F1 by 1.52, 4.14, and 2.47 points on PhD, CMM, and AVHBench, respectively. Compared with the strongest baseline on each dataset, OmniConfess trails GD by 1.70 F1 points on PhD, ToT by 0.06 F1 points on CMM, and AVCD by 1.08 Token-F1 points on AVHBench. Across all six datasets, OmniConfess improves Nemotron’s Avg. F1 by 8.51 points over Base.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response to the Reviewers and Area Chair (Part 1/4).

Thank you for taking the time to read our paper carefully and share your constructive feedback. Your comments helped us explain the rationale behind OmniConfess’s design, clarify the conditions of our comparisons, and document the evaluation in greater detail. The table below summarizes our responses to your comments on method design and validation and points to the corresponding sections of the revised manuscript.

ThemeReviewer comments, clarifications, and revisionsManuscript location
1. Method Design and ValidationWe clarify how channel diagnoses guide correction and where the additive approximation is used.§4.3, §4.4, RQ2, and the associated appendices
EQKh W1 · We clarify how Confession’s diagnoses can be evaluated independently of downstream correction. Changes in the Yes/No margin after controlled evidence replacement provide a reference for the channel on which the model depends. We added a controlled audio and video replacement experiment on CMM and report channel agreement and the judgment flip-rate gap.RQ2, Evidence Interrogation Strategies and appendix, Faithfulness Evaluation
EQKh W2 · We clarify how the correction controls distinguish a uniform Yes/No score shift from example-specific channel selection. The random-channel control uses the same candidates and rescoring budget as OmniConfess. We added global label-bias calibration and equal-budget random-channel correction.RQ2, Module Ablation and appendix, Correction Controls
72LA W1 · We explain why we use an additive approximation to balance correction quality and inference cost. The two single-channel passes provide channel-specific evidence scores for correction and allow us to estimate the joint removal effect. Measuring that effect directly requires a third pass if both scores are retained. We added a 2 × 2 audio–visual intervention on CMM to quantify approximation error and the cost of the extra pass.RQ2, Evidence Interrogation Strategies and appendix, Audio–Visual Interaction Analysis, Table 8
EQKh W3 · We clarify what each anchoring strategy holds fixed. Shared Prefix fixes the scoring prefix across evidence conditions. Commitment Anchoring also fixes the full candidate response and its alternative tokens. Figure 5(b) compares average F1 and hallucination localization AUPRC.RQ2, Anchoring Analysis, Figure 5(b)
Q1hb W1.2/W1.3/Q1 & bZg4 W1/Q2 · We clarify how candidates and diagnostic states are produced and used. The initial candidate is generated greedily, and each alternative is the highest-probability unselected token under the full-evidence distribution. Rescoring holds the candidate, alternatives, and scoring prefix fixed. The three discrete states support post-hoc interpretation, while continuous evidence scores and separate thresholds govern correction. We document evidence interventions, score differences, and diagnostic state assignment.Commitment Anchoring, text following Eq. (9), and appendix, OmniConfess Implementation Details
Q1hb W3 · We explain the rationale for both correction rules. Full subtraction of the residual preference and irrelevant-channel contribution leaves the estimated relevant-evidence contribution as the corrected judgment margin. The three free-form risk terms share a log-probability margin scale, and unit weights provide a fixed default without additional fitting. We added worked examples on CMM and HaloQuest, including a single-example analysis that removes each risk term in turn.Appendix, Case Studies, Correction Mechanisms
AuthorsOpenReview ↗

General Response to the Reviewers and Area Chair (Part 2/4).

ThemeReviewer comments, clarifications, and revisionsManuscript location
2. Prompt Fairness and Evaluation ProtocolWe clarify what information the prompts, channel selection, and ablation-based rescoring each use. The main evaluation uses the original task inputs without externally supplied sample-level channel annotations.§4.1, Prompting Setup, Appendix F.2, Relevance Mapping Sensitivity, and Table 10
Q1hb W1.1/Q2 & EQKh W3 · We clarify where channel assignments come from and what a new task requires. On CMM, the main evaluation derives the relevant channel from the original question. Dataset annotations are used in a separate control. A new task requires a relevance rule based on information available at inference time. We compared question-based, annotation-based, and swapped audio–video assignments under the same correction rule.Appendix, Evidence Interrogation, Table 10, and Relevance Mapping Sensitivity
Q1hb W2 & EQKh W3 · We clarify how the ablation isolates task-level evidence-use instructions within OmniConfess. Automatic channel assignment and correction remain fixed when the instructions are removed. We added the w/o Evidence-Use Instructions setting.RQ2, Module Ablation, Figure 5(a)
bZg4 Q1 and follow-up & Q1hb W2 · We clarify that giving Base the OmniConfess prompt did not improve its mean F1 across the six subsets. On PhD, both methods receive the same instruction to answer from the image. OmniConfess uses the score difference from rescoring with and without the accompanying text to guide correction. We added a prompt control on all 3,540 examples and provided the PhD and RAGTruth prompts.§4.1, Prompting Setup, and Appendices A.1 and F.2
bZg4 Q2 and follow-up · We clarify that ablation-based rescoring uses original inputs available to all methods, which share the same evaluation examples and final scoring protocol. On PhD, OmniConfess removes the accompanying text for internal rescoring. On CMM, it identifies the relevant modality from the original question and removes audio and video in separate passes. Table 10 specifies what each intervention removes and retains, including the text evidence boundaries in PubMedQA and RAGTruth.Appendix, Evidence Interrogation, Table 10
AuthorsOpenReview ↗

General Response to the Reviewers and Area Chair (Part 3/4).

ThemeReviewer comments, clarifications, and revisionsManuscript location
3. Expanded Evaluation and ReproducibilityWe clarify which tasks the comparisons cover, how results are aggregated, and how to reproduce the experiments.Experimental Setup, Main Results, RQ2, RQ4, RQ5, and the associated appendices
72LA W2 & EQKh W4 · We clarify the benchmark’s modality coverage and judgment labels. PubMedQA and RAGTruth use text only, PhD and HaloQuest use images and text, and CMM and AVHBench use audio, video, and text. The benchmark appendix documents Yes/No distributions and PubMedQA’s label conversion. We also report subset sizes, candidate sources, difficulty-based selection, and manual verification.Experimental Setup, Datasets, and appendix, OmniGroundBench Details, Table 3
72LA W2 & EQKh W6 · We clarify that Avg. F1 weights each subset with a reported score equally. Judgment and free-form generation also have separate summaries because they use different F1 metrics. The task-family breakdown reports mean Yes-F1 and mean Token-F1.Main Results, Table 1 caption, and appendix, Results by Task Family, Table 4
72LA W2 · We clarify the conditions of the direct comparison with Token-Guard. Both methods use Qwen2.5-Omni-7B, the same 3,540 examples, and the same scoring protocol. We compare F1, generated token counts, and end-to-end latency. We adapted Token-Guard to handle audio and video inputs and added the comparison.Appendix, Comparison with Token-Guard
72LA W2 · We clarify why enabling TAF left F1 unchanged in our reproduction. Its layer-selection rule selected no layers for intervention on Qwen2.5-Omni-7B across the four subsets with visual inputs. The other two subsets use text only. We report the TAF enabled/disabled check and describe the PhD text-interference ablation.72LA W2 response and appendix, Evidence Interrogation, Table 10
Q1hb W4 and follow-up · We clarify that the free-form module ablation covers HaloQuest, RAGTruth, and AVHBench. Figure 5(a) uses the Qwen2.5-Omni-7B evaluation setup from Table 1, and overall F1 gives equal weight to the six subsets. We added the RAGTruth and AVHBench ablations and aligned the text with the updated figure.§5.3, Module Ablation, Figure 5(a)
EQKh W3 & bZg4 Q6 · We clarify how OmniConfess relates to prior attribution, decoding, and preference-training methods. OmniConfess holds a candidate fixed, measures its dependence on each evidence channel through rescoring, and uses those measurements to locate and correct unsupported spans at inference time. We expanded the discussion of SAFE, CaRe, OmniTrace, MM-SHAP, HA-DPO, SENTINEL, and TPR.Related Work and RQ2, Anchoring Analysis
EQKh W5 · We clarify that OmniGroundBench is the benchmark introduced in this work and distinguish it from Yang et al.’s OmniHalluBench. We updated the benchmark name throughout the manuscript and cited the earlier work.Throughout the manuscript and Related Work
EQKh W7 & bZg4 W2 & Q1hb follow-up (Figure 6(b)) · We clarify the scope of the performance claims. The lead across all six subsets and reported metrics applies to Qwen2.5-Omni-7B. The cross-backbone Avg. F1 comparison covers methods evaluated on all six subsets, and we identify the three exceptions on Nemotron. We aligned Figure 6(b) and the text with the 8.51-point Nemotron gain in Table 1.Main Results, Table 1, and §5.5, Cross-Backbone Portability, Figure 6(b)
AuthorsOpenReview ↗

General Response to the Reviewers and Area Chair (Part 4/4).

ThemeReviewer comments, clarifications, and revisionsManuscript location
4. Implementation and ReproducibilityWe clarify the experimental settings, scoring and timing protocols, and access to reproducibility materials.Experimental Setup, RQ5, and associated appendices
Q1hb W1.4 & bZg4 W1/Q2 · We clarify how correction thresholds are selected and shared. Judgment and free-form generation use separate settings, shared across datasets and backbones within each task family. The full evaluation sets include the stratified subsets used for threshold selection. We document the selection protocol and report sensitivity to nearby thresholds.Appendix, OmniConfess Implementation Details
bZg4 W1/Q2 · We clarify that GPT-4o assigns the GAV scores in Table 1 on a 0–10 scale using task-specific evaluation prompts. We also specify the decoding settings and baseline hyperparameters used in the experiments. We document the actual configurations, including Self-Consistency sampling and the VCD contrast coefficient.Experimental Setup, Evaluation Metrics, and appendices, Backbone and Baseline Details and OmniConfess Implementation Details
bZg4 W3 · We clarify that the gain highlighted in the conclusion is the Token-F1 improvement over Base on RAGTruth with Qwen2.5-Omni-7B. We specify the backbone, subset, metric, and comparator in the conclusion.Discussion and Conclusion
bZg4 Q3 · We clarify that MAD’s reported average covers CMM and AVHBench. We also explain how representative GD, PD, and ToT outputs can lead to low scores. We added a baseline failure analysis and documented MAD’s coverage in the main table caption.Table 1 caption and appendix, Results by Task Family, Baseline Failure Analysis
bZg4 Q4 · We clarify that latency is measured end to end per example with batch size one, CUDA synchronization at both timing boundaries, and warm-up runs excluded. Generated token counts include discarded candidates and revisions. Teacher-forced rescoring time is included in end-to-end latency. We documented the timing and token-counting protocol.RQ5, Inference Efficiency, Table 2
bZg4 Q5 · We corrected the reference answer for the PhD cat example and aligned the AVHBench discussion with the outputs shown in the figure. We updated the affected figures and case-study text.Figures 11 and 12 and appendix, AVHBench Case Study
72LA Ethics · We checked and updated the anonymous repository link for reproducibility materials.Anonymous repository link in the manuscript

The reviewer-specific responses follow the original W/Q numbering and provide detailed results and explanations. Thank you again for the time and care you put into reviewing our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.