acceptodds
Under review as a conference paper at ICLR 2027

TAG-GRPO: Mitigating Cross-Modal Hallucinations via Task-Aware Grouped GRPO

Abstract

Large omni-modal models jointly process audio, video, and language, yet they often produce correct answers through incorrect modality reasoning—relying on spurious correlations rather than task-relevant evidence, which results in cross-modal hallucinations in audio-visual understanding. Existing modality-aware preference optimization methods mitigate this issue via offline perturbation pairs; however, their static supervision struggles to capture task-dependent modality roles. To address this, we propose , an online policy optimization framework that extends GRPO from standard response-only rollouts to task-aware, modality-conditioned rollouts. Specifically, TAG-GRPO generates task-aware modality conditions through diffusion-based corruption and routes them into Intact, Robustness, and Sensitivity groups according to the task type. Group-wise advantage normalization then independently optimizes correctness under full evidence, invariance to irrelevant modality perturbations, and sensitivity to the disruption of task-relevant evidence. This explicit behavioral separation encourages the model to ground its reasoning in proper modality evidence, thereby suppressing cross-modal hallucinations beyond what answer-level correctness alone can achieve. Extensive experiments on AVHBench and CMM demonstrate that TAG-GRPO improves audio-visual grounding and reduces cross-modal hallucinations, highlighting the critical importance of optimizing modality-faithful behavior beyond answer correctness.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
large omni-modal models, GRPO, Reinforcement Learning Post-Training

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kncQOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a new framework for mitigating cross-modal hallucinations using GRPO. The main contribution of this work is that it creates counterfactual data by corrupting different modalities of an audiovisual stream and enforce that the correct model behavior across those corruptions is rewarded, on yes/no tasks that are based on object/sound presence.

Critical strengths
  • Formulation of TAG-GRPO is easy and it only requires minor tweaks to an existing GRPO pipeline and requires some additional data.

  • The paper has also shows results on general benchmarks beyond cross-modal hallucination benchmarks to show the general performance trend on performing TAG-GRPO.

Critical weaknesses
  1. Too similar to GRPO and very less novelty. The paper just creates a new dataset with counterfactual modality corruption and trains GRPO with correctness reward on top of it. The dataset construction pipeline (4.1) and the modality intervention (4.2) is similar to MoD-DPO (CVPR 2026). The paper's claim are somewhat loose (see below) and require more rigorous experimentation to justify that they work.

  2. Comparison with GRPO is underspecified. Table 2 and 3 compare TAG-GRPO with standard GRPO but it does not specify the details of the GRPO baseline. Was the GRPO also trained with the same number of data samples, and corruptions as TAG-GRPO? Also, the paper works only with 8 rollouts, but does not specify how the rollouts are divided among the Rob., Sens. and Int. cases. Give only 8 rollouts and GRPO works well with some variance, the results of the paper are underspecified.

  3. Bias towards the "No" response. Equation 6 suggests that in the case of Int. and Rob. the response remains the same but in case of Sens. the response changes to "No" irrespective of the question. This can create a bias towards the No response in general. This is also shown in the results, where the precision increases and recall drops for VDAH and ADVH compared to standard GRPO. Additionally, Table 1 shows that there is also some imbalance in the training data as well in terms of Yes/No questions, that can have additional bias effect towards some classes in some category of questions.

  4. GRPO Rollout information is missing

    • The paper does not mention the split between the (A,V), (A',V), and (A, V') for the A/V tasks. Are the rollouts split between the 8 rollouts?
    • In Table-5, for the AV case, the paper shows ablation by changing the number of rollouts split between the different cases. However, having a single sample in a group makes no sense for GRPO as the advantage would come to be zero and that group will have no contribution to the loss.
  5. No hyper-parameter sensitivity analysis for (wIntw_\text{Int}, wRobw_\text{Rob}, wSensw_\text{Sens}). L352-355 just mention the weights but do not mention the sensitivity of training to these. Moreover, I could not these weights defined in the Method section to specify how they are incorporated to the loss function.

Questions for authors

The authors should address the above concerns mentioned in weaknesses and answer the questions in between. I don't have more questions as there are details regarding GRPO that are not clear as of now. I look forward to the author discussion period to make the assessment of the work better. Good luck to the authors!

6 comments
AuthorsOpenReview ↗

Response to Reviewer kncQ (1/2)

We thank the reviewer for the comments.

W1. Novelty relative to GRPO and MoD-DPO.

We acknowledge that our diffusion-based corruption follows VCD Sec\. 3 and takes the strength s = 500 from MoD-DPO Sec\. 4\.2; our dataset is a binary QA set built from VGGSound. The contribution lies in the training objective: task-dependent conditions are routed into groups with evidence-conditioned targets and optimized on-policy. The GRPO baseline uses exactly the same data, so the comparison isolates the objective rather than the dataset, and new controls isolate each ingredient with the same data and rollout budget Sec\. 5\.3, Table 5, App\. D:

  • Same data, ordinary GRPO: over three seeds, TAG-GRPO improves the AVHBench average by 0.59 p=0˙02p = \.002 on Qwen and 4.09 p=0˙33p = \.033 on MiniCPM, and CMM HR by 2.41 and 7.56.

  • Same corrupted inputs, no target flipping All−RobAll-Rob: not consistently helpful; relative to GRPO, AVM changes by −4.59 and +5.70, the latter with HR −5.69.

  • Same conditions and targets, offline TAG−DPOTAG-DPO: the AVHBench average drops relative to DPO by 0.58 and 1.45.

  • MoD-DPO trained on the same data: TAG-GRPO is higher on the AVHBench average by 4.29 and 1.46 and on CMM balanced accuracy by 1.29 and 4.17 (single runs, Tables 2–3).

We have also tightened our claims: the logit-based analysis App\. F shows where TAG-GRPO improves discrimination and where it mainly shifts the operating point, as detailed in W3 below and in the General Response.

W2. GRPO baseline and rollout split.

GRPO uses the same 7,944 training samples, the same regular-expression-based binary reward, the same hyperparameters (LoRA configuration, learning rate, number of epochs), and 8 rollouts per sample, all on the intact input with the teacher answer; it uses no corruption. TAG-GRPO uses the same budget: for A/V tasks, Int/Rob/Sen = 4/2/2, and for AV tasks, 8 intact rollouts Sec\. 5\.1, App\. B. To address variance, we report three seeds per method with Welch's t-tests App\. C; General Response 1/2. We also doubled the budget to 16 rollouts per sample: TAG-GRPO with 12/2/2 remains above GRPO on all three AVHBench subsets, by 0.44, 1.14, and 1.49, and is nearly tied in CMM balanced accuracy, +0.08 single runs, App\. C.

AuthorsOpenReview ↗

Response to Reviewer kncQ (2/2)

W3. Bias toward No.

We agree that the Sen target pushes toward No, and we now quantify it App\. F; Yes ratio in %, three-seed means:

BackboneMethodADVHVDAHAVMCMM
Qwen2.5-Omni-7BGRPO48.2766.8134.8959.38
TAG-GRPO45.2264.9935.0457.67
MiniCPM-o-2.6GRPO51.4757.9872.9252.69
TAG-GRPO44.4547.6745.3147.69

All label priors are about 50%. In six of eight backbone–subset pairs, TAG-GRPO moves the Yes ratio closer to 50%, mostly by reducing unsupported Yes answers, e.g., MiniCPM AVM from 72.9% to 45.3%. On VDAH, the precision–recall shift the reviewer notes comes from fewer Yes answers relative to a Yes-heavy baseline. On ADVH, however, the Yes ratio drops to 44–45% on both backbones, which is over-correction, and we state it in the Limitations. On Qwen, AUROC still improves on all three AVHBench subsets, so the AVHBench gains are not only a threshold shift. The strength of the push is controllable through w_Sen App\. E, and the Sen objective has a nominal share of only 6% of the overall objective Sec\. 4\.4.

Regarding the training labels, Yes accounts for 63.7%, 37.9%, and 82.2% of the A, V, and AV tasks and 63.4% overall, so the label imbalance favors Yes rather than No; GRPO is trained on the same labels, so the difference between the two methods comes mainly from the Sen target. We also tested a Cannot determine Sen target; it lowers the AVHBench average by 1.99 and 2.22 App\. E.

W4. Rollout split and singleton groups.

  • For A/V tasks, the 8 rollouts are split as Int/Rob/Sen = 4/2/2 over the intact pair, the pair with the task-irrelevant modality corrupted, and the pair with the task-relevant modality corrupted; for t=At=\mathsf{A} these are (A,V)(A,V), (A,V~)(A,\tilde V), and (A~,V)(\tilde A,V).

  • In the AV rollout-allocation table (Table 6, formerly Table 5), both corrupted AV conditions are routed to Sen with target No and normalized jointly, so 6/1/1 and 4/2/2 correspond to Int/Rob/Sen = 6/0/2 and 4/0/4. No singleton group arises; we agree that a singleton would receive zero advantage, and we clarified this in the caption and App. B.

W5. Definition and sensitivity of the group weights.

The weights w_{t,g} enter the objective in Eq. 9 Sec\. 4\.4, which divides by their sum so that the per-sample loss scale stays at 1; removing a group sets its weight to 0. The default is 0.7/0.2/0.1 for A/V tasks and 1/0/0 for AV tasks Sec\. 5\.1. App. E reports sweeps of w_Sen ∈ {0, 0.05, 0.1, 0.2} on both backbones and w_Rob ∈ {0.1, 0.3} on Qwen, together with uniform weights and pooled normalization, all with 8 rollouts. AVM and the AVHBench average peak at the default w_Sen = 0.1 on both backbones, raising w_Sen trades PA for HR on Qwen, and uniform weights lower AVM by 3.14–4.69. The full table is in our response to Reviewer b7jN W3W3.

We thank the reviewer again for the constructive comments, which have substantially improved the paper. We hope our responses and revisions address the concerns, and we would be glad to discuss any remaining questions.

W1. I still have concerns about the novelty of this work. The main novelty is still only the things that were mentioned in my original response.

W2. For GRPO baseline, I still don't understand that why the authors have kept the groups to be uneven among I/R/S sets. Low group size for R/S sets still provides very noisy signal for training. A more balanced set such as 6/5/5 might give a better reward signal. My understanding is that increasing the robustness or sensitivity group size may reduce the performance of the method.

W4. Routing both the corrupted AV conditions to the sensitivity target does not sound like a good idea. It is not clear why that choice is made. Making that choice removes the effect of the Robustness group from Table 6.

W5. It is unclear what happens at higher or equal weights of the robustness and sensitivity groups and when the weights of all the groups are set equal to each other.

My current rating reflects my evaluation. I thank the authors for their hard work preparing the responses!

Reviewer_VBHtOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TAG-GRPO, an online preference optimization framework that extends GRPO to mitigate cross-modal hallucinations in omni-modal LLMs. The key idea is to perform task-aware, modality-conditioned counterfactual rollouts and to route them into Intact, Robustness, and Sensitivity groups, with group-wise advantage normalization and task-group weighting, so the policy is optimized for correctness under full evidence, invariance to irrelevant perturbations, and sensitivity to disruption of task-relevant evidence. Using a compact, automatically annotated training set derived from VGGSound and diffusion-based modality corruptions, the method yields improvements on AVHBench and CMM—particularly in audio-visual matching and hallucination resistance—while showing mixed trade-offs (e.g., precision/recall and perception accuracy vs. hallucination resistance).

Critical strengths
  • The task-dependent treatment of modality corruption is well motivated. Distinguishing robustness to irrelevant-modality corruption from sensitivity to relevant-modality corruption addresses a limitation of treating all perturbations alike. It provides a concrete way to train evidence-dependent behavior.

  • The evaluation covers complementary aspects of audio–visual reliability. Experiments use two omni-modal backbones, two hallucination benchmarks, and two general audio–visual benchmarks. This helps assess both hallucination reduction and possible effects on broader task performance.

  • The ablations and reporting make the tradeoffs visible. The paper examines task composition, behavior groups, corruption coverage, and rollout allocation. It also acknowledges mixed results against ordinary GRPO, precision–recall shifts, and losses in perception accuracy, which makes the contribution easier to assess.

Critical weaknesses
  • The supervision does not clearly separate evidence dependence from conservative answering. Equation (6) assigns “No” whenever the task-relevant modality is corrupted, although unreliable evidence does not necessarily imply a negative answer. The lower recall on modality-specific tasks and reduced perception accuracy make this concern important. These results do not prove conservative bias, but the paper needs controls to distinguish improved grounding from a tendency to answer “No” under corruption.

  • The additional benefit over ordinary GRPO remains uncertain. On Qwen, AVHBench accuracy gains range from 0.05 to 1.01 points. On MiniCPM, audio–visual matching improves, while both modality-specific hallucination accuracies decline. The results lack uncertainty estimates, and no clearly documented control isolates grouping and normalization while keeping corrupted inputs, targets, and rollout budgets fixed. This makes it difficult to identify the source and reliability of the gains.

  • Generalization beyond the chosen corruption is insufficiently tested. Training uses a fixed diffusion step of s=500s=500, without establishing robustness to corruption strength or more realistic interventions such as mismatched audio and temporal misalignment. The model may partly learn a response to synthetic noise rather than evidence dependence more broadly. The automatically generated training labels also lack a reported human quality audit, leaving another source of uncertainty in the supervision.

Questions for authors
  1. How do you distinguish improved grounding from a more conservative answer policy? Since every Sensitivity example targets “No,” can you report yes-rates and false-positive/false-negative rates, and compare against an uncertainty target such as “cannot determine”?

  2. What does grouping add beyond training on the same augmented inputs? Can you compare with ordinary GRPO while holding corrupted inputs, reference targets, rollout budgets, and loss weights fixed, changing only the grouping and advantage normalization?

  3. Are the improvements over GRPO reliable across training runs? Please report multi-seed results for AVHBench and CMM, particularly where the gains are small or accompanied by lower recall or perception accuracy.

  4. Does the learned behavior transfer beyond the chosen corruption? How do results change with diffusion strength and with interventions such as audio–video mismatching or temporal misalignment? This would help establish whether the model learns evidence dependence rather than a response specific to synthetic noise.

6 comments
AuthorsOpenReview ↗

Response to Reviewer VBHt (1/3)

We thank the reviewer for the precise questions. We answer the weaknesses together with the corresponding questions.

W1 / Q1. Grounding vs. a more conservative answer policy.

No is a conservative supervision target meaning that the task-relevant evidence does not support Yes, not a factual claim about the scene. We added four analyses.

Recognizability of corrupted positives App\. I. At s = 500, among manually verified positives, annotators could not recognize any of 50 corrupted audio clips, and could confidently answer Yes for only 8 of 50 corrupted videos, all showing large objects, from frames sampled at the model's input resolution. Such cases account for about 0.2% of the objective weight.

Yes ratios and error rates App\. F; three-seed means; Yes ratio % with FNR/FPR %:

BackboneMethodADVHVDAHAVMCMM
Qwen2.5-Omni-7BGRPO48.27 175˙/142˙17\.5/14\.266.81 37˙/375˙3\.7/37\.534.89 348˙/46˙34\.8/4\.659.38 72˙/260˙7\.2/26\.0
TAG-GRPO45.22 196˙/102˙19\.6/10\.264.99 45˙/346˙4\.5/34\.635.04 349˙/50˙34\.9/5\.057.67 82˙/236˙8\.2/23\.6
MiniCPM-o-2.6GRPO51.47 123˙/153˙12\.3/15\.357.98 119˙/278˙11\.9/27\.872.92 83˙/541˙8\.3/54\.152.69 122˙/176˙12\.2/17\.6
TAG-GRPO44.45 194˙/83˙19\.4/8\.347.67 204˙/158˙20\.4/15\.845.31 253˙/159˙25\.3/15\.947.69 146˙/100˙14\.6/10\.0

All label priors are about 50%. TAG-GRPO moves the Yes ratio closer to 50% in six of eight backbone–subset pairs and lowers the FPR on all subsets except Qwen AVM. ADVH is the exception: the Yes ratio drops to 44–45% on both backbones, which we report as over-correction in the Limitations.

Threshold-free and matched-PA metrics App\. F; checkpoints of Tables 2–3:

BackboneMethodAUROC ADVHAUROC VDAHAUROC AVMAUROC CMMCMM HR at GRPO's PA
Qwen2.5-Omni-7BGRPO0.9180.9250.9120.93473.92
TAG-GRPO0.9280.9320.9280.93473.97*
MiniCPM-o-2.6GRPO0.9320.8970.8690.92480.75
TAG-GRPO0.9270.8800.8310.93987.42*

*Expected HR with interpolation at tied scores.

On Qwen, AUROC improves on all three AVHBench subsets, indicating better discrimination there, whereas the CMM HR gain mainly reflects a more conservative operating point. On MiniCPM, CMM discrimination improves, whereas the AVM gain mainly corrects GRPO's strong Yes bias rather than improving discrimination.

AuthorsOpenReview ↗

Response to Reviewer VBHt (2/3)

Cannot determine target App\. E. As suggested, we trained a variant in which the prompt for all tasks and groups offers "Yes, No, or Cannot determine" and the Sen group is rewarded only for Cannot determine; Int and Rob keep the teacher answer. Evaluation uses the original Yes/No prompt, and no model produced any other answer.

BackboneMethodADVHVDAHAVMAvg.PAHRBal.
Qwen2.5-Omni-7BGRPO (3 seeds)84.1679.4280.2881.2892.7874.0383.40
TAG-GRPO, Sen = No (3 seeds)85.1080.4580.0681.8791.7876.4484.11
TAG-GRPO, Sen = Cannot determine84.1577.9977.5179.8892.9274.0883.50
MiniCPM-o-2.6GRPO (3 seeds)86.2180.1568.8278.3987.8382.4485.14
TAG-GRPO, Sen = No (3 seeds)86.1281.8979.4282.4885.3990.0087.69
TAG-GRPO, Sen = Cannot determine85.2178.8676.7180.2686.4289.0887.75

The abstention target lowers the AVHBench average by 1.99 and 2.22. On MiniCPM, it still corrects GRPO's Yes bias, with AVM 7.89 above GRPO, and matches the No target in CMM balanced accuracy. On Qwen, it returns to GRPO's CMM operating point and falls below GRPO on the AVHBench average. Because the model never abstains under the Yes/No prompt, an abstention target supervises the binary answer only indirectly, and the prompt change may also contribute. We therefore keep No for binary evaluation.

W2 / Q2. What grouping adds beyond the same augmented inputs.

Grouping alone does not improve the AVHBench average at matched weights; its benefit is that group weights become controllable independently of rollout counts. The pooled-normalization variant is the requested control: it uses the same corrupted inputs, the same targets including No for Sen, 4/2/2 rollouts, and the same per-rollout weights of 0.5/0.25/0.25 as the group-wise variant; only the advantage normalization differs, and pooling all rollouts of a sample is ordinary GRPO on the augmented inputs. On Qwen App\. E, group-wise normalization raises AVM by 2.29 but lowers VDAH by 4.10 relative to pooling, with AVHBench averages of 80.51 and 81.20, while the default weighting of 0.7/0.2/0.1 reaches 81.87 over three seeds. A second control, All-Rob, keeps the same perturbed inputs and rollouts but does not flip the target; its AVM is 4.37 (Qwen) and 4.90 (MiniCPM) below TAG-GRPO App\. D. The main source of the gains is therefore the evidence-conditioned target optimized online.

AuthorsOpenReview ↗

Response to Reviewer VBHt (3/3)

W2 / Q3. Reliability across runs.

We now report three seeds per method with Welch's t-tests App\. C; table in General Response 1/2. On Qwen, TAG-GRPO improves ADVH by 0.94 p=0˙19p = \.019, VDAH by 1.03 p=0˙23p = \.023, and the AVHBench average by 0.59 p=0˙02p = \.002, while AVM is on par; HR rises by 2.41 and PA drops by 1.00. On MiniCPM, the single-run decreases on ADVH and VDAH do not persist over three seeds −00˙9,p=8˙1;+17˙4,p=3˙0−0\.09, p = \.81; +1\.74, p = \.30, whereas AVM rises by 10.60 p=0˙11p = \.011, HR by 7.56 p=0˙03p = \.003, and PA drops by 2.44 p=0˙20p = \.020. With 16 rollouts per sample, TAG-GRPO remains above GRPO on all three AVHBench subsets, by 0.44, 1.14, and 1.49, and is nearly tied in CMM balanced accuracy, +0.08 single runs, App\. C. The lower recall on ADVH and VDAH comes with higher precision and lower FPR, and we discuss it together with the PA drop in W1 above and in General Response CC.

W3 / Q4. Generalization beyond the chosen corruption, and label quality.

Diffusion strength App\. E; Qwen2\.5-Omni-7B:

SettingADVHVDAHAVMAvg.PAHR
GRPO (3 seeds)84.1679.4280.2881.2892.7874.03
TAG-GRPO, s = 25084.9580.9277.3581.0791.6776.42
TAG-GRPO, s = 500 (3 seeds)85.1080.4580.0681.8791.7876.44
TAG-GRPO, s = 75085.0482.5879.1682.2691.3377.58

TAG-GRPO stays above GRPO on the AVHBench average for s = 500 and 750, by 0.59 and 0.98. At s = 250, AVM drops by 2.71 and its Yes ratio falls from 35.0% to 29.9%, consistent with weak corruption leaving evidence that conflicts with the No target; s therefore needs to be large enough to remove the evidence. All three settings lie on the same CMM PA–HR line. We did not train with other interventions such as audio–video mismatching or temporal misalignment, and we now state this in the Limitations. The evaluation inputs are unperturbed, however, so the reported gains are not measured on diffusion noise.

Teacher-label audit App\. I. Two annotators independently labeled 150 stratified samples, 50 per task, viewing only the task-relevant modality. Reweighted to the training distribution, label accuracy is 88.7% and 90.7%; inter-annotator agreement is 95.3% Cohen′sκ=09˙0Cohen's κ = 0\.90. About 80% of the errors are missed positives, i.e., the teacher answers No where the annotators answer Yes, which also explains part of the PA drop shared by all trained methods.

We thank the reviewer again for the constructive comments, which have substantially improved the paper. We hope our responses and revisions address the concerns, and we would be glad to discuss any remaining questions.

Reviewer_b7jNOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

TAG-GRPO is a framework designed to address cross-modal hallucinations in omni-modal large language models by introducing task-adaptive modality-conditioned rollouts and behavior-oriented group optimization. The framework categorizes responses into three groups—Intact, Robustness, and Sensitivity—to independently learn evidence-grounded reasoning, invariance to irrelevant noise, and sensitivity to the absence of task-relevant evidence, respectively, enabling fine-grained control over modality fidelity. Experimental results on AVHBench and CMM demonstrate that TAG-GRPO improves audio-visual grounding and effectively mitigates hallucinations by promoting modality-grounded reasoning rather than merely optimizing answer accuracy.

Critical strengths

None

Critical weaknesses

1. Insufficiently Established Contribution Beyond Prior Modality-Aware Optimization

The methodological contribution beyond prior modality-aware preference optimization is not sufficiently established. MoD-DPO already encourages invariance to irrelevant-modality corruption and sensitivity to relevant-modality corruption, including exchanging these roles for audio- versus visual-related prompts. OmniDPO likewise uses corrupted audio and video inputs to construct modality-preference pairs.

TAG-GRPO differs by using online rollouts and task-dependent behavioral grouping. However, the Introduction’s criticism that prior methods use predefined modality relationships is not fully resolved by the proposed method: the task-relevant modality sets, routing rules, and group-specific targets are also predefined. Online response sampling should be distinguished from online adaptation of modality-relevance rules. The paper would benefit from a clearer account of its incremental contribution and controlled comparisons that isolate the benefits of online optimization and grouping under matched intervention, supervision, and training-budget conditions.

2. Limited Evidence for Hallucination Mitigation Beyond Binary Decision Tasks

The training data and reward are restricted to Yes/No decisions, and the main hallucination evaluations focus on binary judgments. Although the paper reports additional accuracy evaluations on Daily Omni and AVHBench and provides illustrative cases in Figure 1, these results do not directly establish reduced hallucinations in free-form generation. In particular, the factuality of detailed descriptions or individual claims in open-ended responses is not systematically evaluated. The scope of the claims should therefore be clarified, or supported by generation-based hallucination evaluations and corresponding qualitative analyses.

3. Missing Controlled Validation of Group-Wise Normalization and Loss Weighting

Section 4.4 argues that pooling rewards across behavioral groups would create harmful cross-objective competition and introduces non-uniform weights to balance the objectives. However, the experiments do not directly isolate these two design choices. Table 4 evaluates the inclusion of behavior groups, while Table 5 changes the rollout allocation for AV training; neither compares pooled versus group-wise advantage normalization or uniform versus non-uniform loss weighting under otherwise matched conditions. Moreover, the group ablations show mixed effects across metrics rather than consistent superiority of the full configuration. Please explain how the weights (0.7,0.2,0.1)(0.7,0.2,0.1) were selected and provide controlled comparisons and weight-sensitivity analyses with matched rollout and optimization budgets.

4. Limited Analysis of the PA–HR Trade-off

The paper acknowledges the decrease in PA accompanying improved HR on CMM, but does not isolate its cause. Given that the Sensitivity group always uses a No target, it remains unclear how much of the improvement reflects better modality grounding rather than a shift toward more conservative predictions.

Questions for authors

Although my current evaluation is negative, there is considerable room for increasing my score depending on how the authors address the concerns raised in the Weaknesses section. I encourage the authors to carefully review these concerns and provide detailed responses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer b7jN (1/2)

We thank the reviewer for the detailed assessment.

W1. Contribution beyond prior modality-aware optimization.

We agree that MoD-DPO already assigns invariance and sensitivity roles depending on whether a prompt is audio- or visual-related, and that OmniDPO builds preference pairs from corrupted audio and video. We also agree that the modality-relevance rules of TAG-GRPO, i.e., the task-relevant modality sets, routing, and group targets, are predefined. "Online" in our paper refers to on-policy sampling and optimization, not to adapting these rules, and we have made this distinction explicit in the Introduction. Our contribution is to optimize these evidence-conditioned behaviors on-policy within one grouped objective. The new controls isolate its parts under the same data, conditions, and rollout budget Sec\. 5\.3, Table 5, App\. D:

ComparisonQwen: Δ Avg.MiniCPM: Δ Avg.
TAG-DPO − DPO (same conditions and targets, offline)−0.58−1.45
TAG-GRPO − GRPO (same supervision, online; three seeds)+0.59+4.09
  • Online optimization. With identical conditions and targets, offline TAG-DPO is below DPO, whereas TAG-GRPO is above GRPO. With on-policy rewards, a Sen group that already answers No receives zero advantage, while offline pairs keep pushing toward No; consistent with this, on MiniCPM TAG-DPO over-corrects AVM toward No, with a Yes ratio of 21.5% versus 31.2% for DPO, while TAG-GRPO stays near the label prior at 45.3%.

  • Evidence-conditioned targets. All-Rob uses the same perturbed inputs but keeps the teacher answer. It does not consistently help: relative to GRPO, AVM changes by −4.59 and +5.70, the latter with HR −5.69. Flipping the target raises AVM over All-Rob by 4.37 and 4.90.

  • Grouping and budget. See W3 below.

Against MoD-DPO trained on the same data, TAG-GRPO is higher on the AVHBench average by 4.29 and 1.46 and on CMM balanced accuracy by 1.29 and 4.17 (single runs, Tables 2–3).

W2. Evidence beyond binary decisions.

We agree that training and the main evaluations are binary, and we have narrowed our claims to discriminative audio-visual hallucination Sec\. 5\.2, Limitations. We added an open-ended evaluation App\. G: GAVIE on 1,105 AVHBench captioning samples, judged by GPT-5.5. TAG-GRPO does not degrade captions, and on MiniCPM it obtains the highest GAVIE-A/R among all methods, +0.60/+1.32 over Prompting. However, its differences from GRPO are at most 0.17 and inconsistent across backbones, so we do not claim reduced hallucination in free-form generation. Daily Omni and AVIBench App\. H show no decrease in average accuracy relative to Prompting. Claim-level factuality evaluation of long-form responses remains future work.

AuthorsOpenReview ↗

Response to Reviewer b7jN (2/2)

W3. Group-wise normalization and loss weighting.

We revised Sec. 4.4 and no longer claim harmful cross-objective competition. Under group-wise normalization, a group whose rewards are all identical receives zero advantage, and each group's contribution is set by its weight rather than by its rollout count; Eq. 9 defines the weights, and dividing by their sum keeps the per-sample loss scale at 1. New controls on Qwen2.5-Omni-7B, all with 8 rollouts and the same hyperparameters App\. E:

Weights Int/Rob/SenNormalizationADVHVDAHAVMAvg.PAHR
0.7/0.2/0.1 (default, 3 seeds)group-wise85.1080.4580.0681.8791.7876.44
0.7/0.2/0 (Sen rollouts to Int)group-wise84.7779.9178.9481.2192.3375.25
0.7/0.2/0.05group-wise84.7781.1478.4681.4691.8376.75
0.7/0.2/0.2group-wise85.3982.2376.4981.3790.5079.67
0.7/0.1/0.1group-wise84.7780.0477.7280.8492.4275.17
0.7/0.3/0.1group-wise84.5180.6178.1481.0992.0876.00
1/3 eachgroup-wise84.6083.3675.3781.1190.0080.33
0.5/0.25/0.25group-wise85.2179.3976.9280.5190.5079.08
0.5/0.25/0.25pooled85.4883.4974.6381.2089.9280.83

AVM and the AVHBench average peak at the default w_Sen = 0.1, also on MiniCPM App\. E. Uniform weights raise the effective Sen weight and lower AVM by 3.14–4.69. Pooled normalization with 4/2/2 rollouts equals per-rollout weights of 0.5/0.25/0.25; at these matched weights it lowers AVM by 2.29 but raises VDAH by 4.10. Group-wise normalization therefore does not dominate pooling at matched weights; its benefit is that the default weighting, which gives the highest AVM and AVHBench average, can be set independently of rollout counts. The weights were selected on AVHBench without a separate validation split, which we state in the Limitations.

On the mixed group-ablation effects: the Table 4 variants also remove the rollouts of the removed group. With these rollouts reassigned to Int App\. D, Sen still raises AVM by 1.12 on Qwen and 8.26 on MiniCPM, and on MiniCPM removing Rob over-corrects toward No PA810˙8;addingRobbackraisesPAby43˙1andVDAHby48˙2PA 81\.08; adding Rob back raises PA by 4\.31 and VDAH by 4\.82. We agree that no group improves every metric.

W4. PA–HR trade-off.

The PA drop has two sources. First, part of the PA drop relative to Prompting is shared by all methods trained on the teacher labels: Gemini's own CMM PA is 88.67, and about 80% of its label errors are missed positives App\. I. Second, the additional drop relative to GRPO comes mainly from Sen: with the Sen rollouts reassigned to Int, PA is 92.33 on Qwen and 88.00 on MiniCPM, versus 91.78 and 85.39 with Sen. This part is controllable through w_Sen: on Qwen, raising w_Sen from 0 to 0.2 moves PA from 92.33 to 90.50 and HR from 75.25 to 79.67. To separate grounding from conservativeness, we compare HR at GRPO's PA App\. F. On Qwen, HR differs by only +0.05 and CMM AUROC is unchanged, so the CMM gain mainly reflects a more conservative operating point. On MiniCPM, HR is 6.67 higher at matched PA and AUROC rises by 0.015, indicating improved discrimination.

We thank the reviewer again for the constructive comments, which have substantially improved the paper. We hope our responses and revisions address the concerns, and we would be glad to discuss any remaining questions.

Thank you for preparing the detailed rebuttal and conducting additional experiments.

The additional experiments provide evidence that the proposed approach can outperform standard GRPO under matched supervision conditions. However, the experimental evaluation remains limited in scope. Furthermore, I believe the methodological novelty is still limited, as the proposed approach primarily extends existing modality-aware optimization strategies.

I appreciate the authors' efforts to address my concerns and strengthen the experimental validation. Accordingly, I will raise my score to 2 (Weak Reject). Nevertheless, I still have reservations regarding the originality and distinct methodological contribution of this work.

We sincerely thank the reviewer for carefully considering our rebuttal and additional experimental results, and for their willingness to raise the score. We greatly appreciate the constructive feedback and acknowledge the remaining concerns regarding methodological novelty and experimental scope.

Comments on the paper

8 comments
AuthorsOpenReview ↗

General Response (1/2): Summary of revisions and multi-seed results

Dear Reviewers,

We thank all reviewers for their careful and constructive feedback. The reviews share four concerns: AA what TAG-GRPO adds beyond GRPO and prior modality-aware preference optimization, BB whether the No target of the Sensitivity SenSen group improves grounding or only makes answers more conservative, CC the PA–HR trade-off on CMM, and DD the reliability and scope of the gains. We ran additional experiments and revised the paper; revised text is in red. The main changes are:

  1. Multi-seed statistics App\. C. Three seeds for GRPO and TAG-GRPO on both backbones, with Welch's t-tests.

  2. Fully specified baseline and rollout protocol Sec\. 5\.1, App\. B. GRPO uses the same training data, reward, hyperparameters (LoRA configuration, learning rate, number of epochs), and rollout budget of 8 as TAG-GRPO, with all rollouts on the intact input. TAG-GRPO uses Int/Rob/Sen = 4/2/2 rollouts for A/V tasks and 8 intact rollouts for AV tasks.

  3. Component controls Sec\. 5\.3, new Table 5, App\. D. TAG-DPO uses the same conditions and targets with offline DPO; All-Rob uses the same perturbed inputs without flipping the target; budget-matched variants reassign the rollouts of a removed group to Int.

  4. Weights, normalization, corruption strength, and Sen target Sec\. 5\.3, App\. E. Sweeps of w_Sen and w_Rob, uniform weights, pooled vs. group-wise normalization at matched per-rollout weights, diffusion strength s ∈ {250, 750}, and a Cannot determine Sen target.

  5. Decision bias and discrimination App\. F. Yes ratios, FNR/FPR, AUROC, Acc@50, and CMM HR at matched PA.

  6. Teacher-label audit and recognizability of corrupted inputs App\. I.

  7. Open-ended captioning with GAVIE App\. G and a 16-rollout budget App\. C.

Table 5 in the original submission (AV rollout allocation) is now Table 6.

Three-seed results mean±std;Δ=TAG−GRPO−GRPOwithWelch′sp−valuemean ± std; Δ = TAG-GRPO − GRPO with Welch's p-value:

BackboneMethodADVHVDAHAVMAvg.PAHRBal.
Qwen2.5-Omni-7BGRPO84.16±0.3279.42±0.1180.28±0.1481.28±0.0892.78±0.1374.03±0.3583.40±0.15
TAG-GRPO85.10±0.1880.45±0.3380.06±0.3281.87±0.1091.78±0.2976.44±0.4284.11±0.06
Δ pp+0.94 0˙19\.019+1.03 0˙23\.023−0.22 3˙7\.37+0.59 0˙02\.002−1.00 0˙16\.016+2.41 0˙02\.002+0.71 0˙06\.006
MiniCPM-o-2.6GRPO86.21±0.2780.15±1.6868.82±0.6478.39±0.5887.83±0.8382.44±1.4785.14±0.42
TAG-GRPO86.12±0.5381.89±1.9179.42±2.3182.48±1.5685.39±0.4690.00±1.4687.69±0.54
Δ pp−0.09 8˙1\.81+1.74 3˙0\.30+10.60 0˙11\.011+4.09 0˙33\.033−2.44 0˙20\.020+7.56 0˙03\.003+2.55 0˙04\.004

With three seeds, p-values are indicative. In summary, TAG-GRPO improves the AVHBench average and CMM HR over GRPO on both backbones, at the cost of lower PA. We characterize where these gains reflect better discrimination and where they reflect a shift of the operating point in the General Response 2/22/2 and in the individual responses.

AuthorsOpenReview ↗

Response to Reviewer qgwN (1/2)

We thank the reviewer for the careful reading.

W1 / Q1. Is No a factual label or a conservative target?

It is a conservative target: No means that the task-relevant evidence does not support Yes, not a factual claim that the queried object or sound is absent. Sec. 4.3 now frames Sen accordingly, as penalizing unsupported affirmative predictions. We also validated corrupted, originally positive examples as suggested App\. I. Among positives whose Yes labels were manually verified, at s = 500 neither annotator could recognize anything in any of 50 corrupted audio clips 9595% Wilson CI 0–7\.1%. For video, using frames sampled at 2 FPS at the model's input resolution, both annotators could confidently answer Yes for only the same 8 of 50 videos 1616%, CI 8\.3–28\.5%, all showing large objects. Extrapolated to the training set, such cases amount to about 140 samples that appear only in the two Sen rollouts with weight 0.1, about 0.2% of the objective weight. We also trained a Cannot determine target for Sen; it lowers the AVHBench average by 1.99 and 2.22 on the two backbones, and on Qwen it returns to GRPO's operating point App\. E; full table in our response to Reviewer VBHt. Whether the gains reflect grounding or a change in response policy is addressed in Q2 below.

W2 / Q2. Stability across runs and comparison at similar PA or positive-prediction rates.

Stability. Three seeds per method with Welch's t-tests are in App. C and General Response 1/2. The AVHBench average improves by 0.59 on Qwen p=0˙02p = \.002 and 4.09 on MiniCPM p=0˙33p = \.033, HR by 2.41 and 7.56, and PA drops by 1.00 and 2.44. On MiniCPM, the single-run decreases on ADVH and VDAH do not persist over three seeds −00˙9,p=8˙1;+17˙4,p=3˙0−0\.09, p = \.81; +1\.74, p = \.30.

Operating-point-matched comparison App\. F. We score each probe by PYesYes/P(YesP(Yes+PNoNo) at the first token and report AUROC, accuracy when 50% of probes are predicted Yes, matching the label prior Acc@50Acc@50, and CMM HR at GRPO's greedy PA:

BackboneMethodAcc@50 ADVHAcc@50 VDAHAcc@50 AVMAcc@50 CMMCMM HR at matched PA
Qwen2.5-Omni-7BGRPO84.3384.6384.7586.0873.92
TAG-GRPO84.6885.1685.1886.2573.97*
MiniCPM-o-2.6GRPO87.1581.9278.6885.4280.75
TAG-GRPO86.0979.8375.9187.8387.42*

*Expected HR with interpolation at tied scores.

On Qwen, AUROC and Acc@50 improve on all three AVHBench subsets, so the AVHBench gains are not only a threshold shift; on CMM, AUROC is unchanged and HR at matched PA differs by only 0.05, so the CMM gain mainly reflects a shift toward fewer positive predictions. On MiniCPM, the opposite holds: CMM discrimination improves AUROC+00˙15,Acc@50+24˙1,HR+66˙7atmatchedPAAUROC +0\.015, Acc@50 +2\.41, HR +6\.67 at matched PA, whereas the AVM gain mainly corrects GRPO's strong Yes bias Yesratio770˙Yes ratio 77\.0% vs\. 46\.2% and AVHBench AUROC and Acc@50 are lower. These single-checkpoint analyses are indicative.

AuthorsOpenReview ↗

Response to Reviewer qgwN (2/2)

W2 / Q3. What differs from standard GRPO.

We agree that with identical conditions and per-input grouping the two objectives coincide; the differences lie in the conditions and targets. Both methods use the same 7,944 training samples, the same regular-expression-based binary reward, the same hyperparameters (LoRA configuration, learning rate, number of epochs), and 8 rollouts per sample Sec\. 5\.1, App\. B.

GRPOTAG-GRPO
A/V tasks: inputs8 rollouts on the intact pair4 intact IntInt, 2 with the task-irrelevant modality corrupted RobRob, 2 with the task-relevant modality corrupted SenSen
A/V tasks: targetsteacher answerteacher answer for Int and Rob; No for Sen
AV tasks8 intact rollouts, teacher answeridentical to GRPO
Advantage normalizationover the 8 rollouts of a samplewithin each group
Loss weightsnoneInt/Rob/Sen = 0.7/0.2/0.1, normalized by their sum Eq\. 9

Two controls separate these differences App\. D, E: All-Rob keeps the corrupted inputs but not the No target, and pooled normalization keeps inputs and targets but normalizes over all rollouts of a sample. All-Rob does not consistently help AVM−45˙9and+57˙0relativetoGRPOAVM −4\.59 and +5\.70 relative to GRPO, and pooling trades AVM for VDAH at matched weights. The gains therefore come mainly from the evidence-conditioned targets optimized online; grouping makes the group weights controllable.

Q4. Rollout counts, the AV 6/1/1 setting, and the AV-only configuration.

  • For A/V tasks, the allocation is Int/Rob/Sen = 4/2/2; for AV tasks, all 8 rollouts are intact Sec\. 5\.1, App\. B.

  • In the AV rollout-allocation table (Table 6, formerly Table 5), the two corrupted AV conditions are both routed to Sen with target No and normalized jointly as one group. The 6/1/1 and 4/2/2 settings therefore correspond to Int/Rob/Sen = 6/0/2 and 4/0/4, with weights normalized to 0.875/0/0.125. No singleton group arises; the reviewer is right that a singleton would receive zero advantage under our normalization. We clarified this in the table caption and App. B.

  • The AV-only row of Table 4 trains only on AV tasks with the same routing, i.e., 6/0/2. The rows that remove a group or a corruption type also remove the corresponding rollouts, leaving 6 rollouts per A/V sample; App. D reports budget-matched variants in which these rollouts are reassigned to Int.

We thank the reviewer again for the constructive comments, which have substantially improved the paper. We hope our responses and revisions address the concerns, and we would be glad to discuss any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.