- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CARE-Flow is a training-free causal framework for reducing inter-image hallucination in multi-image MLLMs. The paper distinguishes inter-image hallucination — where visual evidence is grounded but used incorrectly across images — from conventional single-image hallucination. It identifies three failure modes: incorrect evidence selection (grounded substitution), missing required evidence (omission), and incorrect evidence combination (operator error).
The framework operates in three stages. First, it specifies the evidence structure required by the question before any response is generated, fixing the required operand roles and composition operator. Second, it causally certifies each proposed operand by intervening on image-specific attention value-output paths and comparing the necessity scores of declared versus undeclared sources. Operands that fail certification are marked as unknown rather than silently dropped. Third, it performs a hard reset discarding all visual states and runs a fresh text-only computation over only the certified evidence, preventing downstream inference from bypassing the established mediator. Responses are then verified against both a certified bridge and a structure-matched null bridge, with the front-door gap used to filter responses that are supported mainly by linguistic priors.
The paper also introduces CARE-Bench, a 12,000-question diagnostic benchmark covering operand substitution, operand omission, and operator errors across COCO-VQA, museum images, and controlled generated scenes. Experiments across three MLLMs and seven benchmarks show consistent accuracy improvements, with the largest gains on CARE-Bench. Ablations confirm that causal certification and the hard reset are the two most critical components.
Critical strengths
S1: Technically distinctive approach to evidence certification. Decomposing attention output by image source and intervening on value-output paths is more direct than methods that rely solely on attention magnitude to infer evidence provenance. Combining this with explicit evidence specification and a hard reset provides a principled mechanism for controlling which visual information enters downstream inference, rather than redistributing attention weights as in DAB.
S2: Ablation results support the core components. Table 3 shows that removing causal certification drops macro-average accuracy from 40.47 to 31.75, and removing the hard reset drops it to 35.20. Replacing causal certification with attention-based certification recovers only to 37.00, remaining 3.47 points below the full method. These gaps suggest the components are not interchangeable and each contributes meaningfully.
S3: Evaluation covers multiple models and public benchmarks. Results are reported across three frozen backbones on six public benchmarks plus CARE-Bench, with mean and standard deviation over five runs. Appendix analyses further break down results by model scale, domain, and evidence complexity, which strengthens the generalization claim.
Critical weaknesses
W1: The certification criterion may be too strict for redundant or jointly required evidence.
Certification requires every declared source to contribute more strongly than any undeclared source. This fails in two common cases. First, when two images contain the same object, the redundant image also receives a high necessity score, causing the declared source to fail certification even when it is correct. Second, for facts that require two images jointly — such as a change across views — intervening on each image alone produces low scores for both, so neither can be certified even though both are genuinely required.
The paper does not report how often certification rejects correct operands, what fraction of unknown-marked operands were semantically valid proposals, or how these rejections affect final accuracy. Experiments with redundant sources, jointly required evidence, and varying intervention strengths would clarify when the criterion is unnecessarily strict and at what cost.
W2: The distinction from DAB and related methods needs sharper articulation.
The paper compares against DAB (Dynamic Attention Balancing), which redistributes attention across images to reduce multi-image hallucination. The introduction states that CARE-Flow differs by explicitly testing source dependence through intervention rather than adjusting attention allocation. However, the paper does not include a direct analysis of when the two methods succeed and fail on the same examples, nor does it explain why attention redistribution is insufficient for the failure modes CARE-Flow targets. Since both methods operate without training and both address cross-image evidence use, a clearer empirical or mechanistic comparison — for example, showing cases where DAB fails due to insufficient source separation but CARE-Flow succeeds due to certification — would strengthen the contribution claim.
W3: The hard reset discards all visual computation, which may hurt performance when certified evidence is incomplete.
After certification, CARE-Flow removes all visual tokens and key-value caches and proceeds with text-only inference over the certified bridge. When certification fails on one or more operands — marking them as unknown — the downstream model must reason over an incomplete evidence set with no access to any residual visual signal. Table C.7 shows that atom-veracity errors account for a substantial and growing fraction of residual errors after CARE-Flow, suggesting that some failures arise from the model being unable to recover visual information that was not successfully certified. The paper does not ablate a softer fallback — for example, retaining low-confidence operands rather than marking them unknown, or preserving a limited visual context alongside the bridge — and it is unclear whether a less aggressive reset would improve performance on examples where certification partially fails.
W4: CARE-Bench is heavily dominated by one domain.
Table 5 shows that COCO-VQA accounts for 11,132 of the 12,000 questions (92.8%), with Te Papa contributing only 368 and Generated 500. The benchmark statistics and aggregate accuracy are therefore almost entirely determined by the COCO-VQA domain. Domain-macro reporting is used in the appendix but not consistently in the main results, which makes it difficult to assess how well CARE-Flow generalizes to the less common domains. The imbalance also raises the question of whether the benchmark construction pipeline scales to more visually diverse or linguistically varied settings, since the vast majority of instances come from a single well-studied dataset.
Questions for authors
For facts that require two images jointly — such as identifying that the same object appears across views — intervening on one image at a time will produce low necessity scores for both, since neither alone is sufficient. Does this mean the framework systematically fails to certify valid multi-source operands of this type? And has the authors considered a joint intervention approach, for example masking pairs of images together, to handle such cases?
4 comments
Response to cTGs (2/4)
W2: Comparison with DAB.
DAB enhances multi-image processing through token-level attention balancing, while CARE-Flow controls inference-level evidence formation and use. DAB redistributes attention over image tokens; CARE-Flow explicitly specifies question-required evidence roles, tests operand-level source dependence through interventions, and constrains how certified operands are composed. These operations directly address source-role assignment and cross-image evidence-combination errors. In source identification, intervention-based attribution achieves 0.72 AUROC / 0.58 AUPRC versus 0.61/0.42 for attention-based attribution.
To evaluate the two methods on the same individual questions, we conducted additional paired evaluations on 500 Generated-domain questions per backbone (Table R3). CARE-Flow achieves net accuracy gains of +5.60%, +5.60%, and +5.00% over DAB on LLaVA-NeXT-7B, Qwen3.5-9B, and InternVL3-8B. CARE-Flow-only correct cases outnumber DAB-only correct cases for all three backbones, demonstrating consistent question-level advantages in the paired comparison. A matched certification-rule comparison also yields +4.20%, +4.00%, and +4.40% over attention-based certification, supporting the contribution of operand-level source testing. The reconstructed CARE-Flow accuracies agree with the Appendix Generated-domain results.
We have incorporated the paired gains into Section 6.3 and added the full comparison to the Appendix, with both revisions highlighted in blue.
Table R3. Paired outcome analysis on Generated-500 ( per backbone). Full method compares CARE-Flow with DAB; Cert. rule only compares causal with attention-based certification. Each row partitions matched questions into four disjoint outcomes totaling 100%. Net gain = CARE only − Baseline only.
| Comparison | Backbone | Baseline | n | Both correct (%) | Baseline only (%) | CARE only (%) | Both wrong (%) |
|---|---|---|---|---|---|---|---|
| Full method | LLaVA-NeXT-7B | DAB | 500 | 28.00 | 5.60 | 11.20 | 55.20 |
| Full method | Qwen3.5-9B | DAB | 500 | 42.60 | 3.20 | 8.80 | 45.40 |
| Full method | InternVL3-8B | DAB | 500 | 27.00 | 4.60 | 9.60 | 58.80 |
| Cert. rule only | LLaVA-NeXT-7B | Attn. Cert. | 500 | 29.60 | 5.40 | 9.60 | 55.40 |
| Cert. rule only | Qwen3.5-9B | Attn. Cert. | 500 | 44.60 | 2.80 | 6.80 | 45.80 |
| Cert. rule only | InternVL3-8B | Attn. Cert. | 500 | 27.40 | 4.80 | 9.20 | 58.60 |
Response to cTGs (3/4)
W3: Hard reset and incomplete evidence.
Hard reset restricts downstream inference to the established evidence bridge, ensuring that subsequent generation uses certified operands. The existing ablation shows that removing hard reset reduces macro-average accuracy from 40.47% to 35.20%.
Appendix C.7 also shows that the absolute rates of Atom Conflict and A-V (incorrect answers associated with atom conflicts) both decrease across all three backbones. Their higher share among residual errors reflects larger reductions in other error categories.
To evaluate this concern directly, we conducted additional certification-stratified experiments on hard reset and evidence recovery using 1,500 model–question records (500 Generated-domain questions × three backbones; Table R4): 780 fully certified and 720 partially certified, with groups fixed by the original CARE-Flow decisions. For fully certified cases, hard reset improves accuracy from 62.18% to 66.67% and lowers structural hallucination from 31.67% to 26.92%. The same pattern holds for partially certified cases (13.47% → 16.11% accuracy; 60.56% → 57.64% structural hallucination).
Targeted evidence recovery further improves partially certified cases to 25.14% accuracy and 48.47% structural hallucination, while the same recovery without a final reset yields 22.08% and 51.81%. This comparison distinguishes two complementary functions: recovery supplies previously unresolved evidence, and reset enforces the use of the resulting certified bridge. Among the tested strategies for partially certified records, targeted recovery followed by hard reset achieves the highest accuracy and lowest structural hallucination.
We have added the certification-stratified findings to Section 6.3 and the full hard-reset and recovery evaluation to the Appendix, with the additions highlighted in blue.
Table R4. Certification-stratified evaluation on 1,500 model–question records. Accuracy, structural hallucination, and other wrong are mutually exclusive outcomes and sum to 100% within each subgroup (up to rounding). Subgroups are fixed across strategies.
| Certification status | Strategy | n | Accuracy ↑ | Structural halluc. ↓ | Other wrong ↓ |
|---|---|---|---|---|---|
| Fully certified | CARE-Flow | 780 | 66.67% | 26.92% | 6.41% |
| w/o Hard Reset | 780 | 62.18% | 31.67% | 6.15% | |
| Partially certified | CARE-Flow | 720 | 16.11% | 57.64% | 26.25% |
| w/o Hard Reset | 720 | 13.47% | 60.56% | 25.97% | |
| Low-confidence fallback | 720 | 17.08% | 56.11% | 26.81% | |
| Targeted evidence recovery | 720 | 25.14% | 48.47% | 26.39% | |
| Recovery w/o Hard Reset | 720 | 22.08% | 51.81% | 26.11% |
Response to cTGs (4/4)
W4 — CARE-Bench domain imbalance and diagnostic coverage.
We address two concerns: whether COCO-VQA's 92.8% share drives the gains and whether additional questions contribute distinct diagnostic structures. CARE-Bench spans complementary sources: COCO-VQA (everyday scenes), Te Papa (less familiar real images), and Generated (controlled visual facts). The construction pipeline validates evidence and answer uniqueness in eligible real-image pools; the final domain sizes reflect valid-question yield.
To evaluate the effect of domain weighting, we conducted additional balanced evaluations (Table R5). Gains are positive for all nine backbone–domain combinations (+6.40% to +14.19%), including Te Papa and Generated. With equal domain weights, macro gains are +12.84%, +9.66%, and +7.13% on LLaVA-NeXT-7B, Qwen3.5-9B, and InternVL3-8B. Equal-size evaluation (368 questions/domain, ) yields gains of 12.84 ± 0.82, 9.66 ± 0.76, and 7.13 ± 0.73 (mean ± SD, %). COCO-VQA accounts for one-third of the balanced score. Therefore, CARE-Bench's domain imbalance does not alter our main empirical conclusion: CARE-Flow consistently improves performance across all three visual domains.
We also measured new diagnostic structures discovered per additional 100 questions under coarse and fine taxonomies. Marginal discovery decreases from the first to the last window in all three domains (Table R5). These results quantify diminishing marginal structural novelty as the benchmark grows and show that successive questions increasingly consolidate coverage of the diagnostic configurations captured by both taxonomies.
We have added the domain-balanced findings to Sections 5 and 6.2 and the full balanced and diagnostic-discovery analyses to the Appendix, with the additions highlighted in blue.
Table R5. A: Domain-macro accuracy and domain-wise rates on all 12,000 questions (COCO-VQA / Te Papa / Generated). B: Balanced evaluation with 368 questions per domain, reporting mean ± SD (%). C: New coarse/fine evidence configurations per 100 questions, comparing first and last windows. Columns 5–6 mean Vanilla/CARE-Flow for A–B and First 100/Last 100 for C.
| Analysis | Backbone / Domain | n | Metric | Vanilla / First 100 | CARE-Flow / Last 100 | Difference |
|---|---|---|---|---|---|---|
| A: Domain macro | LLaVA-NeXT-7B | 12000 | Accuracy (%) | 33.17 | 46.01 | 12.84 |
| A: Domain rates | LLaVA-NeXT-7B | 12000 | COCO / Te Papa / Generated (%) | 27.18 / 45.92 / 26.40 | 38.72 / 60.11 / 39.20 | 11.54 / 14.19 / 12.80 |
| A: Domain macro | Qwen3.5-9B | 12000 | Accuracy (%) | 44.05 | 53.71 | 9.66 |
| A: Domain rates | Qwen3.5-9B | 12000 | COCO / Te Papa / Generated (%) | 41.53 / 47.61 / 43.00 | 50.21 / 59.51 / 51.40 | 8.68 / 11.90 / 8.40 |
| A: Domain macro | InternVL3-8B | 12000 | Accuracy (%) | 30.85 | 37.98 | 7.13 |
| A: Domain rates | InternVL3-8B | 12000 | COCO / Te Papa / Generated (%) | 29.02 / 33.32 / 30.20 | 36.63 / 40.71 / 36.60 | 7.61 / 7.39 / 6.40 |
| B: Balanced | LLaVA-NeXT-7B | 1104 | Accuracy: mean ± SD (%) | 33.17 ± 0.86 | 46.01 ± 0.94 | 12.84 ± 0.82 |
| B: Balanced | Qwen3.5-9B | 1104 | Accuracy: mean ± SD (%) | 44.05 ± 0.95 | 53.71 ± 0.96 | 9.66 ± 0.76 |
| B: Balanced | InternVL3-8B | 1104 | Accuracy: mean ± SD (%) | 30.85 ± 0.88 | 37.98 ± 0.93 | 7.13 ± 0.73 |
| C: Discovery | COCO-VQA | 11132 | New coarse configs /100: first → last | 23.29 | 0.00 | -23.29 |
| C: Discovery | COCO-VQA | 11132 | New fine configs /100: first → last | 26.28 | 0.00 | -26.28 |
| C: Discovery | Te Papa | 368 | New coarse configs /100: first → last | 7.46 | 0.07 | -7.39 |
| C: Discovery | Te Papa | 368 | New fine configs /100: first → last | 8.68 | 0.49 | -8.19 |
| C: Discovery | Generated | 500 | New coarse configs /100: first → last | 77.43 | 2.07 | -75.36 |
| C: Discovery | Generated | 500 | New fine configs /100: first → last | 159.22 | 20.58 | -138.64 |
Question for Authors — Joint intervention. CARE-Flow supports multi-image source sets . Individual-source necessity is measured with the other images retained, allowing each source to contribute conditionally to a joint fact; joint intervention measures collective support. At , joint + source-constrained certification reduces valid rejection from 31.63% to 11.22%, improves correct acceptance from 68.37% to 88.78%, and changes false acceptance from 2.04% to 3.06% (Table R2); valid acceptance also improves at and 0.6. These results show that individual interventions identify conditional contributions, while group-level certification recovers additional valid jointly supported evidence.
Response to cTGs (1/4)
We thank the reviewer for the constructive feedback on source certification and multi-image evidence use.
W1: Redundant and jointly required evidence.
CARE-Flow fixes question-required evidence roles before binding sources. The original criterion Dj>0 compares individual declared-source necessity with competing-source necessity and therefore tests conditional source dependence.
For redundant evidence, multiple images may support the same fact, but acceptable source attribution must satisfy the question-required role; visually matching yet question-irrelevant sources are distinguished from valid alternatives. For jointly required evidence, non-decomposable facts (e.g., cross-view identity) may retain a multi-image source set Sj. Even when a source is insufficient in isolation, weakening it while retaining the other images can reduce support for the joint fact, yielding positive conditional necessity Njk.
To evaluate these cases directly, we conducted additional controlled experiments comparing original individual-source and joint + source-constrained certification across both evidence types and three intervention strengths (Table R2). At ϵ=0.9, redundant valid rejection decreases from 66.33% to 7.14%, correct acceptance rises from 33.67% to 92.86%, and final-answer accuracy increases from 31.63% to 87.76%; false acceptance changes from 6.12% to 6.63%. For jointly required evidence, valid rejection decreases from 31.63% to 11.22%, correct acceptance rises from 68.37% to 88.78%, and final-answer accuracy improves from 64.29% to 83.67%, with false acceptance changing from 2.04% to 3.06%.
Across ϵ∈{0.3,0.6,0.9}, joint + source-constrained certification lowers valid rejection for both evidence types at every tested strength (Table R2). Group-level support combined with question-specific source constraints consistently recovers valid operands under redundant and joint source dependencies. The reported false-acceptance and retention rates further characterize the tradeoff between valid-evidence recovery and source selectivity.
We have summarized the certification findings in Section 6.3 and added the complete intervention-strength analysis to the Appendix, with the new material highlighted in blue.
Table R2. Controlled diagnostics on redundant and jointly required evidence (%). Valid rejection and correct acceptance are fractions of valid proposals; false acceptance is the fraction of invalid proposals accepted; retention is the fraction of all proposals retained. Accuracy is final-answer accuracy on the corresponding controlled questions.