acceptodds
Under review as a conference paper at ICLR 2027

Everything a Correct Answer Never Told You: Learning from Expert-Corrected Explanations in Colonoscopy Video QA

Abstract

In colonoscopy video question answering, a correct answer can rest on the wrong visual evidence: across the general-purpose open-source models we evaluate, 10.60% of correct responses are justified by hallucinated observations. Motivated by this finding, we introduce ColonExplainVQA, a dataset for evaluating multimodal models and for training them to answer from corrected visual evidence. It contains 7,962 video-question pairs across nine tasks from three public datasets. We minimally correct hallucinated claims in conclusions generated by Qwen 3.7 Plus while preserving supported content and retaining the original responses. These corrections provide supervision for learning both the reference answer and the visual evidence that supports it. Using ColonExplainVQA, we develop Qwen-Colon-8B by fine-tuning Qwen3-VL-8B-Instruct through supervised fine-tuning followed by direct preference optimization in modules selected through Colon Function Layer (CFL) analysis. On our manually reviewed test set, Qwen-Colon-8B leads all fifteen baselines, including proprietary models far larger than itself, with 79.15% answer accuracy; among its correct answers, the hallucination rate falls to 0.95%. Answer accuracy alone is an incomplete target; the evidence behind an answer can be curated, measured, and improved. Upon acceptance, we will publicly release the ColonExplainVQA annotations, clip-reconstruction metadata, and all associated code, subject to source-data access and licensing terms.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Video question answering, Multimodal large language models, Colonoscopy

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7r1EOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ColonExplainVQA, a colonoscopy video question-answering dataset containing 7,962 examples across nine tasks, with original and human-corrected explanations. The authors highlight that correct answers may still rely on hallucinated visual evidence and propose Qwen-Colon-8B to address this issue. The training pipeline combines evidence-curated supervised fine-tuning (SFT) with the Colon Function Layer (CFL) followed by direct preference optimization (DPO). Experiments on a manually reviewed test set demonstrate improvements in answer accuracy and reference-consistency metrics over multiple baseline models.

Critical strengths
  1. Novel and meaningful benchmark

The paper addresses an important limitation of conventional medical VQA evaluation by distinguishing answer correctness from supporting visual evidence. ColonExplainVQA retains original and minimally corrected explanations, enabling targeted analysis of hallucinated observations. The independent annotation process, clinical adjudication, and manually verified test set strengthen the reliability of the dataset.

  1. Encouraging results with efficient ablations

Qwen-Colon-8B achieves competitive results across answer accuracy and reference-consistency metrics. The comparisons between original and curated SFT supervision, together with vanilla, random-layer, last-layer, and CFL-guided DPO, provide useful evidence for the effectiveness of the overall approach. In particular, the results suggest that both supervision quality and selective adaptation influence model performance.

Critical weaknesses
  1. The proposed Reference Contradiction Rate (RCR) primarily measures explicit contradictions between generated and curated conclusions, rather than directly evaluating all unsupported visual claims. As acknowledged in Section 4.1 and Appendix A.2, details absent from the reference are treated as neutral, and the absence of contradiction does not guarantee complete visual grounding. Although the judgments are human-verified, it remains unclear how reliably the evaluation captures unsupported but non-contradictory visual claims. This limits the interpretation of the reported hallucination reductions, particularly the 0.95% rate among correct answers, as a comprehensive measure of visual-evidence reliability.

  2. The benefits of human-corrected visual explanations are not sufficiently isolated from those of answer correction. Section 3.1.2 reports that 28.2% of examples involve answer changes, while 27.2% retain the answer but revise the explanation. However, the SFT ablation in Table 2 changes both answer labels and explanatory content simultaneously. Similarly, DPO preferences include both incorrect-answer responses and hallucinated explanations with correct answers. Consequently, the extent to which the observed improvements specifically arise from correcting visual evidence, rather than improved answer supervision, remains unclear.

Questions for authors
  1. Appendix A.2 treats claims absent from the reference conclusion as neutral rather than contradictory. Could the authors clarify whether such claims are directly verified against the original videos during human review? In particular, can the reported 0.95% hallucination rate capture unsupported visual claims that do not explicitly contradict the reference?

  2. Given the two correction types reported in Section 3.1.2, could the authors isolate the effects of answer correction and conclusion-only correction? For example, a comparison with answer-corrected-only supervision would help establish the additional benefit of human-corrected visual explanations.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 7r1E

Thank you for your detailed review and constructive feedback. We address your points below.


W1 and Q1: Direct-video validation of conclusions

Overview. We add a direct-video audit and compare its metrics with RCR (Appendix A.3, manuscript Table 4), clarifying the scope of the original 0.95% reference-contradiction rate (Abstract; Appendix A.2).

The original 0.95% is the reference-contradiction rate among correctly answered questions on the full 796-question test set. Human review verified reference consistency; reference-neutral claims were not systematically checked against videos.

We therefore conducted a direct-video audit on a shared, task-stratified random sample of 239/796 questions (~30%), balancing review workload with coverage of all eight tasks. Sampling was independent of correctness and RCR labels. We evaluated 2,629 original conclusions from 11 models. Two of three reviewers independently checked each video and its conclusions, with model identities hidden and references initially withheld. A conclusion was flagged if any visual claim contradicted or lacked support in the video, including reference-neutral claims. All three reviewers jointly resolved disagreements.

Table 1. Direct-video audit on the shared 239-question subset (%). Best in bold; second-best underlined.

ModelAcc. ↑RCR ↓VUR ↓VURc ↓VJA ↑
Qwen3-VL-8B-Instruct51.0554.3954.3915.5743.10
Qwen3.8-27B61.5139.3340.5910.20‾\underline{10.20}55.23
MiniCPM-V 4.558.1641.0043.5115.1149.37
Gemini 3.1 Pro76.57‾\underline{76.57}30.5432.2213.1166.53‾\underline{66.53}
Qwen3.8-Max64.8534.3136.4012.9056.49
Doubao-Seed-2.0-Pro73.2228.87‾\underline{28.87}29.29‾\underline{29.29}10.2965.69
Qwen-VL-Max56.9037.2438.0815.4448.12
Hulu-Med-32B51.0546.0351.8818.8541.42
Hulu-Med-30A341.0045.6146.8611.2236.40
Lingshu-32B49.7957.3257.7415.9741.84
Qwen-Colon-8B83.2618.8319.253.0280.75

On the audited subset, reference-based RCR and direct-video VUR have a mean absolute difference of 1.52 percentage points. For Qwen-Colon-8B, the rates are 18.83% and 19.25%, respectively, differing by 0.42 percentage points.

Under direct-video review, our model retains the lowest VUR (19.25%) and VURc (3.02%, 6/199), versus 29.29% and 10.20% for the best baselines, respectively, and the highest VJA (80.75%, versus 66.53%). These results support RCR's usefulness for model-level comparison and demonstrate that our model's advantage persists when all visual claims are assessed directly against the videos, including reference-neutral claims.


W2 and Q2: Separating answer and conclusion correction

Overview. We add matched full-split SFT controls to isolate answer and conclusion correction (Appendix B.1, manuscript Table 5).

We added SFT controls with identical training examples, initialization, budgets, checkpoint-selection rules, and the same 796 test questions. Q0/Q1 supervise only original/corrected answers. M0 uses original answers and conclusions; MA, MC, and MAC correct answers, conclusions, or both, respectively. Terminal option markers in MA/MC were adjusted for consistency with supervised answers.

Table 2. Separating answer correction and explanation correction (%). SFT-only comparisons. Best in bold; second-best underlined.

ConfigurationAcc. ↑Macro Acc. ↑RCR ↓CR_correct ↓Joint Acc. ↑
M0: SFT–Plus68.4757.0431.0310.0961.56
Q0: Original answers only63.9452.1842.2115.7253.89
Q1: Corrected answers only68.8457.3636.8112.7760.05
MA: Answer correction69.6058.0129.159.3963.07
MC: Conclusion correction71.98‾\underline{71.98}59.88‾\underline{59.88}24.12‾\underline{24.12}4.36‾\underline{4.36}68.84‾\underline{68.84}
MAC: SFT–Curated72.6160.3423.493.9869.72

The gains cannot be attributed to answer correction alone. MA provides the requested control with correct reference answers and original conclusions. MA and MAC share identical answer supervision and matched training settings, differing only in conclusion correction. MAC improves all five metrics over MA, including more than halving CR_correct (9.39%→3.98%). This shows that correcting answer labels alone does not account for the improvement in conclusion consistency.

Together with Q0→Q1, which establishes the benefit of answer correction, these controls support complementary contributions from answer correction and conclusion correction: curated conclusions provide useful supervision beyond correct answer labels, improving both answer correctness and conclusion consistency.


We hope these results address your concerns.

Dear Reviewer 7r1E,

As the rebuttal period comes to an end, we hope to make the best use of the remaining time for further discussion. We have addressed all comments and concerns raised in your review with utmost care, and hope that our rebuttal and revisions have met your expectations.

We look forward to further discussion. Thank you again for your time and effort!

Reviewer_gF3JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article compiled 7962 video Q&A (VQA) samples from three public datasets. It preserves the original interpretation of the model while using human intervention to correct unsupported descriptions in the video content, thereby improving the interpretation provided for VQA related to colonoscopy. The training process consists of two stages: first, supervised fine-tuning using correct answers (SFT); Secondly, contrastive feature learning (CFL) is applied to identify specific update layers, and then direct preference optimization (DPO) is performed on these selected modules. The final model achieved an accuracy rate of 79.15% in 796 test questions, with a conflict rate of only 0.95% between the provided explanations and the video content that correctly answered the questions.

Critical strengths
  1. This work preserves the mapping between the original interpretation and its corrected counterpart, including 2163 samples, where the answer remains unchanged and only the interpretation is corrected, which has greater research value than standard supervised fine-tuning (SFT) based solely on correct answers.
  2. Before performing training and testing segmentation, the author grouped the data based on source videos, clinical cases, and shared segments to ensure that adjacent segments did not cross the training and testing sets.
  3. Ablation studies, including original interpretation SFT, manually corrected interpretation SFT, full layer DPO, and DPO applied to specific (random or final) layers - validated the effectiveness of data correction and the impact of target updates on specific model layers.
  4. The author provided detailed information on the protocol between annotators, the number of cases with disagreements, and the clinical review process, which provided reasonable credibility for this work.
Critical weaknesses
  1. This article presents the results in the form of visual illusion rate, but the actual evaluation mainly assesses whether the generated explanation contradicts the reference explanation. Fabricated details not covered in the reference explanation can be classified as neutral. Therefore, a low conflict rate is not enough to prove that the video truly supports these explanations.
  2. In a section of the main text, responses containing "correct answers but incorrect explanations" are classified as negative samples of DPO, while elsewhere, the focus mainly shifts to negative samples containing incorrect answers. The final composition of preference pairs is not clearly defined.
  3. SFT Plus includes incorrect answers and explanations, while SFT Cured corrects both. Therefore, it is impossible to distinguish the respective contributions of correcting answers and correcting explanations.
  4. The text and case study illustrations in Figures 1 and 2 are too small. In addition, the DPO formula and its accompanying explanations are separated by graphics or tables across pages, which hinders readability.
Questions for authors
  1. Why is the abstract directly referred to as "visual illusion rate" when the actual measurement of the paper conflicts with the reference explanation?
  2. The benefits of manually correcting explanations are not separated from the benefits of correcting incorrect answers. What would be the result if we only correct the answer and maintain the original explanation? What if we only train for correct answers without providing explanations? Can training on samples with 'originally correct answers, only corrected explanations' independently improve the quality of explanations?
  3. Does CFL recognize answer sensitive modules or visual evidence processing modules?
  4. The description of DPO data is inconsistent, and it is necessary to confirm what was actually used for training.
  5. It is currently difficult to determine whether the method's returns can exceed this dataset. The 15 baselines are mainly models that have not been adapted to this dataset. This comparison can demonstrate the value of specialized training, but it is not sufficient to prove that CFL is a better solution than existing adaptation methods.
6 comments
Reviewer_gF3JOpenReview ↗

Guidelines on the Use of AI Tools for Manuscript Review

This review process utilized AI to translate and organize the core content of the manuscript. Subsequently, human reviewers conducted a thorough assessment and finalized the review results. Human oversight ensured the overall quality and formatting of the manuscript, and the final conclusions were determined by human judgment rather than by AI.

AuthorsOpenReview ↗

Response to Reviewer gF3J

Thank you for your careful review and valuable constructive feedback. We have carefully considered your comments and provide our detailed responses to each point below.


W1 and Q1: Direct-video validation of conclusions

Overview. We add a direct-video audit of conclusions (Appendix A.3, manuscript Table 4) and clarify the scope of the original reference-contradiction metric (Abstract; Appendix A.2).

We agree that “hallucination rate” was broader than the metric measured. The original 0.95% measures reference contradictions among correctly answered questions on the full test set, and does not capture all unsupported visual claims.

We additionally audited original conclusions from 11 models on a shared, task-stratified random 239-question subset (~30%). Two of three reviewers independently checked each video and its conclusions, with model identities hidden and references initially withheld. Any unsupported or video-contradictory visual claim was flagged; all three reviewers jointly resolved disagreements.

Table 1. Direct-video audit on the shared subset (%). Best bold; second-best underlined.

ModelAcc. ↑RCR ↓VUR ↓VURc ↓VJA ↑
Qwen3-VL-8B-Instruct51.0554.3954.3915.5743.10
Qwen3.8-27B61.5139.3340.5910.20‾\underline{10.20}55.23
MiniCPM-V 4.558.1641.0043.5115.1149.37
Gemini 3.1 Pro76.57‾\underline{76.57}30.5432.2213.1166.53‾\underline{66.53}
Qwen3.8-Max64.8534.3136.4012.9056.49
Doubao-Seed-2.0-Pro73.2228.87‾\underline{28.87}29.29‾\underline{29.29}10.2965.69
Qwen-VL-Max56.9037.2438.0815.4448.12
Hulu-Med-32B51.0546.0351.8818.8541.42
Hulu-Med-30A341.0045.6146.8611.2236.40
Lingshu-32B49.7957.3257.7415.9741.84
Qwen-Colon-8B83.2618.8319.253.0280.75

Our direct-video audit checks visual claims against the videos, including reference-neutral claims. On the shared 239-question subset, Qwen-Colon-8B achieves the lowest VUR (19.25%) and VURc (3.02%, 6/199), versus 29.29% and 10.20% for the respective best baselines. It also achieves the highest VJA (80.75% versus 66.53%), indicating that its advantage extends to correct answers accompanied by conclusions without flagged unsupported visual claims.

These results show that our model’s advantage persists under direct-video assessment, supporting improved visual grounding on the audited subset beyond reference consistency.

W3 and Q2: Separating answer and conclusion correction

Overview. To address these concerns, we:

- 1. Add matched full-split SFT controls to separate answer correction from conclusion correction (Appendix B.1, manuscript Table 5); - 2. Add a conclusion-only correction comparison on examples whose original answers were already correct (Appendix B.2, manuscript Table 6).

1. Matched full-split SFT controls (Appendix B.1, manuscript Table 5).

We added SFT controls to separate answer correction from conclusion correction. M0 uses original answers and conclusions; MA corrects answers only, MC corrects conclusions only, and MAC corrects both. Q1 uses correct answers without conclusions. Terminal option markers in MA/MC are aligned with their supervised answers.

We also compare CE–Original and CE–Corrected on the same training-split subset where the original answers were already correct and only the conclusions required correction. Within each comparison, training examples, initialization, training budgets, and checkpoint-selection rules are matched. All models are evaluated on the same 796 test questions.

Table 2. Answer and conclusion supervision controls (%). All comparisons are SFT-only. Best in bold; second-best underlined.

ConfigurationAcc. ↑Macro Acc. ↑RCR ↓CR_correct ↓Joint Acc. ↑
M0: Original answers + original conclusions68.4757.0431.0310.0961.56
Q1: Correct answers only68.8457.3636.8112.7760.05
MA: Correct answers + original conclusions69.6058.0129.159.3963.07
MC: Original answers + corrected conclusions71.98‾\underline{71.98}59.88‾\underline{59.88}24.12‾\underline{24.12}4.36‾\underline{4.36}68.84‾\underline{68.84}
MAC: Correct answers + corrected conclusions72.6160.3423.493.9869.72
CE–Original (subset)66.9655.7236.3111.8259.05
CE–Corrected (subset)69.2258.1625.885.9965.08

The gains cannot be attributed to answer correction alone. MA→MAC holds correct-answer supervision fixed: correcting conclusions improves accuracy from 69.60% to 72.61% and reduces CR_correct from 9.39% to 3.98%. MAC also outperforms correct-answer-only supervision (Q1), which achieves 68.84% accuracy and 12.77% CR_correct.

Conclusion correction also improves performance without correcting the original answer labels. M0→MC retains those labels while correcting conclusion content, increasing accuracy from 68.47% to 71.98% and joint accuracy from 61.56% to 68.84%, while reducing CR_correct from 10.09% to 4.36%.

2. Conclusion-only correction on a matched subset (Appendix B.2, manuscript Table 6).

The CE comparison further isolates this effect on examples whose answers were already correct. With identical answer labels, correcting conclusions reduces RCR from 36.31% to 25.88% and CR_correct from 11.82% to 5.99%, while improving accuracy from 66.96% to 69.22%.

Together, these controls support a distinct contribution from conclusion correction: it improves both answer accuracy and conclusion consistency beyond answer-label correction, with all gains observed at the SFT stage before DPO.


Q3: Answer sensitivity and visual-evidence processing in CFL

Overview. We clarify CFL's targeted visual-pathway interventions and development-set screening procedure in Section 3.2.2 and Appendix C.

CFL links visual processing to correct answering through targeted interventions, using answer probability to measure the task relevance of each intervened pathway. Its probes cover visual-information processing across language, vision, and merger modules:

  • Language attention: exclude visual-token K/V entries during a single cached answer-prefix query, preserving the prefill cache and nonvisual K/V entries.
  • Language MLPs: zero outputs only at visual-token positions during prefill.
  • Visual attention and mergers: zero selected branch or merger outputs during fresh visual forward passes.

With model parameters fixed, CFL measures the decrease in correct-answer log probability, using probabilities normalized over valid answer options. Scores are averaged equally across tasks, giving each task equal weight in module ranking.

These rankings prioritize candidate update sites, and development-set DPO screening evaluates the adaptation utility of their combinations. CFL therefore combines two complementary signals: how disrupting visual processing affects answering, and how adapting the selected modules improves task performance. This connects visual-pathway sensitivity with empirical adaptation utility for colonoscopy video QA.

Reviewer_FmEzOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies visual hallucinations in colonoscopy video question answering, where models may provide correct answers supported by incorrect or unsupported explanations. The authors introduce ColonExplainVQA, a dataset containing 7,962 video-question pairs across nine tasks, with human-reviewed corrections to model-generated explanations. They further propose Qwen-Colon-8B, which combines supervised fine-tuning on curated responses with Colon Function Layer (CFL)-guided direct preference optimization to selectively update task-relevant modules. On the proposed benchmark, the model achieves 79.15% answer accuracy, outperforming 15 evaluated baselines, while reducing the reference contradiction rate among correct answers to 0.95%. The paper aims to improve both answer accuracy and the faithfulness of supporting explanations in medical video understanding.

Critical strengths
  1. A useful resource for studying errors in explanations. ColonExplainVQA contains 7,962 video-question pairs with original and reviewed explanations, including 2,163 cases where the answer remains correct but the explanation requires correction. Independent annotation, clinical adjudication of difficult cases, and additional verification of the test set are valuable quality controls. Extending paired corrections to video QA is a useful development beyond image-based gastrointestinal resources such as Gut-VLM [1].

  2. Meaningful improvements on standard accuracy, with sensible split controls. Qwen-Colon-8B achieves 79.15% answer accuracy, compared with 72.61% for curated SFT and 68.09% for the strongest evaluated proprietary baseline. Task-macro accuracy also improves. The source-group splits and exclusion of test data from model selection strengthen these results. Thus, the paper provides evidence of useful domain adaptation beyond improvement on its proposed explanation metrics alone.

[1] Khanal, B., et al. Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models. MICCAI, 2025.

Critical weaknesses
  1. The evaluation does not fully establish visual faithfulness. Appendix A.2 evaluates explicit contradictions between generated and curated text; details absent from the reference are treated as neutral, and the automated judge does not receive the video. Human verification improves the reliability of these judgments, but the protocol does not clearly establish an independent visual audit of all generated claims. Consequently, the headline 0.95% supports a low reference-contradiction rate among correct answers; broader hallucination and grounding claims require additional evidence. Evaluation also remains within the constructed benchmark, without external transfer tests or missing/shuffled-video controls.

  2. The contribution is not sufficiently isolated by the comparisons. Learning from human corrections and selecting functionally relevant layers build on established approaches such as RLHF-V and VFL [2,3]. The SFT comparison changes both answer labels and explanations, so it cannot isolate the benefit of correcting visual evidence. CFL also changes the adapted module families relative to vanilla DPO, while parameter and tuning-budget matching for random/last-layer controls is unclear. All runs use seed 42, and the main results lack confidence intervals or variation across seeds; Figure 5's intervals concern sensitivity scores only. Missing decoding and per-model input details further limit reproducibility and assessment of comparison fairness.

[2] Yu, T., et al. RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback. CVPR, 2024. Paper.

[3] Shi, C., Yu, Y., and Yang, S. Vision Function Layer in Multimodal LLMs. NeurIPS, 2025. Paper.

Questions for authors
  1. Validity of the explanations: Could blinded reviewers assess all claims in a representative sample directly against the videos, including claims labeled neutral? Please also clarify Figure 11's curated reference, which infers low-grade adenoma despite stating that no distinct polyp is visible. How are source-provided pathology labels separated from conclusions supported by the clip?

  2. Benefit of correcting evidence: What happens when both SFT conditions use correct reference answers and differ only in whether explanations are corrected? An answer-only baseline and an ablation on conclusion-only corrections would help identify the contribution of the proposed supervision.

  3. Controlled comparisons and uncertainty: Do CFL's gains persist when random/last-layer controls match its module families, trainable parameters, training steps, and selection budget? Please report multiple training seeds and paired confidence intervals that account for source-group dependence. The reported 0.95% corresponds to approximately six contradictions among 630 correct answers, making uncertainty particularly relevant.

  4. Generation protocol: What are the exact prompts, model snapshots, temperature, top-p, greedy versus sampled decoding, output-token limits, reasoning budgets, and number of generations per question? Training frame settings are reported, but what frame-sampling and resolution budgets did each baseline receive, particularly the medical and proprietary models?

  5. Generalization and frontier comparisons: Do the gains transfer to an independent dataset or new question formulations? Could the authors add representative comparisons with GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon, or Kimi K3, taking model availability at the experimental cutoff into account? These comparisons should use clearly documented visual-input and inference budgets.

5 comments
AuthorsOpenReview ↗

Response to Reviewer FmEz

Thank you for your thoughtful review and valuable suggestions. We address each point below. New results and clarifications are included in the revised manuscript and highlighted in blue.


W1 and Q1: Direct-video validation of conclusions and clarification of the pathology example

Overview. To address these concerns, we:

- 1. Clarify the interpretation of the original 0.95% reference-contradiction rate (Abstract; Appendix A.2); - 2. Add the direct-video audit protocol and report the shared-subset results and video-based metrics (Appendix A.3, manuscript Table 4); - 3. Remove the disputed pathology-correction image and its caption (Appendix D.2).

1. Interpretation of the original metric (Abstract; Appendix A.2; direct-video audit in Appendix A.3).

The original 0.95% measures reference contradictions among correct answers on the full 796-question test set. We will use “reference-contradiction rate among correct answers” in the abstract and add the direct-video audit protocol and results to Appendix A.2.

2. Direct-video audit protocol and validation results (Appendix A.3, manuscript Table 4).

We conducted a direct-video audit on a shared, task-stratified random sample of 239/796 test questions (~30%), covering all eight test tasks and 2,629 original conclusions from 11 models. Sampling was independent of model correctness and reference-contradiction labels.

Each video and its conclusions were independently assessed by two of three reviewers, with model identities hidden and reference conclusions initially withheld. Reviewers checked all visual claims, including reference-neutral claims. A conclusion was flagged if any visual claim contradicted or lacked support in the video. All three reviewers jointly resolved disagreements.

Table 1. Direct-video audit on the shared subset (%). Best in bold; second-best underlined.

ModelAcc. ↑RCR ↓VUR ↓VURc ↓VJA ↑
Qwen3-VL-8B-Instruct51.0554.3954.3915.5743.10
Qwen3.8-27B61.5139.3340.5910.20‾\underline{10.20}55.23
MiniCPM-V 4.558.1641.0043.5115.1149.37
Gemini 3.1 Pro76.57‾\underline{76.57}30.5432.2213.1166.53‾\underline{66.53}
Qwen3.8-Max64.8534.3136.4012.9056.49
Doubao-Seed-2.0-Pro73.2228.87‾\underline{28.87}29.29‾\underline{29.29}10.2965.69
Qwen-VL-Max56.9037.2438.0815.4448.12
Hulu-Med-32B51.0546.0351.8818.8541.42
Hulu-Med-30A341.0045.6146.8611.2236.40
Lingshu-32B49.7957.3257.7415.9741.84
Qwen-Colon-8B83.2618.8319.253.0280.75

RCR retains the original reference-based labels on this subset. VUR measures conclusions containing at least one video-unsupported visual claim; VURc conditions this rate on correct answers. VJA requires a correct answer and no flagged unsupported visual claim.

Qwen-Colon-8B achieves the lowest VUR (19.25%) and VURc (3.02%, 6/199), versus 29.29% and 10.20% for the respective best baselines. It also achieves the highest VJA (80.75% versus 66.53%).

Our model’s advantage therefore persists when visual claims are assessed directly against the videos, including reference-neutral claims, supporting improved visual grounding on the audited subset beyond reference consistency.

3. Disputed example and its removal (Appendix D.2).

The “low-grade adenoma” label follows the source-provided annotation. The apparent mismatch may reflect incomplete coverage of the source video in the displayed example. We will temporarily remove this example from Figure 11

W2 and Q2: Separating answer and conclusion correction

Overview. To address these concerns, we:

- 1. Add full-split SFT controls with correct answer labels held fixed (Appendix B.1, manuscript Table 5); - 2. Add a matched conclusion-only correction ablation (Appendix B.2, manuscript Table 6).

1. Matched full-split SFT controls (Appendix B.1, manuscript Table 5).

We added matched SFT comparisons with identical correct reference answers, an answer-only baseline, and an ablation restricted to conclusion-only corrections. Within each comparison, training examples, initialization, training budgets, and checkpoint-selection rules are matched. All models are evaluated on the same 796 test questions.

Table 2. Answer and conclusion supervision controls (%). Bold highlights the corrected-conclusion condition within each matched pair.

ConfigurationAcc. ↑RCR ↓CR_correct ↓Joint Acc. ↑
Q1: Correct answers only68.8436.8112.7760.05
MA: Correct answers + original conclusions69.6029.159.3963.07
MAC: Correct answers + corrected conclusions72.6123.493.9869.72
CE–Original (conclusion-only subset)66.9636.3111.8259.05
CE–Corrected (conclusion-only subset)69.2225.885.9965.08

Q1, MA, and MAC use the full training split. In MA, terminal option markers are aligned with the supervised answers; the original visual claims are otherwise retained.

Correcting conclusions improves performance with correct-answer supervision held fixed. MA→MAC increases accuracy from 69.60% to 72.61% and joint accuracy from 63.07% to 69.72%, while reducing CR_correct from 9.39% to 3.98%. MAC also outperforms the correct-answer-only baseline, Q1, which achieves 68.84% accuracy and 60.05% joint accuracy.

2. Conclusion-only correction on a matched subset (Appendix B.2, manuscript Table 6).

The CE comparison isolates conclusion correction on examples whose original answers were already correct. Both conditions use identical examples and answer labels, differing only in whether conclusions are corrected. CE–Corrected improves accuracy from 66.96% to 69.22%, reduces RCR from 36.31% to 25.88%, and reduces CR_correct from 11.82% to 5.99%.

Together, these controls show that conclusion correction provides useful supervision beyond correct answer labels: it improves both answer accuracy and conclusion consistency, including when the original answers require no correction. All gains are observed at the SFT stage, before DPO.

W2 and Q3: Controlled comparisons and uncertainty

Overview. To address these concerns, we:

- 1. Add five-seed, family-matched DPO comparisons (Appendix G.3, manuscript Table 12); - 2. Report paired source-group bootstrap confidence intervals (Appendix G.3, manuscript Table 13).

1. Five-seed family-matched DPO comparisons (Appendix G.3, manuscript Table 12).

We evaluated family-matched Random-DPO, Last-DPO, and CFL-DPO over five training seeds. Each strategy updates three visual-attention layers and 18 language-MLP layers, with approximately 14.49M trainable parameters and 300 DPO steps. All use the same SFT initialization, frozen reference model, preference data, and validation rules.

Table 3. Matched DPO comparisons on ColonExplainVQA (%, mean ± SD across five training seeds). Best means in bold.

MethodAnswer Acc. ↑Macro Acc. ↑RCR ↓CR_correct ↓Joint Acc. ↑
Family-matched Random-DPO76.13 ± 0.4664.82 ± 0.4222.04 ± 0.703.63 ± 0.2873.37 ± 0.66
Family-matched Last-DPO76.66 ± 0.3665.28 ± 0.3421.41 ± 0.343.25 ± 0.2374.17 ± 0.52
CFL-DPO79.10 ± 0.3468.07 ± 0.4415.63 ± 0.541.02 ± 0.1978.29 ± 0.48

CFL-DPO achieves the best mean performance across all five metrics. Relative to Random-DPO and Last-DPO, it improves answer accuracy by 2.97 and 2.44 percentage points, respectively.

2. Paired source-group confidence intervals (Appendix G.3, manuscript Table 13).

The 796 test questions span 38 source groups: 24 from Hyper-Kvasir (387 questions), seven from REAL-Colon (379), and seven from SUN-SEG (30). We performed 20,000 paired bootstrap replicates, resampling 24, seven, and seven groups with replacement within the respective data sources and retaining all questions from each sampled group. All methods use identical group and training-seed-block resamples.

For each replicate, we recompute Joint Acc., CR_correct, and their paired differences. CR_correct uses each method’s own correct-answer denominator in that replicate. The 95% confidence intervals are the 2.5th and 97.5th percentiles of the bootstrap differences.

Table 4. Paired differences and source-group bootstrap 95% CIs (percentage points). Differences are CFL minus the corresponding control.

ComparisonDifference95% CI
ΔJoint Acc.: CFL − Random+4.92[3.42, 6.92]
ΔJoint Acc.: CFL − Last+4.12[2.95, 5.71]
ΔCR_correct: CFL − Random−2.61[−4.19, −1.16]
ΔCR_correct: CFL − Last−2.23[−3.54, −1.01]

All four intervals favor CFL and exclude zero. The original 0.95% (6/630) describes one run; the five-seed mean is 1.02 ± 0.19%. The paired comparisons assess joint accuracy and conditional reference contradictions.

CFL’s gains persist under matched module families, parameter budgets, and optimization steps. The five-seed results and paired confidence intervals accounting for source-group dependence support improved answer correctness and conclusion consistency relative to both random and last-layer selection.


W2 and Q4: Generation protocol and visual-input budgets

Overview. We provide the evaluation settings, shared generation instruction, and retry policy in Appendix A.1 (manuscript Table 3).

All answering models, including medical and proprietary baselines, receive the same questions and answer options. Frames are sampled across the full clip at 12 fps, with an input resolution of 448×448 pixels and a configured limit of 16,384 visual tokens. Reference answers and curated conclusions are withheld.

SettingEvaluation configuration
Temporal samplingFull-clip coverage at 12 fps
Input frame resolution448×448 pixels
Visual-token limit16,384
Initial generationsOne per question
Maximum attemptsFive total, including the initial generation
Retry conditionsRequest errors or missing, truncated, or invalid outputs
Local decodingGreedy
Temperature and top-pNot explicitly specified
API decoding controlsProvider defaults

Shared generation instruction, verbatim:

Watch the provided video clip or clips and answer the multiple-choice question. First provide a concise visual conclusion in 1-3 English sentences describing observations that support your answer. Describe only visible evidence and preserve uncertainty when it is unclear. Do not provide a step-by-step reasoning process. After the conclusion, select exactly one listed option. Both fields are required. Use this order: Conclusion: <visual evidence> then Answer: <option letter>.

This protocol standardizes the supplied visual inputs and response format across models. One generation is requested initially; additional attempts address the failures listed above. We do not use sampling-based best-of-k selection.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of changes in the revised PDF

Summary of changes in the revised PDF

We thank all reviewers for their detailed and constructive feedback. We have strengthened the revised manuscript with the following additional experiments and evaluations. New and revised material is highlighted in blue.

  • Direct-video validation of conclusions [FmEz, gF3J, 7r1E]. We add a blinded audit of 2,629 original conclusions from 11 models on a shared, task-stratified sample of 239 test questions. The audit checks visual claims directly against videos, including reference-neutral claims, and reports video-based metrics alongside reference consistency (Appendix A.3; Table 4).

  • Controlled evaluation of answer and conclusion correction [FmEz, gF3J, 7r1E]. We add matched SFT controls separating answer-label correction from conclusion correction, including answer-only supervision and comparisons with correct answer labels held fixed. A separate matched-subset experiment isolates conclusion correction when the original answers were already correct (Appendices B.1–B.2; Tables 5–6).

  • Matched adaptation comparisons and second-backbone validation [gF3J]. We compare CFL-DPO with Uniform LoRA-DPO, DoRA-DPO, and AdaLoRA-DPO under matched budgets. We also evaluate adapted general-purpose and medical-domain models and extend CFL-DPO to InternVL3.5-8B, testing its utility beyond the Qwen backbone (Appendices G.1–G.2; Tables 10–11).

  • Multi-seed evaluation and statistical uncertainty [FmEz]. We add five-seed comparisons of Random-DPO, Last-DPO, and CFL-DPO with matched module families, parameter budgets, and optimization steps. We report paired source-group bootstrap confidence intervals for differences in joint accuracy and conditional reference contradictions (Appendix G.3; Tables 12–13).

  • Cross-benchmark evaluation [FmEz, gF3J]. We evaluate the fixed Qwen-Colon-8B checkpoint on 1,485 Colon-Bench Prompted VQA questions without benchmark-specific fine-tuning, extending the evaluation beyond ColonExplainVQA (Appendix E; Table 8).

  • Additional frontier-model comparisons [FmEz]. We evaluate GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon, and Kimi K3 on the original 796-question ColonExplainVQA test set, expanding the comparison to 19 baselines across answer accuracy and reference-consistency metrics (Appendix F; Table 9).

AuthorsOpenReview ↗

Summary of Discussion

We thank the Area Chair and all three reviewers for their time and constructive feedback. The discussion sharpened two central questions about ColonExplainVQA: how often a correct answer contains a claim unsupported by the video, and whether correcting the explanation helps when the answer label is already correct.

Score updates

  • 7r1E: 2 (weak rejection) → 3 (weak acceptance)
  • FmEz: 2 (weak rejection) → pending reviewer update.
  • gF3J: 2 (weak rejection) → pending reviewer update.

Strengths recognized by the reviewers

  • A benchmark that separates answer correctness from explanation quality. All three reviewers recognized the value of preserving original and expert-corrected explanations. In 2,163 examples, the answer stays correct while the explanation requires correction. [FmEz, gF3J, 7r1E]
  • Annotation quality and evaluation design. Reviewers highlighted the independent annotations, clinical adjudication, and verification of the test set. They also recognized the use of source-group splits to keep related clips out of different partitions. [FmEz, gF3J, 7r1E for annotation quality; FmEz, gF3J for split design]
  • Empirical gains. Reviewers noted improvements in answer accuracy and found the SFT and DPO ablations informative about the roles of corrected supervision and selective adaptation. [FmEz, gF3J, 7r1E]

Concerns acknowledged as resolved during discussion

Reviewer 7r1E explicitly confirmed that the direct-video audit and matched SFT controls addressed their two main concerns. We also incorporated their follow-up suggestions on metric definitions and highlighted the relevant appendix revisions.

  • Direct verification of visual claims. A blinded audit checked 2,629 conclusions from 11 models against the videos for the same 239 questions, including claims absent from the curated references. We clarified that the original 0.95% measures explicit reference contradictions among correct answers on the full test set. Under direct video review, 3.02% (6/199) of Qwen-Colon-8B’s correct answers on the audited subset contained an unsupported visual claim, compared with 10.20% for the best baseline. (Section 4.1, Appendix A.3, Table 4)
  • Contribution of correcting explanations. On a matched training subset where the original answers were already correct, changing only the conclusions improved test accuracy from 66.96% to 69.22% and reduced reference contradictions among correct answers from 11.82% to 5.99%. This directly addresses whether explanation correction adds value with answer labels held fixed. (Appendices B.1–B.2, Tables 5–6)

Additional revisions in response to reviewer concerns

  • Adaptation controls and statistical uncertainty. We added budget-matched comparisons with Uniform LoRA-DPO, DoRA-DPO, and AdaLoRA-DPO, and tested CFL-DPO on a second backbone, InternVL3.5-8B. Across five training seeds, CFL-DPO also outperformed random- and last-layer selection on joint accuracy and reference contradictions among correct answers. All four paired, source-group bootstrap confidence intervals favored CFL-DPO and excluded zero. (Appendices G.1–G.3, Tables 10–13) [FmEz, gF3J]
  • Evaluation beyond ColonExplainVQA. Without further fine-tuning, the fixed Qwen-Colon-8B checkpoint achieved 80.20% accuracy on 1,485 Colon-Bench Prompted VQA questions. We also expanded the original test-set comparison to 19 baselines. (Appendices E–F, Tables 8–9) [FmEz, gF3J]
  • Preference-pair construction. We clarified how DPO uses examples whose original answer is correct but whose conclusion contains an erroneous or unsupported visual claim. In these pairs, the answer stays fixed and the preference signal comes from the corrected conclusion. (Section 3.2.3, Appendix H) [gF3J]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.