acceptodds
Withdrawn from ICLR 2027

Corruption-Risk Reversal in Adaptive Visual Evidence Routing for Medical Vision–Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Adaptive visual computation is usually motivated as an efficiency problem: additional visual tokens are spent only when a global view is insufficient. For an autoregressive vision–language model (VLM), however, an acquired crop is not simply more information. It changes both the visible evidence and the generation context, and can therefore overturn a correct global answer. This reliability problem is formulated as cost-sensitive evidence routing among answering, acquiring a view, and abstaining. The formulation separates the availability of a better action from the router's ability to select it, and decomposes acquisition value into helpful corrections and harmful corruptions. Empirically, every policy is reconstructed from immutable paired counterfactual outputs with a shared global response, exact-image grouping, revision-pinned models, and policies frozen before transfer. On historically exposed M3CoTBench development, signed-utility routing raises MedGemma-4B accuracy from 24.55% to 30.91% (; 95% image-group bootstrap CI ). Under locked PathVQA shift, the same rule lowers accuracy from 70.31% to 63.67% (; CI ). A separately frozen confidence-ranked plurality policy likewise lowers Qwen accuracy on 2,996 prospective 3MDBench cases from 40.92% to 39.05%, correcting 16 answers and corrupting 72. On DDI, a conservative unanimity gate avoids added errors only by making no replacements, while global accuracy conceals 0.58% malignant sensitivity. Across these evaluations, the observed pattern is termed corruption-risk reversal: uncertainty continues to identify fragile cases, but ceases to distinguish acquisitions that repair an answer from those that damage it. These results establish neither a universal failure of cropping nor clinical safety. They instead show that efficiency claims alone are insufficient without action headroom, transferable harm estimation, safe aggregation, and nontrivial coverage.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Medical vision-language models, adaptive evidence acquisition, selective prediction and abstention, uncertainty estimation, distribution shift, multimodal reasoning, cost-sensitive decision-making, AI safety and reliability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_pHe2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper explores adaptive visual computation in autoregressive medical VLMs. They challenge the assumption that an acquired crop is not simply more information, rather it changes both the visible evidence and the generation context, which can change the answer. With paired counterfactual design and evaluations across 4 datasets, they identify a failure mode named corruption-risk reversal.

Critical strengths

None. Very confusing, hard-to-read, and oddly written paper. Unfortunately, I didn’t find anything that stands out to me.

Critical weaknesses

Some critical weaknesses in my eye:

  1. The paper is overly reliant on rigid and naive action spaces. The primary interventions depend on fixed spatial locations or simple CLIP-ranked windows. So, these observed corruptions can be the result of forcing an autoregressive VLM to answer arbitrary, context-less image patches; rather than some fundamental flaw in adaptive computing as described. I think, more sophisticated methods or more semantically grounded approaches with dynamic cropping can avoid these. So, now, overall, it weakens the motivation and confuses the contributions. In the same way, I feel the external validity is too narrow as per the framing (adaptive visual computation) used in the paper.
  2. The paper doesn’t provide any working solution either. Their proposed router fails under transfer; 3MDBench policy seems harmful, and DDI fate achieves some corruption control only by making zero replacements. Proposition 2 gives a bound only if uniform continuation-value error is established, which is not the case here.
  3. The paper says they have provided code, all data and all for full reproducibility. But, nothing is given.

Also, I must say, most of the work reads and sounds heavily AI-generated (not assisted). It is unnecessarily verbose, hard to read, and nothing is clear. Seems like a ghost problem (not well-established) with a ghost solution (not well-established) without any proper connection/use.

Questions for authors
  1. Firstly, what are you trying to do in this paper? Let’s be real; please clarify what is the problem, why it is important, how it works, and what are you trying to do in this paper.
  2. Following Critical Weakness 1, I think, it is important to see what happens if we expand the action family to adaptive zooming, multi-scale cropping, tool calling. Does the corruption-risk reversal effect stays or it goes away? Is it a result of the rigid cropping system used?
  3. Why is this problem important? I didn’t see a proper research gap, establishing the problem and its importance and connection to previous works? What does the paper bring to the table, in terms of contribution?
  4. Then, about the critical weakness 2, as the solution is not robust as we can see, I think it’s important to explore if a harm-aware router with lower confidence can be bound on signed utility? Can we do some target-support test to check if we can achieve non-trivial coverage under PathVQA/#MDBench/DDI; and if so, at what target risk? Can we train a router specifically to get a transferable harm calculator, using some domain-adversarial or contrastive learning techniques?
  5. Coming back to corruption-risk reversal, how would you formally test it against simple explanations, such as, calibration failure or general negative transfer (such as pre-specified AUC/based-rate contrasts with CIs)? Will it hold strongly enough to establish the problem?
  6. Can we trace exactly why these autoregressive VLMs corrupt a previously correct answer? Specially, how do the attentions sink or the tokens compete or KV-cache dynamics shift when such a crop occurs (in mech-interp sense)?
  7. Can you provide all the things you claim that you provided? (reproducibility)
  8. Explore and all more related works, such as ReCoVERR [1].
  9. Can you please connect the findings to something useful and meaningful? Also, can you please improve overall writing and presentation of the work?
  • [1] Selective “Selective Prediction”: Reducing Unnecessary Abstention in Vision-Language Reasoning (Srinivasan et al., Findings 2024)
Reviewer_xkL4OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how additional visual acquisitions can correct or corrupt medical VLM answers and whether these outcomes remain predictable under distribution shift. It develops a paired evaluation protocol and reports negative transfer, harmful multi-view aggregation, and limited replacement coverage across several medical benchmarks.

Critical strengths
  1. The paper studies a clear question: whether a crop can improve or corrupt an answer.
Critical weaknesses
  1. The presentation is hard to follow. Terms such as "corruption-risk reversal," "action headroom," and "aggregation regret" do not have clear definitions.
  2. The main reversal result is not consistent across models. MedGamma improves on M3 but drops on PathVQA.
  3. An ablation study is missing: the proposed approaches combine several components, but the paper does not show which component is actually helpful.
Questions for authors
  1. In Proposition 1, the paper states: "This elementary identity motivates the estimator but is not itself a novel result." Does this mean that the novelty is limited?
Reviewer_KncKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at adaptive visual computation (dynamic cropping and zooming) in medical VLMs from a reliability standpoint. The core premise is that giving a VLM cropped sub-regions isn't purely additive, it shifts token generation context and often corrupts answers that were already correctly solved from the global view. I read the paper with great interest. The methodology is very rigorous, though I have some reservations about the experimental claims.

The authors frame this as a cost-sensitive routing problem, decomposing net performance into helpful recoveries versus harmful corruptions. Using a locked audit setup with cached model calls, they evaluate policies across M3CoTBench, PathVQA, and a prospective 3MDBench split. While a simple logistic router gives MedGemma a +6.36 pp bump on M3CoTBench, the exact same frozen policy tanks accuracy on PathVQA by -6.64 pp (5 helps vs 22 corruptions). Similarly, on 3MDBench, voting across quadrant views consistently lowers Qwen's accuracy. The authors dub this "corruption-risk reversal": model uncertainty flags fragile predictions, but fails to predict whether acquiring closer views will actually fix or break the answer under distribution shift.

Critical strengths

(1) The evaluation rigor is noticeably higher than most adaptive inference papers. Usually these papers tune thresholds dynamically or report aggregate metrics that gloss over how many correct predictions got flipped to incorrect. Here, tracking exact counts of helps vs harms, locking the manifests with SHA-256 hashes, and enforcing a strict wall between exploratory tuning and transfer evaluation makes the empirical findings much more trustworthy.

(2) It provides a timely reality check on the "more resolution is always better" assumption. Showing that naive multi-crop consensus actually harms performance on 3MDBench (dropping accuracy by up to 3.97 pp under 5-view voting), and highlighting that harm-ranking AUC drops to 0.25 on PathVQA, is an important caution for people trying to push adaptive VLMs straight into clinical deployment.

Critical weaknesses

(1) I'm not convinced the "corruption-risk reversal" isn't largely an artifact of an extreme shift in task format. The router is trained on M3CoTBench (complex multi-choice questions with rationales) and then evaluated on PathVQA, which is mostly binary Yes/No questions with high baseline accuracy (~70%). In a binary setting, any perturbation has a high chance of flipping a right answer to wrong. It feels like the surge in corruptions (22 harms vs 5 helps) could just be an out-of-domain linear classifier failing on a completely different label prior, rather than a deep, fundamental property of visual evidence routing.

(2) Chopping an image into rigid 2x2 quadrants makes little clinical sense, especially for histopathology. If a crop bisects a cell cluster or lands entirely on empty slide background/stroma, the VLM is basically forced to guess based on language priors alone. Without testing a more clinically sensible region proposal method (or at least an attention-guided crop), it's hard to tell if the high corruption rate is inherent to adaptive zoom or just a byproduct of feeding the model uninformative slices.

(3) The router parametrization is quite weak. It's essentially a multinomial logistic regression built on token log-probs, response lengths, and question n-grams. That a linear model on surface-level statistics fails to rank harm under cross-dataset shift is almost expected. Claiming this as a broad conceptual failure of uncertainty-based routing might be overstating what is really just the poor out-of-distribution transfer of a basic linear model.

Questions for authors

(1) How much does the binary nature of PathVQA drive the 22 corruptions? If you fit or tune the confidence threshold directly on a PathVQA validation split (in-domain), does the harm AUC still collapse below random chance, or is this purely a cross-distribution transfer issue?

(2) For those 22 cases where PathVQA global answers were broken by cropping, what did the crops actually contain? Were they centered on diagnostic lesions, or did the model get sent empty slide backgrounds or split tissue?

(3) On 3MDBench, multi-view voting reduced accuracy. Did you experiment with putting both the global image and the cropped regions into the context window together in one prompt, instead of decoding them separately and voting across outputs?

(4) Did you try training a slightly more capable router (e.g. an MLP over pooled vision-language embeddings) rather than n-grams and length heuristics? Does feature representation help stabilize the harm ranking across datasets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.