- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper explores adaptive visual computation in autoregressive medical VLMs. They challenge the assumption that an acquired crop is not simply more information, rather it changes both the visible evidence and the generation context, which can change the answer. With paired counterfactual design and evaluations across 4 datasets, they identify a failure mode named corruption-risk reversal.
Critical strengths
None. Very confusing, hard-to-read, and oddly written paper. Unfortunately, I didn’t find anything that stands out to me.
Critical weaknesses
Some critical weaknesses in my eye:
- The paper is overly reliant on rigid and naive action spaces. The primary interventions depend on fixed spatial locations or simple CLIP-ranked windows. So, these observed corruptions can be the result of forcing an autoregressive VLM to answer arbitrary, context-less image patches; rather than some fundamental flaw in adaptive computing as described. I think, more sophisticated methods or more semantically grounded approaches with dynamic cropping can avoid these. So, now, overall, it weakens the motivation and confuses the contributions. In the same way, I feel the external validity is too narrow as per the framing (adaptive visual computation) used in the paper.
- The paper doesn’t provide any working solution either. Their proposed router fails under transfer; 3MDBench policy seems harmful, and DDI fate achieves some corruption control only by making zero replacements. Proposition 2 gives a bound only if uniform continuation-value error is established, which is not the case here.
- The paper says they have provided code, all data and all for full reproducibility. But, nothing is given.
Also, I must say, most of the work reads and sounds heavily AI-generated (not assisted). It is unnecessarily verbose, hard to read, and nothing is clear. Seems like a ghost problem (not well-established) with a ghost solution (not well-established) without any proper connection/use.
Questions for authors
- Firstly, what are you trying to do in this paper? Let’s be real; please clarify what is the problem, why it is important, how it works, and what are you trying to do in this paper.
- Following Critical Weakness 1, I think, it is important to see what happens if we expand the action family to adaptive zooming, multi-scale cropping, tool calling. Does the corruption-risk reversal effect stays or it goes away? Is it a result of the rigid cropping system used?
- Why is this problem important? I didn’t see a proper research gap, establishing the problem and its importance and connection to previous works? What does the paper bring to the table, in terms of contribution?
- Then, about the critical weakness 2, as the solution is not robust as we can see, I think it’s important to explore if a harm-aware router with lower confidence can be bound on signed utility? Can we do some target-support test to check if we can achieve non-trivial coverage under PathVQA/#MDBench/DDI; and if so, at what target risk? Can we train a router specifically to get a transferable harm calculator, using some domain-adversarial or contrastive learning techniques?
- Coming back to corruption-risk reversal, how would you formally test it against simple explanations, such as, calibration failure or general negative transfer (such as pre-specified AUC/based-rate contrasts with CIs)? Will it hold strongly enough to establish the problem?
- Can we trace exactly why these autoregressive VLMs corrupt a previously correct answer? Specially, how do the attentions sink or the tokens compete or KV-cache dynamics shift when such a crop occurs (in mech-interp sense)?
- Can you provide all the things you claim that you provided? (reproducibility)
- Explore and all more related works, such as ReCoVERR [1].
- Can you please connect the findings to something useful and meaningful? Also, can you please improve overall writing and presentation of the work?
- [1] Selective “Selective Prediction”: Reducing Unnecessary Abstention in Vision-Language Reasoning (Srinivasan et al., Findings 2024)