- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes REVELIO, a framework for discovering interpretable failure modes of vision-language models (VLMs), defined as combinations of user-defined concepts (e.g., {cyclist, foggy}) under which a model produces incorrect responses. Scenes are generated from concept sets using CARLA for autonomous driving and an image generation model for indoor robotics, with ground-truth answers determined by predefined safety rules. Two search strategies, diversity-aware beam search and Gaussian-process-based Thompson sampling, explore the combinatorial concept space under a budget of 1,000 VLM queries, using five samples per concept set and a 60% failure-rate threshold. Across five VLMs, the authors report that both methods generally outperform random search, with performance varying by application. Some discovered failure modes also transfer across models.
Critical strengths
The formulation is a useful and practical step beyond existing evaluation. Benchmarks report aggregate accuracy, and adversarial attacks return isolated failing inputs. REVELIO instead describes failures as human-readable conditions evaluated over a distribution of scenes, which separates recurring failures from one-off errors. Because it needs only black-box access, it applies to closed-source frontier models, which are often the ones considered for deployment.
Critical weaknesses
- The reliability of the failure-rate estimates is a concern. Each concept set is judged on 5 samples, runs are not repeated, and indoors the average retested failure rate of the top candidates falls below the 60% threshold for most models. The threshold is also somewhat below the 67% failure rate expected from random guessing on the three-option driving task. As a result, it is not always clear which reported failure modes reflect consistent model behavior rather than noise. More samples, repeated runs, and comparison against chance would help address this.
- The paper reports how many failure modes each method finds, but not what fraction may be missed. Only about 0.1% of concept sets are tested, and the random baseline suggests that more failure modes likely exist. A comparison with simpler alternatives, such as exhaustively testing all single concepts and pairs, would be informative. Without such a reference, the method appears to surface examples of failures rather than fully characterize where a model fails.
Questions for authors
- Failure modes are defined as concept combinations, and Appendix H reports strong interactions, yet the GP uses an additive kernel that cannot represent interactions. How does it guide the search toward genuinely conjunctive failure modes, and how many reported modes are a single concept plus extras?
- The paper treats failure modes as interpretable because they are expressed as named concepts. How do the authors verify that the listed concepts actually cause the failure, rather than irrelevant concepts included in the set (e.g., default weather) or uncontrolled scene elements outside it? Appendix G treats supersets as distinct failure modes; could the authors report results after reducing each mode to its minimal failing subset? Evidence that the concepts identify the cause would strengthen the interpretability claim in my assessment.
2 comments
Interpretability
Our method is designed to surface "sufficient" conditions for high failure rate, similar to popular interpretability methods such as [1]. Thus, a causal exposition is out of scope for this work. The interpretability of the search comes not just from our semantically-meaningful, realistic, high-level concepts, but also from their intuitive conjunctive combinations being assessed for failure modes. Popular works on interpretability [1,2], similar to us, commonly use sufficient sets of high-level, realistic features of input as explanations that are understandable to human experts. Uncontrolled scene elements are set randomly in each observation, hence they cannot consistently contribute to failure rate of failure modes. The irrelevant, default settings similarly impact all concept sets and hence cannot directly be attributed the failures of selective failure modes.
Minimal concept sets:
We reduce each algorithm’s observed failure modes to the minimal elements under set inclusion, keeping a failing set only if no failing proper subset of it was observed. We report raw and reduced PFMs (new Appendix I, Table 24):
| VLM | Thinking | Random PFM | Random reduced | BS PFM | BS reduced | GPTS PFM | GPTS reduced |
|---|---|---|---|---|---|---|---|
| Gemini 3 Flash | minimal | 3.5 | 2.5 | 15.5 | 12.0 | 18.5 | 16.5 |
| Gemini 3 Flash | low | 1.0 | 1.0 | 12.5 | 10.5 | 16.0 | 11.0 |
| Gemini 3 Flash | medium | 1.5 | 1.0 | 6.5 | 4.5 | 10.5 | 6.0 |
| Gemini 3 Flash | high | 1.5 | 1.5 | 6.0 | 5.0 | 9.0 | 4.5 |
| Claude Sonnet 4.6 | default | 5.0 | 4.5 | 24.0 | 12.0 | 27.0 | 12.5 |
| Claude Haiku 4.5 | default | 38.0 | 20.5 | 48.0 | 23.5 | 60.5 | 32.5 |
| Qwen3-VL-235B | default | 31.0 | 18.0 | 41.0 | 21.5 | 51.5 | 24.5 |
| GLM-4.6V-Flash | default | 27.5 | 20.0 | 42.0 | 22.0 | 52.5 | 23.5 |
References
[1] Ribeiro et al., Anchors: High-Precision Model-Agnostic Explanations, AAAI 2018.
[2] Ribeiro et al., "Why Should I Trust You?": Explaining the Predictions of Any Classifier, KDD 2016.
We thank the reviewer for their time and valuable questions and inputs. We address your concerns and questions below:
Please see general response for new experiments increasing number of samples for failure modes.
Please see general response for new experiments with more runs of the search.
With corrected values, fixing a typo, (updated Table 1, in blue), the reviewer's concern no longer holds for BS. Its candidates retest at 72.3% on average across the eight indoor models, with the mean at or above 60% on five of eight models, and 63 of 80 BS candidates individually retest at ≥60% (Random: 12 of 80). GPTS validates less strongly indoors, consistent with Table 2, where BS is the more effective indoor algorithm.
We report results for the driving domain when the failure mode threshold is increased to 80%, above the random chance of 67% (Appendix G.4). The results for indoor domain were already above 50% random chance (for two options) with 60% threshold.
We respectfully would draw the reviewer's attention to the size of the combinatorial concept set space, which makes it practically infeasible to "fully characterize" all model failures. With just 30 concepts (as in driving domain), ignoring the modifiers, we have approximately a billion concept sets (power set is of size 230) that are prohibitive to evaluate exhaustively. With a practical budget of 200 concept sets evaluated, we thus strategically navigate the search space for as many failure modes as possible within the budget. That is also a reason why we start with a small, 5 sample budget to quickly evaluate each candidate concept set to assess more concept sets in a search run. We do not claim that our method is meant to find every failure mode, but to discover examples that show some of the problems.
The surrogate Gaussian Process only ranks proposals. The measured failure rate (FR) of the full set decides whether they are considered.