- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
this paper presents RuleRouter, which selectively replaces a frozen VLM's predictions using confusion-derived textual rules and an ensemble of benefit–harm routers. the authors use an extra development set to decide the rules, adaptation, and intervention thresholds before evaluation. Reported Macro-F1 gains are 5.53 percentage points on EuroSAT, 3.17 on AID, and 0.46 on the low-performing So2Sat setting
Critical strengths
I think the biggest strength of the paper is just the reported scores with very comprehensive metrics, and multiple runs to mitigate the noise.
Critical weaknesses
- the biggest problem of the paper is presentation. Vague definitions like v1, v2 everywhere, the figure 2 as the main pipeline is hard to read with misaligned formats and complex annotations. In general, I wouldn't put the presentation of this paper to a 'ready-for-review' tier.
- The different routing components lack separate benefit. Shared-target comparisons do not establish gains from dual heads or replica averaging, and voting requirements from 1/5 through 4/5 produce identical predictions at the frozen cutoff, I'm not convinced each component actually works as reported together
Questions for authors
- At identical realized intervention coverage, does the full router reduce harm wrt benefit-only routing, with paired uncertainty?
- Can you quantify how often activated clauses are visually correct and discriminative for their class pair?