FIRA: Fixed-Support Intervention and Readout Audit for Visual Token Pruning
Abstract
A stable visual-token ranking can coexist with failures shared by the compared selectors. Fixed-Support Intervention and Readout Audit (FIRA) holds selected positions fixed while recording actual outputs and candidate-restricted decisions. On 800 A-OKVQA images, changing the readout reduces historical LLaVA delete–zero ranking interactions by 19.625 points in NF4 and 18.625 in FP16. Qwen2.5- VL with MMTok/DivPrune supports instead exhibits shared failures that largely cancel in selector comparisons. Native OCRBench-v2 text-image choices provide 536 questions and seven complete arms. On 429 confirmation questions, full-input accuracy is 68.30%, the shared excluded-correct shift is 19.35 [15.81,23.10] points and the differential shift is 3.73 [-1.18,8.51], with 97.5% paired-image intervals. The task meets its frozen difficulty criterion. Another 7,504 decisions confirm positive shared shifts at 25/75% retention. Saved traces from 194 receipts reveal different multi-token candidate decisions in 59/970 Qwen and 116/970 Monkey OCR cells. These results motivate reporting absolute answer outcomes alongside relative rankings under explicit answer contracts.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.