acceptodds
Under review as a conference paper at ICLR 2027

Certified Interface Aliases: Exact Collisions in Vision-Language Preprocessing, and When They Exist

Abstract

Vision–language verifiers and routers must distinguish errors repairable by more reasoning from those caused by visual evidence never reaching the language model. This distinction lacks ground truth because annotators see full-resolution images while models receive preprocessed tensors. We introduce AliasForge to create cases where the relevant fact is provably absent from the interface. Fixed-point resampling makes the pre-rounding resize an exact integer linear map that can send nonzero integer perturbations to zero. Hiding a label-flipping perturbation there produces images with opposite step-correctness labels but bit-identical interface states. Every verifier therefore has the same output law on both members, giving pair-balanced accuracy exactly one half and zero gain from language-side repair. From the resize configuration alone, a lattice criterion supplies realizable collisions and certifies their absence within the specified construction family. It resolves all twenty screened configurations, seventeen as constructible and three as non-constructible. We construct certified pairs across three architectures and certify four additional processors, with zero decision-logit gap on all scored pairs and none of the controls. The pairs also screen routers that waste computation on re-attention or further reasoning. On natural items, per-item routing headroom exists, but no tested interface-only router improves over stopping. Our fiber ceiling bounds the headroom recoverable from the interface.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision-language models, image preprocessing, fixed-point resizing, verifiers, process reward models, test-time compute routing, certification, lattice reduction, image-scaling attacks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kP4AOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper investigates whether preprocessing steps that are often applied to images before they are passed on to a Vision Language Model (VLM) may prevent the information that the model needs to make certain decisions from reaching the model. The authors find that the downsampling employed in such pipelines uses fixed-point integer weights, this can be exploited by constructing a pattern which yields the same results after the preprocessing but which changes the label that should be assigned to the actual image (they describe this as adding a perturbation which is in the "null space" of the downsampling) and call such a pair of images a "certified pair". Testing or verifying the correctness of the downstream model will never identify the issues introduced before the preprocessing step. An algorithm called AliasForge for constructing such perturbations is presented by the authors and is demonstrated on Qwen2.5-VL-7B, LLaVA-1.5-7B and VisualPRM-8B as well as other processors. The authors further prove that every fixed-point downscaling algorithm has such a null space and bound their smallest size. They also study whether verifiers (here this term is used to describe a model that judges whether in a reasoning mechanism a prediction made by a model is correct) and routers (which take the verdict and make a decision based on it - rerun the verifier, look at a subsection of the image in more detail, ...) can be wasteful given that they only look at postprocessed images.

Critical strengths
  • The authors introduce a thorough approach for checking whether their reimplementation of a resize step is identical to that of a library and openly state when no decision can be made. The main proofs also look correct to me and the main issue that the paper targets seems like one worth investigating.
  • The experimental design is thorough, verdicts were calculated before any certified pairs were computed. The authors employ various controls to double-check the different pieces of the paper, and it is nice to see that the vulnerability identified in the paper also holds up in popular models that are deployed in practice such as CLIP, SigLIP or LLaVa.
Critical weaknesses
  • The main movation of the paper is that it is difficult to decide whether a model failing for a certain task is caused by a lack of visual information in the input or by the model itself. However, the authors find and acknowledge that their certified pairs can't answer this question. The pairs are stripe patterns which only hold up for specific downsampling settings, so models processing images at the original resolution or resize them using floats are not affected by this. The experimental evaluation then checks whether routers waste compute, but a router's performance on certified pairs in Table 11 seems to mostly be unrelated to its performance on real inputs. Moreover, the cost parameter is assumed to be 0 in a large part of the experimental evaluation, so I am unsure whether the "wasted compute" in Table 2 isn't mostly a consequence of that assumption. Overall, these points seem to limit the practical relevance of the paper.
  • I found the router study difficult to interpret. I am not familiar with the literature in this area, but I found some things here unclear. The IAS score that the authors propose only see an image and not a question or the model's reasoning, so it doesn't seem surprising to me that from only the image information a simple classifier would not be able to decide whether to spend more compute. I also found it unclear what exactly the baseline is here, the authors introduce a "placebo" where the verifier gets the same critique prompt without being told to look at the image again, the appendix shows that this is ~7 points worse than the verifier itself. Against the original veridct, actions are 7-9 points worse and even picking the best crop is slightly worse. I'm not sure if the "stop" baseline mentioned in the experiments section is the placebo or the original verdict, but when compared to the original verdict it seems that every action makes the verifier perform worse. This performance is explained with an upper bound on what a router can gain from the processed image, but the routers are only trained on 1280 items with thousands of inputs, so the routers might just not train properly on the small training set. I don't think that this can be ruled out.
  • The paper is full of jargon, new terms that (to me at least) seem to be defined by the authors somewhere and then referenced again so the reader has to keep scrolling around the paper to check again what these terms meant. I also found significant portions of the paper very difficult to understand, even when re-reading sentences multiple times. The writing should be substantially improved.
Questions for authors
  • Could the authors clarify if "stop" is the placebo or the original verdict?
  • The idea seems related to invariance-based adversarial examples ("Excessive Invariance Causes Adversarial Vulnerability", Jacobsen et al., 2018 and "Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations", Tramel et al., 2020), could the authors discuss the connection between those papers and their work?
Reviewer_pXxbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AliasForge, a framework for constructing certified interface aliases in vision-language models (VLMs). The key observation is that fixed-point image resizing can map distinct high-resolution images to bit-identical preprocessed representations. By constructing image pairs with opposite ground-truth labels but identical interface states, the authors establish cases where the relevant visual evidence is provably unavailable to the downstream model.

The paper provides theoretical guarantees showing that no verifier operating on the identical interface representations can distinguish the two images or improve pair-balanced accuracy beyond chance through language-side computation alone. It further develops a lattice-based constructibility criterion that determines whether realizable integer perturbations exist for a given resizing configuration.

The authors validate AliasForge across three VLM architectures and additional preprocessing pipelines, correctly characterizing all 20 screened configurations, including 17 constructible and three non-constructible cases. Finally, the paper applies certified aliases to evaluate test-time compute routing strategies. Although natural benchmark items exhibit positive oracle routing headroom, the tested interface-only routing methods fail to achieve statistically significant improvements over always stopping under the primary evaluation protocol.

Critical strengths

S1. A rigorous and verifiable framework for identifying information loss at the visual interface.

The paper introduces a principled approach to identifying visual information that is provably absent from the representation available to a VLM. Unlike previous methods based on approximate representation similarity or learned probes, AliasForge constructs image pairs with opposite ground-truth labels but exactly identical interface states. This enables a model-independent guarantee that no downstream verifier or language-side computation can distinguish the pair.

The constructibility analysis is particularly compelling. By exploiting the exact integer structure of fixed-point resampling, the authors develop a lattice-based criterion for determining whether realizable collisions exist under specific preprocessing configurations. The theoretical treatment is supported by empirical validation across multiple model architectures, and the negative controls provide additional evidence that the observed collisions arise from the proposed mechanism.

This constitutes a meaningful methodological contribution to the study of perceptual limitations in multimodal models.

S2. A principled connection between certified perceptual failures and test-time compute allocation.

The paper goes beyond demonstrating preprocessing collisions by investigating their implications for verifier reliability and test-time compute routing. The certified pairs provide a ground-truth setting in which additional language-side reasoning is provably ineffective, allowing routing policies to be evaluated for unnecessary computation without relying on potentially unreliable error-type annotations.

The fiber-ceiling analysis also offers a useful theoretical perspective on the information constraints faced by routers. Importantly, the authors report negative routing results rather than overstating the effectiveness of their proposed accessibility score. The combination of formal guarantees, controlled experiments, and careful reporting of unsuccessful routing attempts makes this aspect of the paper informative.

Critical weaknesses

W1. The practical applicability of the certified construction is limited by its dependence on specific preprocessing configurations and carefully constructed visual patterns.

Although the paper establishes exact collisions under certain fixed-point resizing configurations, the constructed examples require specific resizing ratios, filters, and integer perturbations that fit within the pixel-value constraints. The authors acknowledge that natural images rarely arrive at the eligible resolutions.

The natural-image experiment further illustrates this limitation. Among 200 benchmark images resized to an eligible configuration, only 48 supported certified pairs at the specified amplitude, and only eight preserved clearly legible marks. This raises concerns about how frequently the proposed construction can produce meaningful perceptual failures under realistic input distributions.

Moreover, the certification mechanism relies on a particular form of preprocessing information loss. While this is valuable as a controlled measurement instrument, it remains unclear to what extent these examples reflect the types of perceptual failures that commonly occur in deployed VLM systems.

This limitation matters because the paper motivates the method through practical verifier and routing failures, but the connection between the certified construction and naturally occurring perceptual errors is not empirically established.

W2. The routing experiments do not fully establish whether the observed limitations arise from insufficient interface information or the particular routing methods and interventions evaluated.

A central motivation of the paper is to understand whether interface-derived information can guide the allocation of additional computation. However, the proposed IAS-ROUTE does not significantly outperform always stopping on natural examples. Other tested interface-only routers also fail to obtain meaningful improvements, despite positive per-item oracle routing headroom.

The fiber-ceiling result provides a theoretical upper bound on the value recoverable from a given interface signal. Nevertheless, the experiments do not establish that this ceiling is sufficiently low to explain the observed routing failures. As the authors acknowledge, the performance of learned routers provides only lower bounds on the interface-specific ceiling.

Furthermore, the tested interventions exhibit limited average effectiveness: re-attention and additional reasoning provide almost no aggregate improvement, while the evaluated re-encoding intervention reduces performance on average. Consequently, the negative routing results may reflect limitations of the chosen interventions, routing architectures, or training procedures rather than an intrinsic inability to infer useful routing decisions from the interface.

A stronger connection between the theoretical information bound and the empirical routing failures would make this contribution more convincing.

Questions for authors

Q1. How representative are the certified aliases of naturally occurring perceptual failures?

The construction requires specific preprocessing configurations, and only eight of the 200 resized benchmark images support clearly legible certified marks in the reported natural-image experiment. Could the authors provide further analysis of the prevalence of eligible configurations in realistic VLM workloads, or evidence connecting certified aliases to naturally occurring perception failures?

This would help clarify whether AliasForge should primarily be viewed as a controlled diagnostic instrument or whether it can also inform practical error diagnosis in deployed systems.

Q2. Can the authors better distinguish information-theoretic limitations from limitations of the tested routing methods?

Proposition 5 provides an upper bound on the routing value recoverable from the interface, but the experiments do not directly estimate this bound. The negative results therefore leave open whether useful information exists in the interface but is not successfully exploited by the tested routers.

Could the authors provide additional analysis or controlled experiments to better characterize the gap between the empirical router performance and the theoretically attainable interface-only routing value?

Q3. How much do the routing results depend on the effectiveness of the available interventions?

Table 2 shows that the evaluated re-attention and reasoning actions yield negligible average improvements, while re-encoding performs worse than stopping. Could the authors clarify whether stronger or more targeted interventions would change the routing conclusions?

In particular, the positive performance of the oracle-selected crop suggests that image-dependent intervention selection may be important. It would be useful to understand whether the limited effectiveness of the evaluated re-encoding action contributes to the poor performance of the learned routers.

Q4. What is the scope of the non-constructibility guarantees for the screened configurations?

The paper carefully distinguishes the absence of realizable null-space perturbations within the specified construction family from the absence of arbitrary interface collisions. Could the authors further clarify which excluded configurations admit broader collision constructions outside the analyzed family, if any are known?

This distinction seems important for interpreting the practical implications of the constructibility criterion and its potential use in assessing preprocessing robustness.

Reviewer_eYTrOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces certified interface aliases: pairs of images with different ground-truth labels that reach a vision-language model as bit-identical pre-processed states, because the deployed integer fixed-point re-sampler has a nontrivial integer null space. The authors build ALIASFORGE, a constructor that hides a label-flipping perturbation inside the resize kernel, and prove with Theorem 2 that no verifier or language-side repair can beat the chance on the image pair. They derive a constructibility criterion computable from the resize operator alone, validate the instrument on three architectures plus four additional pipelines, and then use certified pairs as a measurement tool to audit interface-only routers. Their main finding is negative: although natural items exhibit substantial per-item routing headroom (split-sample V ≈ 0.056), no tested interface-only router improves on always stopping. The fiber ceiling through Proposition 5 is offered as the explanation. Therefore, extending certification beyond fixed-point resampling will be the next step.

Critical strengths
  1. The paper does not merely report that routers fail; it proves why they can fail structurally (Theorem 2, Proposition 3, Proposition 5). The data-processing-inequality framing is the correct one, and Proposition 3(b) (Neyman–Pearson/TV bound) is a sharp statement: only an exact collision certifies chance without assumptions about the verifier.

  2. The constructibility criterion (Theorem 6, Proposition 9) operates purely on the reconstructed integer resize matrix — no weights, no queries, no rendering. This is an unusually strong property for an audit tool, and the prospective prediction on a second architecture (Table 1) is a meaningful validation design.

Critical weaknesses
  1. All certified witnesses require an integer downscale ratio with a fixed-point resampler. Of the 20 declared configurations, only integer-ratio bicubic cases are constructible; 1.5× and 2.5× are excluded. Natural benchmark images had to be resized into the eligible regime (none of 200 had a native eligible size).

  2. The authors argue the fiber ceiling explains the null, but Table 11 shows oracle-supervised routers reading (z, Q, R), which cannot be fiber-limited, since (z,Q,R) identifies the item, also achieve at most +0.010 (unadjusted p = 0.019, not surviving Holm). If text-conditioned routers with full item identity barely beat stopping, the bottleneck may be the action set's low average value or noise, not the interface-only information constraint.

  3. The relationship between the constructed pairs and natural errors is assumed, not measured. The paper is explicit that "we do not measure how often natural errors are interface-limited." The absent-side calibration anchor (IAS = 1/2 on certified pairs) is a clever idea, but there is no evidence that the score's behavior on natural interface-limited errors resembles its behavior on constructed ones.

Questions for authors
  1. Given that hidden-state routers reading (z,Q,R) achieve only +0.010, how much of the routing null do you attribute to V_Φ < V versus to high variance / low realized action value? Can you bound V_Φ from above for any of your six router families, or is only a lower bound available?

  2. Can you re-run the median-threshold diagnostic with the non-stop action chosen without access to the scoring replicate (e.g., fixed by development-half optimization only), to show the random-score gain goes to zero? If it does not, what does that imply about the action-gain measurement pipeline?

  3. Have you attempted an adversarial search for image-dependent fields not in your enumeration (e.g., metadata, position IDs derived from geometry, cached prefill state)? What would falsify the completeness assumption, and how would a practitioner detect incompleteness in a new deployment?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.