acceptodds
Under review as a conference paper at ICLR 2027

Ghost Your System: Audit Fibers Hide Hijacks Behind Passing Checks

Abstract

Deployment pipelines routinely accept non-identical model components produced by quantization, export, or compilation as long as they preserve the behavior checked during qualification. However, qualification observes only the declared component interface, whereas the deployed system later composes that component with downstream consumers. A malicious replacement must preserve every checked output on the same execution it redirects. For reusable components, this requires one fixed replacement that operates across inputs. We show that the check may discard distinctions that downstream consumers still use. The states it treats as equivalent form an audit fiber. We prove that qualification is unsafe whenever a component can reach two states in one fiber that induce different downstream behavior. For normalized representations, we derive an exact targeted attack within the fiber and compile it into FiberRouter, a fixed malicious replacement. Experiments across image and text representations and a Qwen model KV-cache handoff demonstrate reliable downstream hijacking while preserving the qualification that authorizes deployment.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Security, Model Supply Chain, Compositional Integrity

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.60 over 5 reviews, above 70% of the others.
Reviewer_1u3wOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether a component can pass qualification yet change the behavior of the system that uses it. The authors call this adversarial qualified substitution and define audit fibers as states that produce the same checked output. Their analysis characterizes safe substitution by whether downstream consumers are constant on the fibers reachable by allowed implementations. For normalized representations with a linear readout, they derive an exact target-steering rule and turn it into a fixed FiberRouter component. They also give an orthogonal weight substitution that preserves the checked output and norm for all inputs and a KV-cache construction that preserves full logits during a specified finite teacher-forced replay while influencing the first unaudited prediction. Experiments cover image and text representations and KV-cache handoffs, including serialized artifacts, hidden qualification inputs, and distribution shift.

Critical strengths
  1. The analysis does more than count unchecked dimensions. It asks whether an allowed implementation can reach those directions and whether a downstream consumer is sensitive to them. This helps explain when the attack should work. The KV result is particularly interesting: it identifies value-cache changes that leave all vocabulary logits unchanged along the specified replay. The image and text experiments, matched controls, and stronger-check ablations provide evidence for this mechanism across different component boundaries.

  2. The same-execution constraint is an important part of this work. The replacement must preserve every checked output on the very input for which it changes downstream behavior; retaining average accuracy is not enough. I also find the fixed-component results useful. FiberRouter and the orthogonal weight construction go beyond finding a suitable perturbation for one feature vector. The serialized-artifact tests and tests on inputs selected after the artifact is fixed support this distinction.

Critical weaknesses
  1. In the main example, classification outputs are checked, but the full embedding is later used for retrieval. The experiments show that this setup can be exploited. What is less clear is whether deployed systems actually approve a replacement on this basis. The cited optimization and compatibility work make the scenario plausible, but does not establish this particular combination of checks and downstream reuse. A concrete deployment configuration would help assess the practical reach of the result.

  2. The evidence for the patched-weight model is not as complete as for the feature transformation. Full-output preservation is reported for 5,000 cached features. When the projection weights are actually patched and the model is run on raw images, the reported test uses one image per audited class, 100 in total, and reports unchanged predictions. An unchanged predicted class does not establish that all logits and the feature norm stay within tolerance. I would like to see these errors measured directly on the full raw-image cohort after saving and reloading the modified model, with the runtime precision and tolerance stated.

  3. The KV experiments show that the first unaudited prediction can be influenced after a fixed teacher-forced replay. The stochastic-decoding tests are useful, but their lower target sampling rates also show that control depends on the generation path. I read this as evidence for the stated finite-replay attack, not for control over arbitrary continuations or complete task outcomes. It is also unclear whether the proposed replay check is used in an existing cache-handoff workflow.

  4. Target Hit@1 and Hit@10 measure how often the attacker gets its chosen item into the results. They do not show what happens to the rest of the ranking, and classification accuracy is not a measure of retrieval quality. The appendix does report non-target behavior for a sparse, source-gated attack, but the main FiberRouter policy routes every checked state. For that setting, I would like to know how retrieval relevance or ranking stability changes with the attack budget.

Questions for authors
  1. Can the authors give a concrete deployment configuration that qualifies an encoder through the stated classification readout and norm, then reuses its full embedding for retrieval? What additional compatibility or retrieval checks does that pipeline apply, and how would they change the attack surface?

  2. For the actual patched-weight model, what are the maximum errors in the full checked-logit vector and representation norm on the complete raw-image validation set? Please report the serialization format, runtime dtypes, and acceptance tolerances.

  3. Does the finite teacher-forced replay check come from an existing cache-handoff workflow, or is it a proposed evaluation setting? What do the stochastic-decoding results establish about control of the first unaudited prediction, and what remains untested at the task level?

  4. For the main FiberRouter policy, how do ordinary retrieval quality or ranking stability change across attack budgets? Reporting these alongside target Hit@1 and Hit@10 would help separate target promotion from broader disruption of the retrieval results.

Reviewer_sYPbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors investigate the qualification checks associated with different variants of a given model in deployment. Models are only checked on a tiny fraction of inputs compared to an original known-good model. The authors consider a malicious replacement mechanism in which the replacement model mimics the original model on those fixed inputs, and show that such checks do not consider inputs that matter for downstream consumers. The so-called "audit fibers" are basically this an insight about this limited form of evidence. The authors derive an attack against such fibers and form "FiberRouter," which I would summarize briefly as a malicious replacement mechanism.

Critical strengths
  • The authors tell a defender basically how many probes are needed in order to guarantee their notion of safety, which is reasonably defined. The safety claim criterion is framed around downstream behavior. The authors being able to quantify how many linear probes are required is straightforward and compelling.

  • The authors show that an attack can succeed within the noise margin of a given qualifier check. It seems the attack sits around 5 orders of magnitude beneath the tolerance for noise.

Critical weaknesses
  • The strength is perhaps overstated. From the paper's motivation about the Intel criteria of "accept if within 1% accuracy," it is not relevant whether the attack can achieve 10^-15 tolerability of output, because the model would definitionally preserve classification behavior. So I don't quite understand then the purpose of that motivation given the evidence presented.

  • Related: to reach the 82% attack effectiveness, it seems the authors assume the attacker knows specific retriever and gallery. And I suppose this makes sense given the nature of the attack and needing to account for "downstream consumer" behavior. But this is perhaps unrealistics.

Questions for authors
  • What happens to the routing policy when the attacker is not given information about the consumer or its gallery? Does the attack effect survive at all?

  • Can you explain which of the two gates is supported by each row of Table 2? Like the Intel 1% accuracy gate? or the other more strict gate?

Reviewer_6owYOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a threat in the qualification process in model deployment with transformations like quantization, export, and compilation, which was designed for preserving the model behavior. The core claim of the paper stands on that a malicious replacement could provide such a qualified replacement that passes the test while affecting downstream behaviors in action. The paper demonstrates successful attacks in different types of models and scenarios and provide detailed theoretical analysis.

Critical strengths
  • The paper is written in an easy-to-follow way. The figures are presented in a crystal clear style and overall I enjoy reading it.
  • The authors provide detailed theoretical analysis as well as experiments across scenarios (with quantization, representation and KV-cache handoffs) and different models (vision, text, language models). The experiment analysis is sound and convincing to my knowledge.
  • The authors provide supplementary materials for coding artifact, which facilitates reproducible research.
Critical weaknesses
  • Technical novelty. The idea appears close to null-space-constrained editing. For example, the weight-level version (Prop. 3.2, Procrustes rotation of the final projection) is literally an edit to the last layer constrained to preserve a specified linear readout. The attacker looks at a 100-dim "summary" of a 768-dim output, and the idea of the rest 668 dims stay hidden to the checker, while being manipulatable is natural. Reference [1] provides a similar observation: they use fully invertible networks (i-RevNets), mapping representations back to an exact input and decompose such representation into a component that classifier's logits read and the residue. They observe the component of class A and a residue of B combined inverts to an image of class B yet classified as A. Therefore, semantically meaningful changes can live entirely in directions the output ignores. The core geometry seems the same in spirit. Authors are also encouraged to look at [2] and [3] and derive theoretical novelty to sharpen their contributions.
  • Practicality of threats. Because the blind spot is in what is measured, not where, no choice of test inputs helps like in backdoor detection problems, as repeatedly stressed by the paper, but the blindness is more like an artificial one in practice. 1) In the representation one where authors cite Hugging Face Optimum and claim that a manipulation of Az (100-dim) of z (768-dim) stay hidden to a checker. Yet in practical deployment, for an encoder, the output is z itself. The threat model only works if the integrator checks the task head and never the embedding. For an encoder whose stored gallery must stay compatible, that’s a strong and arguably strawman assumption. Table 2 reports that the benign FP16->8-bit replacement shifts embeddings by a median of 5°; Table 7 reports the attack’s mean full-feature chord at 25° budget as 0.389 (22° of embedding angle from 2arcsin(0.389/2)). It seems that the shift in embedding space is separable for benign and malicious replacment overall by looking at this angle, and the authors do not provide demonstrations that the defenders cannot. 2) Its stealthiness is weak. FiberRouter remaps every class and about 82% of all retrieval queries return the attacker’s item. Any user, click-through metric, or the retrieval benchmark itself would notice almost immediately. 3) The KV-cache threat model looks constructed and I am not sure whether this is a realistic one. In disaggregated serving, prefill and decode nodes typically belong to the same operator. Who is the untrusted party producing the cache? In addition, the attack is optimized for each prompt, not a reusable universal one, so the attacker must see the prompt and the clean continuation, which is also less practical. While theoretically sound, the presented threat seems hardly matter in practice.
  • The paper is math-heavy, yet novel derivation or theorems are only part of it, while some of them are textbook linear algebra restated in the new context, and the logic flow may appear confusing to the general audience. For example, Thm 4.1 (composition criterion) is stating that a function is constant on the fibers of a iff it factors through a, that is to say, a check (defense) is sufficient iff downstream uses only what the check sees, which is why the attack is viable at its core. This is essentially a definition, not something to be proven. Prop 3.2 is stating that orthogonal maps (rotation or relfection R) fixing row(A)≅O(q), i.e. when Rv=v for every v in row(A), which is the direction checkers can see (Az), these maps are exactly the rotations of a q-dim space, which will not affect what outputs checkers see. This is followed by Prop. 3.3, which is again an application of "a linear map doesn’t see its null space" applied to attention heads and the cached values' change. Thm 4.1 and Thm 4.2 again revolve around the consumer’s readout restricted to the directions the check can’t see, with Thm 4.1 stating that it's safe iff that readout is 0, and Thm 4.2 stating that attack strength = spectral norm of that readout. For general audience, this logic flow is buried under heavy math, and I would highly recommend adding a toy example for better clarity.

[1] Jacobsen et al., ICLR 2019, "Excessive Invariance Causes Adversarial Vulnerability."

[2] D'Amour et al., JMLR 2022, "Underspecification Presents Challenges for Credibility in Modern Machine Learning."

[3] Tramèr et al., ICML 2020, “Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations.”

Questions for authors

I could be missing some key points due to the limited time window for reviewing this paper. I am happy to raise my score if the authors could substantially and convincingly resolve these questions.

  • Can you explain in more details on how your theoretical insights link to the papers I referred to? In particular, beyond attacking practical systems, is there any insight in geometry that also inspires understanding and defenses of this phenomenon, and how does this compare to a clean-accuracy-preserving backdoor or a null-space edit under the same gate?
  • Does the presented attack work in any realistic deployment scenario? What concrete deployment qualifies only the task head of an embedding model while retaining a gallery that depends on the full embedding and how would downstream monitoring fail to detect a policy that redirects most queries?
  • Under a full-embedding cosine or tolerance check, what is the separation between benign quantization drift and the attack’s required drift? Is there any stronger attack that could bypass this check?
Reviewer_3ryeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces adversarial qualified substitution: a replacement component preserves the complete qualification transcript yet redirects an unchanged downstream consumer. The authors formalize audit fibers, derive exact steering for normalized linear checks, compile fixed policies via FiberRouter, and realize weight-substitution and KV-cache attacks. They characterize safe substitution and reachable downstream control theoretically. Experiments across CLIP/SigLIP vision retrieval, BGE/Qwen text embeddings, and Qwen KV-cache handoffs show high targeted hijacking with qualification drift near numerical precision, including hidden inputs and deployment shift.

Critical strengths
  1. Novel security problem and framing. The paper defines adversarial qualified substitution and audit fibers, exposing a real gap between component qualification and downstream composition. This reframing is consequential because it generalizes beyond specific backdoors to reusable model and cache handoffs.

  2. Strong theory with actionable criteria. It derives exact audit-fiber steering, a composition safety criterion, and reachable downstream-control bounds. These results are consequential because they explain when qualification is insufficient and connect attack feasibility to check coverage and consumer sensitivity.

Critical weaknesses
  1. Threat-model realism depends on downstream knowledge. The strongest targeted results require attacker-chosen targets tied to the deployed gallery/consumer and calibration utilities; several experiments assume K1–K3 downstream knowledge. The consumer-blind setting reports only continuous exposure, not targeted Top-1. Since the paper’s headline claim is targeted hijacking by a supplier-submitted component, this dependence on integrator-private downstream information materially narrows the practical threat and weakens the central security claim.

  2. Exact guarantees are interface-specific. The exact audit-preserving construction is established for normalized representations with a linear readout plus norm check. Nonlinear checks rely on iterative relinearization with weaker evidence. This matters because the paper’s broad conclusion is supported conceptually, but the strongest exactness and preservation claims do not extend to common nonlinear or multi-probe qualification interfaces.

Questions for authors

Clarification requests

  1. What is the minimal downstream knowledge needed for targeted hijacking? In the strict K0 setting (qualification only, no consumer family or gallery), can FiberRouter or a consumer-blind direction achieve targeted Top-1, or only continuous exposure?
  2. How exact is preservation for nonlinear and multi-probe checks? The paper reports near-machine-precision drift for linear readout+norm, but nonlinear qualification yields 69.17% joint top-1 at 30°. Do the authors claim exact audit-preserving substitution for nonlinear checkers, or only empirical preservation within tolerance?
  3. Is the downstream consumer fixed and known at artifact commitment? Several experiments assume K1–K3 knowledge, but a real supplier may not know the retained gallery, retriever, or consumer family. Please state which experiments use only public or qualification-side information and which require deployment-side calibration.

Suggestions for future work

Evaluate against stronger, more realistic qualification suites. The paper shows 32 aligned probes reduce residual yield to 23.8%, but eliminating control may require covering most of the 668-dimensional null space. A practical defense-cost analysis, such as randomized probes, behavioral tests, or multiple downstream proxies, would clarify deployability.

Reviewer_k4VkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines an attack in which ML components are replaced during deployment. In many deployment pipelines, replacement components are accepted if they maintain the behavior verified by a qualification procedure. However, the qualification procedure may only observe a subset of the information consumed by other components. The authors address this discrepancy with the concept of an audit fiber, which is a set of internal states that are indistinguishable to the qualification procedure, yet may result in different downstream behavior. They demonstrate that an attacker controlling a replacement component can modify representations within an audit fiber while preserving the checked behavior and manipulating an unchanged downstream consumer. The paper also introduces FiberRouter, which exploits unchecked representation dimensions to manipulate downstream behavior while maintaining the original output. To illustrate the potential impact, the paper demonstrates attacks against hidden qualification inputs and shows that KV-cache modifications can redirect predictions while maintaining previously audited outputs.

Critical strengths
  • Interesting and well-motivated security problem beyond conventional model-backdoor attacks.
  • Theoretical framework to better understand the attack (audit-fiber abstraction, although the full framework remains somewhat unclear to me).
  • FiberRouter is a practical realization of the attack and the authors demonstrate that the attack can be implemented through a fixed modification of model weights.
Critical weaknesses
  • The attack model and attack scenario needs to be revised (see below for details)
  • The evaluation would be significantly stronger if it would compare the proposed attack against realistic alternatives to the deliberately incomplete qualification procedure. The authors should evaluate if relatively inexpensive embedding-consistency checks already eliminate most of the available attack space as a strawman baseline.
  • The attack introduces potentially substantial changes in downstream behavior, which would be easily detected. The paper repeatedly emphasizes that the malicious replacement passes the same qualification as a benign replacement, but these effects on downstream behavior would be easy to detect?
Questions for authors

My main concern is the practical threat model: the attack assumes that an attacker can replace a component while knowing the qualification interface and, in several experimental settings, substantial information about the downstream consumer is required. How realistic is such an attack scenario? Why would a deployment pipeline that explicitly relies on embedding compatibility validate only classification behavior? In practice, a system that reuses existing embeddings for retrieval would need to validate compatibility with those embeddings, because otherwise even benign encoder modifications could substantially degrade retrieval quality? The authors need to provide convincing arguments.

Related to the previous comment (and slightly overlapping): the attacker controls the replacement artifact, which is already a powerful capability. The paper should explain more clearly why an integrator would accept arbitrary attacker-supplied components rather than artifacts generated by a trusted quantization, export, or compilation process. How realistic is such a scenario?

The KV-cache attack uses a substantially different attacker model given that the paper assumes that an adversary controls the cache state associated with an individual prompt (instead of a replacement component, as in the rest of the paper). Aren't these capabilities significantly different?

The main attack scenario assumes that an integrator will accept a replacement encoder after validating its classification behavior, even though the deployed system also depends on the embeddings' compatibility with an existing retrieval system. It is unclear how frequently such an incomplete qualification procedure would be used in practice. A deployment pipeline that depends on embedding compatibility would normally need to validate this property explicitly. The authors should provide stronger evidence that the threat model is practical in realistic deployment settings. The threat model must be clearly described and justified. Only then can I increase my score.

In the considered scenarios, how would an attacker obtain the ability to submit or modify a replacement component? What prevents the integrator from validating the provenance of the replacement artifact?

How much legitimate utility is lost under the proposed attacks? Can you quantify the trade-off between attack success, benign retrieval performance, and detectability?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.