acceptodds
Under review as a conference paper at ICLR 2027

SteerPrint: Steering-Vector-Guided Fingerprints for Black-Box LVLM Ownership Verification

Abstract

Growing concerns over the unauthorized reuse of large vision-language models (LVLMs) highlight the need for black-box fingerprinting techniques that can trace model origins and verify ownership. Existing black-box fingerprints are built mainly around output-level trigger-response associations, with little explicit grounding in internal features inherited from the source model, leaving their persistence under model derivation without a clear internal basis. To overcome this limitation, we introduce and formalize provenance anchors: internal features that persist in derived models while distinguishing them from non-derived models, and further characterize the conditions under which they can be translated into black-box fingerprint signals. Building on this framework, we propose SteerPrint, which uses behavioral directions in representation space as provenance anchors and distills their behavioral effects into visual fingerprints for black-box ownership verification. We evaluate SteerPrint across recent LVLMs, downstream fine-tuning, and model transformations including quantization, pruning, and model merging. The results show that SteerPrint achieves stronger overall verification performance and transferability across derived models than existing black-box fingerprinting methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large vision-language models, model fingerprinting, ownership verification, model provenance, steering vectors, black-box verification

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_M4D1OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses black-box ownership verification for large vision-language models and proposes SteerPrint. The method optimizes images to reproduce the hidden-state displacement and first-response-token distribution induced by direct activation steering, while using parameter-noise augmentation to encourage transfer to derivatives. Verification requires only image–text queries and returned responses. Experiments evaluate transfer under fine-tuning, compression, and adapter merging, alongside specificity against non-derived models.

Critical strengths

The method supports ownership verification without modifying the released source model or accessing the suspect model’s internals. Its use of reusable images across multiple prompts and behavioral detectors, rather than a single fixed response string, is also practically appealing.

Critical weaknesses
  • Important baselines and challenging evaluation settings are missing. SIF[1] leaves the released LVLM unchanged, distills a detectable signal into images, and uses representation perturbations to improve robustness. Although discussed in the paper, it is not included in the experiments. Furthermore, the evaluation does not test independently fine-tuned models sharing the same parent checkpoint as challenging negatives. The merging experiments are limited to combinations of LoRA adapters on a shared base, leaving full-parameter model merging and low source-contribution ratios unexplored. These omissions limit the evidence for both comparative effectiveness and applicability to realistic, difficult provenance relationships.
  • The applicability of SVs as provenance features remains insufficiently characterized. The method relies on the empirical observation that SVs are preserved in derivatives while differing across non-derived models. However, it remains unclear how much model modification these directions can tolerate, or how similar they can remain between models that share weights or training history without a direct derivation relationship. No method is provided to assess or control this boundary in advance. Consequently, these uncertainties limit the method’s applicability to ownership verification, where high reliability is essential, and raise concerns about its performance in the challenging settings described above. Beyond the conditional analysis in Appendix B, a clearer analysis of these limitations is needed to explain when SV preservation provides reliable evidence of provenance, beyond the evaluated checkpoints.

[1] Yifei Zhao, Qian Lou, and Mengxin Zheng. “SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models.” CVPR 2026.

Questions for authors

Given the uncertain limits of SV preservation, how do the authors justify relying on it for ownership verification that requires high reliability?

Reviewer_TuYTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SteerPrint, a black-box ownership verification method for vision-language models that constructs visual fingerprints around an explicit internal feature, a source-model steering vector, rather than a fixed target string. An image perturbation is optimized across natural prompts so that it reproduces the hidden-state displacement and first-token distribution of direct steering (with parameter-noise augmentation), and verification counts how often held-out prompts paired with the image elicit the target behavior. On 40 LoRA/full fine-tuned derivatives of four LVLMs, plus pruned, merged and quantized variants, SteerPrint reports higher trigger rates than PLA/AGDI/RNA, zero false attributions on 27 non-derived models, and supporting mechanism analyses (SV alignment, knockout/rescue interventions, loss ablations).

Critical strengths
  1. The idea is straight-forward and easy to follow.
  2. Framing the fingerprint as the externalization of an inherited internal direction is a clear and interpretable design principle.
  3. The loss ablations show that SV matching, output distillation and parameter noise are each necessary and complementary.
Critical weaknesses
  1. The headline comparison is not apples-to-apples. SteerPrint uses permissive behavioral detectors (any Chinese response, any refusal) and 300 queries, while baselines keep their own fixed-phrase detectors and 1,000-query protocols; derived models are scored by raw FTR but non-derived models by a paired trigger-only rate that discounts clean-carrier hits. Without matched detectors and a threshold-based classification metric (e.g., AUC over derived vs non-derived), the gains in Table 1 are hard to interpret.
  2. Specificity and attack coverage are narrow. The 27-model non-derived panel is evaluated only for Qwen2.5-VL-7B fingerprints; all derivatives are mild author-made fine-tunes on five VQA sets; Appendix Table 20 shows that merely fake-int4 quantizing the vision encoder drops FTR from 96% to 29%, and input-side preprocessing (random resize/crop, noise) is not systematically tested. The mechanism claim also rests on a weak correlation, and Proposition 1 is a triangle inequality rather than a derived condition.
Questions for authors
  1. What are the Table 1 numbers if PLA/AGDI use the same behavioral detectors and 300-query budget as SteerPrint?
  2. Why are derived models scored with raw FTR but non-derived models with the paired trigger-only rate? Please report both classes under one metric and give AUC.
  3. What are the non-derived false-positive rates for the other three source models?
Reviewer_shPvOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SteerPrint, a post-hoc, non-invasive black-box ownership verification (fingerprinting) framework for Large Vision-Language Models (LVLMs) designed to survive post-release modifications such as fine-tuning, quantization, pruning, and model merging. Leveraging the fact that visual inputs—unlike discrete text tokens—are continuous and fully differentiable, the authors apply Universal Adversarial Perturbations (UAP) to target specific internal activations. Specifically, the method extracts behavioral steering vectors (SVs) from the source model via simple difference-in-means and optimizes reusable visual fingerprints to induce the corresponding hidden-state displacement alongside the first-token output distribution. This allows model owners to verify provenance in a strictly black-box setting solely by observing the generated text responses.

Critical strengths
  • Important & Practical Problem: Protecting the intellectual property of open-weight LVLMs against unauthorized fine-tuning and closed-source commercial API redeployment is a timely and practically significant challenge.
  • Strictly Black-Box & Non-Invasive Verification: The proposed method requires no modification to the original model weights prior to release and operates strictly on generated text outputs during auditing, aligning well with real-world API constraints where logits or hidden states are inaccessible.
  • Comprehensive Empirical Evaluation: The authors conduct extensive experiments across 40 fine-tuned derivative models (covering both LoRA and full fine-tuning across 5 datasets), various model compression/merging setups, and 27 non-derived control models, demonstrating strong fingerprint retention and a low false-positive rate.
Critical weaknesses
  • The technical contributions largely consist of a straightforward integration of established components from adjacent domains rather than fundamentally new primitives. If I understand right, the algorithm aims to optimize the loss between hidden semantics vector with UAP with regularlization loss (Matching KL with first token)

  • As SteerPrint relies on high-frequency pixel-level adversarial perturbations (UAP) to pass through the vision encoder and induce exact hidden-state displacements in the language layers, the input realizability pathway is inherently fragile to standard API-side image preprocessing. In real-world deployments, commercial APIs routinely resize, compress (JPEG), or apply basic image sanitization (e.g., Gaussian blur or diffusion-based purification) before feeding images into the LVLM. Even if the internal steering vector is perfectly preserved inside the derivative model, such trivial input transformations can easily destroy the visual UAP trigger, rendering the black-box verification ineffective.

Questions for authors
  • Ablation of test-time preprocessing methods such as gaussian blur, jpeg compression, as the method seems highly correlated to frequency noise.

  • Can the authors describe the difference between uap method and authors method?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.