- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Knowledge-Boundary Fingerprinting (KBF), a black-box auditing framework for detecting potential model substitution in third-party LLM APIs. The key observation is that factual recall near a model's knowledge boundary can provide a distinctive behavioral fingerprint, including stable but incorrect answers. The proposed framework constructs reference-specific probes, filters them based on response consistency and optional contrastive comparisons, and applies a calibrated binomial test to determine whether a suspect endpoint is statistically consistent with the claimed reference model.
The authors evaluate KBF across 16 production LLM endpoints, reporting successful detection of all 155 economically relevant model substitutions without false positives in 16 same-reference controls. Additional experiments examine robustness under deployment variations, partial-routing scenarios, and real-world relay APIs.
Critical strengths
-
Practical and well-motivated problem. The paper addresses an important issue in the increasingly complex LLM API ecosystem, where users may have limited visibility into the actual models serving their requests. The black-box setting is realistic, and the proposed approach does not require access to model weights, logits, or provider-side information.
-
Interesting use of stable factual recall. The observation that repeatable factual errors can serve as model-specific fingerprints is interesting. The distinction between common knowledge, stable boundary recall, and unstable beyond-boundary recall provides an intuitive motivation for constructing discriminative probes.
-
Extensive empirical evaluation. The experiments cover multiple model families, price tiers, and deployment configurations. The reported detection accuracy is strong, particularly under variations in system prompts, temperature, and retrieval-augmented settings. The partial-routing experiments and real-world API audits further demonstrate the practical relevance of the approach.
Critical weaknesses
-
Problem motivation and practical necessity. While the problem is practically relevant, I feel that the motivation could be further strengthened from a user's perspective. For instance, users may evaluate model quality through direct experience or reduce substitution risks by using official APIs. It would be helpful if the authors could further clarify the scenarios where dedicated auditing is particularly valuable.
-
Novelty and methodological contribution. The framework builds upon several existing techniques, including factual probing, consistency filtering, and statistical testing. While their integration is useful, I am not fully convinced that the advantages of knowledge-boundary fingerprinting over existing approaches are sufficiently established. I would encourage the authors to further clarify the unique contributions and necessity of their method.
-
Scientific insight and limitations. Although the empirical results are promising, I would appreciate a deeper understanding of why knowledge-boundary probes are particularly effective and under what conditions their effectiveness may be limited. Further discussion of factors such as model updates, correlated responses, and adaptive adversaries could help strengthen the analysis and clarify the practical applicability of the approach.
Questions for authors
-
Can the authors provide stronger evidence that knowledge-boundary probes offer an advantage over generic reference-stable, contrastively selected factual questions? In particular, how much of the observed discrimination is attributable to the knowledge-boundary hypothesis itself?
-
How sensitive are the reported false-positive rates and statistical significance levels to correlations among probes? Would a batch-level permutation or bootstrap procedure produce similar conclusions for the primary auditing experiments?
-
How would KBF distinguish an actual model substitution from a legitimate model version update that changes factual recall behavior? How frequently would reference fingerprints need to be regenerated in such scenarios?
-
How robust would the method remain against an adaptive relay operator that attempts to identify auditing requests or selectively forwards knowledge-intensive queries to the genuine reference model?
2 comments
Response on Practicality, Novelty, Probe Correlation, Reference Drift, and Adaptive Attacks (Continued)
4. What if the official APIs get updated and the self-error rates change?
We agree that reference self-error drift is a real concern. KBF assumes a trusted reference backend, but commercial services might change the model behind an endpoint [1][2]. Indeed, the best way to address this issue is to build the reference fingerprint as recently as possible and bind the test result to the fingerprint generation date. That is, we should state the conclusion as “This endpoint is consistent/inconsistent with the fingerprints built on X API at time Y.”
Furthermore, we have empirically shown that most endpoints remained stable (even under silent updates), with calibration allowing for variation. As reported in Appendix D (R5), all 16 models remained consistent through 7 weeks, 14/16 remained consistent at week 9, and 10/11 sampled endpoints remained consistent after approximately 26 weeks. We will clarify that reference endpoints should be retested periodically and fingerprints refreshed when their behavior changes.
5. What if the relay adaptively detects and routes KBF queries to the claimed model while routing other traffic to a cheaper model?
This is a valid concern. Our empirical evaluations provide evidence that KBF can remain effective against basic adaptive routing strategies.
**a) KBF can work without a fixed format, reducing recognizable patterns. Across entity cloze, numerical QA, natural batch QA, and single-question QA, each format achieved TP 10/10 and FP 0/5 across five models (Section 5.4 and Appendix B).
b) Users can mix KBF probes with ordinary queries. We present two approaches to mixing probes and report preliminary experiments on DeepSeek V4 Pro and V4.1 Flash. The main takeaways are: (i) Mixing probes can evade basic attacker detection. When adding a two-turn ordinary conversation as context for each KBF probe batch, all probes passed detectors trained to classify knowledge probing against normal queries. (ii) Mixing probes forces the adaptive router to route user queries to the claimed model, turning adaptive forwarding behavior to the user’s advantage.
Experimental details. (i) We added a two-turn ordinary conversation as context for each single-question QA probe. All 228/228 Pro and 186/186 Flash English probes passed both fixed full-query TF-IDF/logistic-regression detectors trained to distinguish probe-containing queries from normal queries using WildChat [3] and LMSYS-Chat-1M [4]. (ii) We embedded a batch of three probes in each user query and collected their answers from the model response. With each mixing approach, KBF yielded 0/2 FP and 2/2 TP, maintaining detection accuracy. Overall, our preliminary evaluation suggests that KBF offers a more comprehensive and effective defense against adaptive attacks than the baselines discussed in the paper. We will include this analysis in the revision.
[1] DeepSeek. DeepSeek-V3-0324 Release. March 25, 2025.
[2] Google. Continuing to bring you our latest models, with an improved Gemini 2.5 Flash and Flash-Lite release. September 25, 2025.
[3] Wenting Zhao et al. WildChat: 1M ChatGPT Interaction Logs in the Wild. ICLR, 2024.
[4] Lianmin Zheng et al. LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset. ICLR, 2024.
Response on Practicality, Novelty, False-Positive Control, Reference Drift, and Adaptive Attacks
We sincerely thank the reviewer for the constructive feedback. To address these concerns, we performed further empirical analysis and clarified these situations. Please find our detailed responses below.
1. Practicality and necessity of KBF when users can feel or evaluate model quality downgrading.
KBF provides an evidence-based auditing method that ordinary users can run at very low token costs. The auditing result is highly reliable and can detect mixed partial routing even when users do not observe quality degradation from casual inspection of model responses. Moreover, it provides the capability of long-term and real-time monitoring. Based on real-time monitoring, we found that for some ChatGPT accounts, requests explicitly selecting a premium GPT model such as Astra were instead routed to Luna 5.6. This observation highlights the need for reliable, long-term monitoring.
2. Novelty of KBF method and the scientific insight, compared to generic factual question probes with proper screening
The general idea of using factual probes to detect model substitution is valid, and KBF generally follows this framework. The core scientific contribution of KBF is that the knowledge boundary insight provides a method to “efficiently find an abundant number of probes with high discriminative power.” We have provided empirical evidence of KBF’s efficiency compared to generic factual question generation in Appendix H. KBF generation obtained 1.71–3.45 times as many probes as common-fact generation (Table 1), supporting the efficiency gain. By reviewing of probes from four models and archived audit responses, we found that probes at the knowledge boundary accounted for 66.3%–93.3% of cross-model mismatches in the illustrated pairs (Table 2). This supports a key contribution of KBF, which uses stable responses at the boundaries as model identity signals. We will provide more detailed experiments demonstrating this efficiency gain in the revised paper.
Table 1. Retained probes under the same 60-round generation setting. Both approaches use stability and contrastive screening.
Table 2. Contribution to cross-model mismatches. Each cell shows category mismatches / total mismatches for that model pair (percentage). Stable incorrect includes factual errors and/or equivocal answers.
3. Can within-batch probe correlations affect false-positive rates?
This is indeed a valid concern. Although our main approach does not explicitly model probe correlation, it provides a tolerance margin through CP99. We reject only when the one-sided binomial p-value is below 0.05.
Our new FP control experiments yielded 0/60 FPs for each of six models (Table 3), with probe correlations retained. Assuming independent, stable audit rounds, this gives a per-model one-sided 95% FPR upper bound of 4.87%, supporting FP control under the tested conditions. Including the main and September controls, we observed 0/404 FPs in standard audits, plus 0/30 FPs under different wrappers. We will add these analyses to the revision.
Batch bootstrap. We also evaluated a batch-bootstrap alternative, which reduced TP from 155/155 to 154/155 while FP remained 0/16. This suggests that batch resampling may lose detection power by underusing the distinguishing information contributed by individual probes.
Table 3. FP control with fixed probes and calibration. Each model underwent 60 full same-model audits in two collections of 30 rounds.