- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the detection of unauthorized model reuse in black-box ensembles and proposes the REVEAL framework. The method optimizes fingerprint queries to reinforce the target responses of the protected model while reducing the interference of unrelated ensemble members in the aggregation process, thereby enabling ownership verification using only the labels returned by the ensemble. The paper considers both single-label and multi-label classification and empirically demonstrates the effectiveness of REVEAL in ensemble settings compared with conventional fingerprinting methods designed for standalone models. Overall, the topic is practically relevant and the methodological rationale is relatively clear. However, the effectiveness of the proposed approach still depends substantially on the representativeness of the reference models, the assumption of equal-weight probability averaging, and the generalization of the fingerprints to derived models. These issues require further justification.
Critical strengths
(1) Existing model-fingerprinting research has focused primarily on independently deployed standalone models, whereas this paper considers the more challenging scenario in which a protected model is concealed within a black-box ensemble. This extends the scope of research on model ownership verification.
(2) REVEAL develops different reference-model objectives for single-label and multi-label classification according to their respective aggregation mechanisms. Moreover, it performs verification without modifying model parameters and relies only on the labels returned by the ensemble, demonstrating desirable non-intrusiveness and methodological completeness.
Critical weaknesses
(1) The aggregation-neutral responses induced on the available reference models may not transfer to unknown ensemble members in a black-box ensemble. The authors should clarify how the reference models are obtained, state the assumptions required for their use, and evaluate the generalization of the learned fingerprints across different architectures, datasets, and training strategies.
(2) If the reference models assign excessively high probabilities to the target label, an ensemble composed entirely of unrelated models may output that label, increasing false positives. Conversely, if these probabilities are too low, they may overly dilute the protected model’s response, increasing false negatives. The authors should characterize the feasible operating conditions and conduct sensitivity analyses with respect to the decision threshold, target probability, ensemble size, and model-weight distribution.
(3) The optimization simultaneously requires strong responses from the protected model and near-threshold neutral responses from the reference models, and these objectives may conflict. The authors should report the per-query optimization success rate, the distribution of the resulting probability margins relative to the decision threshold, the optimization failure rate, and the stability of the results across different reference-model subsets and random seeds.
(4) The fingerprint queries are optimized only with respect to the original protected model, whereas fine-tuning, pruning, and knowledge distillation may substantially alter the model’s fingerprint responses. The authors should explain why the learned fingerprint signals are expected to survive such modifications and systematically evaluate detection performance under different modification types and modification strengths.
(5) The current framework is primarily developed for equally weighted ensembles. It remains unclear whether REVEAL remains effective under non-uniform model weights or alternative aggregation rules. The authors should analyze the minimum detectable weight of the protected model and its relationship to ensemble size, weight distribution, and fingerprint-signal strength.
(6) Under matched query budgets and false-positive rates, the authors should compare REVEAL with existing non-intrusive fingerprinting methods and backdoor watermarking approaches. They should also report the computational cost of fingerprint generation to demonstrate that the performance gains from aggregation-aware optimization justify its additional overhead.
Questions for authors
(1) How are the reference models obtained, and why can they represent unknown members of a black-box ensemble? In the multi-label setting, is it feasible to reliably control the target probability to just below the decision threshold?
(2) Is REVEAL applicable only to equal-weight probability averaging? How do ensemble size, non-uniform weights, and the protected model’s weight affect detection performance, and what is the minimum detectable weight ?
(3) Why should fingerprints optimized on the original model transfer to variants produced by fine-tuning, pruning, and knowledge distillation?
2 comments
Response (2/2)
4. Response to Concern 4 and Question 3: Generalization to modified models
Regarding fingerprint robustness under model modifications, although fingerprints are generated using the original protected model, common modifications such as FT, PR, NFT, and PFT may preserve part of its learned decision behavior. By reinforcing the protected model's target-specific responses, REVEAL aims to generate stronger ownership signals that can remain distinguishable after such modifications, rather than relying on fragile responses near decision boundaries.
5. Response to Concern 5 and Question 2: Unequal model weights and minimum detectable weight
Impact of non-uniform weight strategy. To further investigate the impact of non-uniform model weights, we conducted additional experiments on CIFAR-100 and PET using five-member ensembles. We varied the reused model's coefficient among 0.1, 0.2, and 0.3. REVEAL consistently achieved the highest AUC across all evaluated settings. Even at the smallest tested coefficient of 0.1, REVEAL achieved AUCs of 0.972 and 0.980 on CIFAR-100 and PET, respectively.
Table R2. Ownership-verification AUC under weighted probability averaging with different coefficients assigned to the reused protected model.
| Dataset | AKH | IPGuard | IrisFP | LiteGuard | REVEAL | |
|---|---|---|---|---|---|---|
| CIFAR-100 | 0.1 | 0.807 ± 0.054 | 0.681 ± 0.073 | 0.715 ± 0.079 | 0.903 ± 0.036 | 0.972 ± 0.008 |
| CIFAR-100 | 0.2 | 0.866 ± 0.033 | 0.739 ± 0.023 | 0.787 ± 0.066 | 0.934 ± 0.016 | 0.988 ± 0.004 |
| CIFAR-100 | 0.3 | 0.879 ± 0.030 | 0.778 ± 0.027 | 0.812 ± 0.058 | 0.943 ± 0.017 | 1.000 ± 0.000 |
| PET | 0.1 | 0.787 ± 0.034 | 0.621 ± 0.082 | 0.834 ± 0.096 | 0.856 ± 0.042 | 0.980 ± 0.002 |
| PET | 0.2 | 0.832 ± 0.024 | 0.683 ± 0.051 | 0.865 ± 0.090 | 0.887 ± 0.038 | 0.995 ± 0.003 |
| PET | 0.3 | 0.851 ± 0.022 | 0.713 ± 0.028 | 0.876 ± 0.065 | 0.915 ± 0.019 | 1.000 ± 0.000 |
Regarding the minimum detectable weight, our experimental results show that REVEAL maintains strong verification performance even at the smallest evaluated aggregation coefficient of 0.1, achieving AUCs of 0.972 and 0.980 on CIFAR-100 and PET, respectively. As expected, AUC gradually decreases as the reused model's weight decreases, reflecting the reduced contribution of its ownership evidence to the aggregated predictions. However, a universal minimum detectable weight is difficult to define, as it depends on the desired verification performance and the specific ensemble configuration.
From a practical perspective, an adversary who incorporates a stolen model into an ensemble typically aims to exploit its predictive capabilities. Assigning an extremely small weight to the stolen model would reduce its influence on the ensemble's final predictions, potentially diminishing the benefit of reusing it. Therefore, reducing the stolen model's contribution to evade ownership verification may introduce a trade-off between verification evasion and the practical utility of model reuse.
6. Response to Concern 6: Baseline comparisons and computational cost
We would like to clarify that REVEAL has already been compared with four existing non-intrusive fingerprinting methods (IPGuard, AKH, IrisFP, and LiteGuard) under the same fingerprint query budget of 100 (Sec. 4.1). The results in Tables 1–3 demonstrate REVEAL's superior verification performance under ensemble reuse.
Regarding backdoor watermarking, such methods typically require modifying the protected model to embed ownership information, whereas REVEAL performs non-intrusive fingerprinting without modifying the protected model. Therefore, these approaches operate under different ownership-verification assumptions and are not directly comparable under the same deployment constraints.
We agree that quantitative profiling is necessary to assess whether the additional optimization overhead is justified. As shown in Table R3, all methods incur relatively low fingerprint-generation costs, with REVEAL achieving competitive efficiency. Moreover, the online verification cost is primarily determined by the number of fingerprint queries. Since all methods use the same query budget, they incur comparable black-box query costs during ownership verification.
Table R3. Fingerprint generation time for different methods under the same query budget ().
| Method | Generation time |
|---|---|
| IPGuard | 1 m 20 s |
| AKH | 35 s |
| IrisFP | 1 m 31 s |
| LiteGuard | 1 m 16 s |
| REVEAL | 55 s |
Response (1/2)
We thank the reviewer for the constructive comments and suggestions. We appreciate the opportunity to clarify the assumptions, design rationale, generalization capability, and practical applicability of REVEAL.
1. Response to Concern 1 and Question 1: Reference-model construction and generalization
As described in Sec. 4.1 and Appendix C, REVEAL uses independently trained ResNet-18 models with different random initializations as references. These models are trained on the same task and dataset as the protected model, without accessing its parameters or outputs, and are used exclusively during fingerprint generation.
Importantly, REVEAL does not require the reference models to share architectures with the actual ensemble members. In our evaluation, unrelated ensemble members are independently and uniformly sampled from nine architectures spanning ResNet, MobileNet, EfficientNet, and DenseNet families, with all evaluation models excluded from fingerprint generation. The strong AUC results across CIFAR-100, PET, VOC, and COCO (Tables 1–3) demonstrate effective verification on ensembles containing unseen models with diverse architectures. Moreover, Appendix B.1 shows that increasing the reference pool from 1 to 4 improves AUC from 0.891 to 0.916 on PET and from 0.872 to 0.891 on VOC, supporting the benefit of reference-model diversity.
While exact aggregation neutrality for arbitrary unseen models is not required or guaranteed, these results establish the effectiveness of reference-based fingerprint generation in the evaluated heterogeneous ensembles. We will further examine unseen models' response distributions and clarify the method's applicability conditions in the revised version.
2. Response to Concern 2 and Question 1: Feasible operating conditions and reference-model target probabilities
We first clarify that reference models are used exclusively during fingerprint generation and are excluded from all testing ensembles. Therefore, their target-label probabilities do not directly determine the predictions of independent ensembles.
For multi-label classification, the target probability is intentionally set slightly below the prediction threshold (ρr=0.45<τ=0.5), aiming to reduce false ownership matches while preserving sufficient target-label evidence when the protected model is present. We agree that this balance depends on the ensemble composition and aggregation weights.
Importantly, our existing experiments already investigate several relevant factors. Appendix B.4 (Fig. 8) evaluates sensitivity to the target probability, while Appendix B.5 (Fig. 9) examines ensemble sizes from 2 to 10. Furthermore, our additional unequal-weight experiments show that REVEAL maintains AUCs of 0.972 on CIFAR-100 and 0.980 on PET even when the protected model's coefficient is only 0.1. These results provide empirical evidence of REVEAL's effectiveness across the evaluated operating conditions. We will clarify these findings and further examine threshold sensitivity in the revised version.
To further assess practical verification reliability, we conducted an ownership-threshold sensitivity analysis over τo∈0.1,0.2,…,0.9, which will be included in the revised version. At the default threshold τo=0.5, REVEAL achieves zero FPR on both CIFAR-100 and PET, with FNRs of 0.020 and 0.189, respectively. Notably, CIFAR-100 achieves zero FPR at τo=0.3, whereas PET requires τo=0.5, illustrating the dataset-dependent trade-off between false positives and false negatives. Together, these results characterize REVEAL's effectiveness across the evaluated operating conditions. We will incorporate these findings and clarify the corresponding applicability conditions in the revised manuscript.
Table R1. FPR / FNR under different ownership-decision thresholds.
3. Response to Concern 3: Feasibility and stability of joint fingerprint optimization
We would like to clarify that the protected-model and reference-model objectives are designed to encourage distinct responses, rather than requiring every fingerprint to exactly reach the prescribed target probabilities. Although these objectives may compete during optimization, REVEAL jointly optimizes them to enhance the separability between pirated and independent ensembles. The strong verification AUCs reported in Tables 1–3 provide empirical evidence that this joint optimization is effective.