- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The manuscript introduces TwinMark, a watermarking framework designed to preserve watermark information during knowledge distillation. The method embeds the same secret payload into two different output representations: feature second-order statistics and class-conditioned logits. The former targets feature-based distillation, while the latter targets logit-based distillation. The paper also provides theoretical analyses of watermark bit recovery and evaluates the proposed method across several datasets, model architectures, and downstream tasks.
Critical strengths
- The paper focuses on an interesting and urgent problem: protecting the intellectual property of well-trained models.
- The authors investigate multiple student architectures and provide ablation studies on watermark capacity and different watermark removal techniques.
Critical weaknesses
- The idea of embedding watermark information into model outputs or learned representations so that it can survive knowledge distillation has already been extensively explored in previous studies. The proposed TwinMark framework essentially combines two watermarking mechanisms, one operating on feature statistics and the other on logits. However, the underlying idea remains largely consistent with existing distillation-resistant watermarking approaches. The paper does not sufficiently demonstrate why these proposed mechanisms constitute a substantial conceptual advancement over prior work.
- A significant weakness is that the proposed feature watermark can be bypassed through a simple orthogonal rotation of the output embeddings. As reported in Table 2, this transformation reduces the watermark detection rate from 10/10 to 0/10 without affecting classification accuracy. Given that the primary objective of the paper is to protect models against unauthorized distillation, this vulnerability substantially limits the practical security guarantees of the proposed framework.
- The paper does not adequately cover the broader landscape of modern watermarking methods. Beyond parameter-based and trigger-based approaches, recent studies such as NWaaS: A Non-Intrusive and Privacy-Preserving Watermarking-As-a-Service System With Adaptive Resource Scheduling (IEEE TDSC, 2026) have explored non-intrusive, side-channel-based ownership verification. The authors should better position TwinMark relative to these developments. Furthermore, experimental comparisons are largely limited to CosWM, without sufficient evaluation against other relevant distillation-resistant watermarking baselines.
- Although the paper develops theoretical certificates for watermark recovery, the evaluated bounds fail to certify any watermark bits for logit-distilled students. Therefore, the theoretical guarantees only partially support the claimed watermark inheritance across different distillation objectives.
Questions for authors
See weaknesses.
Acknowledgments of Reviewers
We thank all three reviewers for their careful reading of our manuscript and for their constructive feedback. We appreciate the suggestions to evaluate TwinMark on larger benchmarks, across more diverse distillation settings, and against a broader set of baselines. We also acknowledge the concerns about robustness to feature transformations, false-positive calibration, and the limitations of the current logit-channel certificates. Given the limited time during the rebuttal period, we will carry out all requested additional experiments and report the results in a future version of the paper.