Steal the Knowledge, Inherit the Mark: TwinMark for Distillation Watermarking via Second Moments and Class Multiplexing

Knowledge distillation can copy a deployed model by training a student to imitate its logits or features. A watermark can therefore survive only through the output that the student learns to reproduce, yet most distillation watermarks protect only one such surface. TwinMark instead embeds a shared secret payload into the statistics preserved by both objectives: the second moment of normalized features and class-conditioned mean logits. Both channels admit linear readouts with sufficient conditions for bit recovery on fixed audit inputs, while statistical detection is evaluated separately under a declared null. In a ten-seed CIFAR-100 study, each watermark is detected in every student that imitates its corresponding output and remains at chance under the unmatched objective. Marking reduces teacher accuracy by 1.3 points, while the audited embedding of feature-distilled students loses 4.4 nearest-neighbor accuracy points. Second-moment bounds certify 50–60 of 64 feature bits in every feature-distilled student, whereas pointwise feature bounds and the evaluated logit bounds certify none. Class multiplexing, which signs the payload differently across classes, increases the logit decoder's rank ceiling and reduces bit error at matched signal energy. Even at approximately $10\times$ over-encoding, a 1,024-bit payload over 100 classes remains detectable in every evaluated logit-distilled student. Detection further extends to 39 of 40 students across four additional architectures and, through the feature channel, to segmentation, object detection, and satellite-navigation jammer detection and classification. These results establish output-matched watermark inheritance under the evaluated distillation protocols, rather than resistance to all utility-preserving transformations.