- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates active defense for black-box model extraction attack in the data-free setting, where an adversary queries a target classifier and trains a substitute model using the returned hard or soft labels. The objective is to reduce the quality of the extracted model while preserving the normal utility of the victim model. The key insight is that synthetic extraction queries from attackers tend to produce more inconsistent class predictions across the victim model’s intermediate layers than in-domain clean samples. Based on this observation, the proposed method uses the discrepancy between intermediate and final predictions to control how strongly a trained residual head modifies the returned logits or labels. Experiments on CIFAR-10, CIFAR-100, and MiniImageNet show that the method consistently lowers clone accuracy while largely preserving clean accuracy.
Critical strengths
- The paper studies an important security problem in model extraction, which is increasingly relevant as deployed models represent valuable intellectual property.
- The evaluation covers both soft-label and hard-label settings, with experiments across several datasets and models, together with extensive ablation studies.
Critical weaknesses
- The assumed attacker capability is not well justified. The evaluated attacks query the model in a way that is largely separated from normal in-domain data, which may itself be the main source of separability. Specifically, the paper states that "the query distribution Q is unknown to the defender and is not assumed to be out-of-distribution" (Line 193). It is unclear why an attacker cannot query in-domain samples like a normal user, or how the defense affects normal users whose queries are also outside the training distribution.
- The robustness to different extraction query designs is not sufficiently evaluated. Tables 4, 8, and 13 show noticeable variation in the maturity signal and defense performance under different query or coverage constructions. It is unclear whether MGSS would remain effective under a broader range of extraction queries, and whether its advantage over baseline DNF (Luan et al., 2025) would still hold under the DFME variants in Table 4. Moreover, MGSS shows large variation across different coverage datasets in Table 13 (40.61%–52.13%), while the baseline DNF reports relatively small variations across different OOD training datasets (its Table 3), raising concerns about MGSS's sensitivity to defense training data.
- The advantage over prior defenses is not clearly established. Lines 161–166 mainly contrast victim-internal signals with external mechanisms, but it is unclear what limitation of prior methods this resolves. For example, what is the practical drawback of early-exit defenses such as DNF that motivates replacing dynamic routing with a residual head?
Questions for authors
- Can the authors justify why this data-free threat model is the most relevant setting, and provide in-domain extraction evaluation similar to DNF (Luan et al., 2025)?
- If normal users are always accessing the defended API, would an attacker who successfully copies this API already have achieved the goal of model extraction?
- Some of the discussion feels abrupt and not well justified. For example, the research question in Lines 048–053 looks reverse-engineered from the proposed method rather than driven by a clearly identified problem. It treats signals outside the frozen victim as a limitation and directly motivates victim-internal signals, but it is unclear why the former is necessarily problematic.
I am open to increasing my score if most of my concerns are clarified, especially for the assumptions about attacker query distributions (weakness 1) and the practical advantages over prior defenses (weakness 3).
Attacker data access. An attacker can submit in-domain images and receives the same response as any user submitting the same input. We focus on DFME because API access alone can yield an accurate substitute when representative data are scarce. Truong et al. (CVPR 2021) reached 92% of the victim’s CIFAR-10 accuracy with 20M queries and no surrogate dataset. An unknown Q means that MGSS constructs its training targets without estimating the attacker’s query distribution. Generated queries can approach the task distribution, and an attacker can exploit predictions preserved in those regions. Extraction from representative in-domain seeds is a separate data-access setting from the DFME protocols evaluated here.
Utility under benign shifts. We compared the original victim and MGSS on the same 10,000 CIFAR-10 images, using fixed checkpoints and CIFAR-10-C corruptions (Hendrycks & Dietterich, ICLR 2019). Each entry is the change in accuracy, in percentage points, relative to the original victim on the same inputs.
Clean accuracy changed from 95.25% to 94.59%. The mean loss across the six shifts was 5.27 points: 1.17 for brightness and blur, 13.46 for Gaussian noise. Under Gaussian noise at severity 2, accuracy fell from 61.04% to 44.49%, and the mean total variation distance between the two models’ output distributions rose from 0.0230 on clean inputs to 0.4439. This identifies a benign shift that substantially affects both labels and probabilities.
Why use cross-stage evidence? Shaping a response requires deciding how much to adjust it and which competing class should gain weight. Intermediate stages reveal which classes had strong support earlier but lost it by the final layer. Our main-text ablation tests both choices under the same CIFAR-10 DFME protocol. Using final confidence alone to set the adjustment strength yielded 80.59% clone accuracy. Retaining cross-stage strength but directing the adjustment toward the final-logit runner-up yielded 47.81%. Using cross-stage evidence for both yielded 39.35%.
Practical advantage. MeCo jointly optimizes the victim and a perturbation generator, ACT uses pseudo-data to learn an active watermark, and DNF trains intermediate exits and selects one by confidence. MGSS trains one residual head with 1.45% as many parameters as the frozen victim and retains a fixed inference path. On the same RTX 3090, total defense training times over five runs (minutes, mean ± SD) were 3.69 ± 0.02 for MGSS, 4.89 ± 0.06 for DNF, 17.20 ± 0.34 for ACT, and 141.70 ± 11.37 for MeCo. At batch size 64, MGSS processed 5,646 images/s versus 5,297 for DNF, retaining 97.7% of undefended throughput. Across three CIFAR-10 clone architectures, MGSS reduced vanilla DFME clone accuracy by 8.78–14.56 points relative to DNF, with clean service accuracies of 94.46% and 94.34%, respectively.
Query and coverage sensitivity. At the same 20M budget, DFME-Smooth, CA-DFME, and DFME-ISF raised ResNet18-8x clone accuracy against MGSS from 39.35% to 48.19%, 46.94%, and 50.73%. Direct comparisons with DNF cover vanilla DFME and MAZE (Kariyappa et al., CVPR 2021). Under MAZE, clone accuracies were 15.68%, 14.20%, and 14.90% for MGSS versus 22.46%, 21.10%, and 20.94% for DNF. In the fixed-seed coverage study, using only strongly augmented target-task images yielded 52.13% clone accuracy and 94.36% service accuracy. Switching coverage to CIFAR-100 yielded 40.61% and 94.46%. Thus, target-task data alone suffice to train MGSS, while coverage choice materially affects defense strength. DNF’s smaller spread was measured with different coverage sources on ImageNet200/Swin.
Cost of copying the protected API. Successfully reproducing the service constitutes extraction. The practical question is what the attacker obtains for a given query budget. On CIFAR-10, DFME trained a ResNet18-8x clone to over 70% accuracy with 5M queries to the undefended API. Against MGSS, the same attack reached 39.35% with 20M queries, while the service retained 94.46% accuracy. Even at four times the query budget, the clone remained well below the accuracy it had already reached against the undefended API.