- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper extends a hard-label cryptanalytic extraction method to CNNs without requiring prior per-layer widths or convolution/pooling hyperparameters (the input shape is still assumed). Building on dual-point collection and clustering, it infers kernel shape, stride, padding, channel count, and max-pooling parameters from recovered weight vectors. Across five small CNNs, the paper reports exact architecture recovery for 6 of 8 attacked layers. The experiments use model gradients to obtain decision-boundary normals and assume the preceding layers are known.
Critical strengths
- The method removes several architecture assumptions made by prior exact CNN extraction attacks. Inferring convolution settings and channel counts from recovered weights, while extending the attack to max pooling, is a useful contribution.
- The evaluation includes varied cases such as 1D convolution, non-square kernels, folded average pooling and batch normalization, and max pooling. It reports incomplete recovery and runtime as well as successful cases, helping readers judge where the method works.
Critical weaknesses
- The experiments do not establish the full hard-label black-box result. The paper computes decision-boundary normals from model gradients and evaluates layers with the true preceding-layer prefix. These choices are disclosed, and end-to-end extraction is left open, but the results demonstrate layer recovery in an idealized setting rather than a complete label-only attack. This limits the practical claim.
- The study covers five small models and eight attacked layers. Six of the eight layers are recovered with their architecture exactly. The second non-square layer is assigned padding (1, 1), whereas the stated target architecture uses padding (2, 1), so its padding height is inferred as 1 rather than 2. The second LeNet-5 layer recovers only 1 of 16 unique kernels. These results support feasibility on the evaluated examples, but do not establish that the approach scales to larger or modern CNNs or that the full end-to-end attack works under the stated black-box threat model.
Questions for authors
Besides addressing the above mentioned weaknesses, address the following:
- Which steps in the reported experiments use gradients or other internal model access, and which use only hard-label queries? Do you have results for any complete layer using hard-label access alone?
- The supplement mentions each model was evaluated over 10 runs, while the tables show single values. Are these values aggregates or individual runs? Please clarify the aggregation and variability in coverage and dual budgets.
4 comments
Reviewer RK3A - Part 2/3
R3-Q1: White-box access and per-layer results
Response: As explained in the response to R1-W1, we have refined our attack to allow for full black-box end-to-end extraction, and thus no steps use gradients or any white-box access. We successfully extracted all layers with our refined attack, including convolutional, fully connected, and the model heads. Table 3, shown below, reports per-layer results for complete end-to-end black-box extraction in the hard-label setting.
Table 3: Per-layer extraction metrics. Coverage is the fraction of a search space S on which no unrecovered effective weight is used (Liu et al., 2026). The reported number of duals is the number of additional duals required to extract the target layer.
| Model | L | Coverage | Max Weight Error | Max Bias Error | Duals (#) | Dual (s) | Hyperparameter (s) | Signature (s) | Sign (s) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1D | 0 | 6/6 | 100% | 10K | 34 | 0.8 | 0.34 | 0.05 | ||
| 1 | 1/1 | 100% | 0 | 0 | 1.5 | 0.01 | 3.4 | |||
| 2 | 10/10 | 100% | - | 1.1 | 0.48 | 0.22 | 2.0 | |||
| Non-Square | 0 | 4/4 | 100% | 10K | 67 | 1.3 | 0.11 | 0.14 | ||
| 1 | 1/1 | 100% | 0 | 0 | 5.6 | 0.11 | 4.2 | |||
| 2 | 10/10 | 100% | - | 7.5 | 0.15 | 3.9 | 16 | |||
| Folded | 0 | 3/3 | 100% | 0.0 | 10K | 130 | 1.6 | 1.4 | 0.06 | |
| 1 | 3/3 | 100% | 0 | 0 | 55 | 0.19 | 132 | |||
| 2 | 10/10 | 100% | - | 8.1 | 0.53 | 21 | 251 | |||
| 2max-0fc | 0 | 3/3 | 100% | 10K | 114 | 9.4 | 0.26 | 0.09 | ||
| 1 | 3/3 | 100% | 10K | 248 | 3.1 | 0.54 | 4.6 | |||
| 2 | 10/10 | 100% | - | 8.4 | 0.34 | 0.91 | 14 | |||
| 2max-1fc | 0 | 3/3 | 100% | 10K | 136 | 11 | 0.30 | 0.08 | ||
| 1 | 8/8 | 100% | 10K | 303 | 12 | 0.93 | 2.5 | |||
| 2 | 55/64 | 99.96% | 530K | 14996 | 90 | 16 | 7.7 | |||
| 3 | 10/10 | 99.96% | - | 7.7 | 1.1 | 0.05 | 125 | |||
| LeNet-5 | 0 | 6/6 | 100% | 20K | 2139 | 106 | 0.85 | 0.23 | ||
| 1 | 16/16 | 100% | 480K | 82382 | 1265 | 17 | 294 | |||
| 2 | 96/120 | 99.84% | 1.2M | 54489 | 481 | 197 | 17 | |||
| 3 | 65/84 | 85.61% | 2.8M | 77178 | 481 | 115 | 29 | |||
| 4 | 10/10 | 85.79% | - | 8.3 | 0.5 | 57 | 248 |
To help understand the numbers in Table 3, consider LeNet-5, where 96/120 neurons in L2 and 65/84 neurons in L3 were successfully extracted, achieving coverage (the fraction of a search space on which no unrecovered effective weight is used) of 99.84% and 85.61%, respectively. Most of the remaining neurons are dead or near-dead. Dead neurons do not contribute to the model's input-output behavior and thus have no impact on extraction fidelity. Near-dead neurons are activated only rarely and have limited impact on fidelity. Since our extraction process stops once the desired fidelity (99%) is achieved, some near-dead neurons remain unextracted, but they could be recovered with additional extraction time. Consequently, despite incomplete neuron recovery, the extracted LeNet-5 achieves 99.84% fidelity.
Max error generally grows with layer depth most visibly in 2max-1fc and LeNet-5, which contain FC layers after the initial convolutional layers. In 2max-1fc, 9 dead or near-dead neurons are missing in L2, lowering coverage from 100% to 99.96%, and increasing the error in the recovered head. For LeNet-5, 43 dead or near-dead neurons are missing in L2 and L3 combined, with high max error (0.1) and low coverage (<86%) across the last few model layers. The overall effect error accumulation recovered model is small, with both models extracted to high fidelity (>99%). Metrics for full end-to-end model extraction are reported in Table 4. Since coverage is measured as the fraction of the search space for which no unrecovered weight is used, low coverage is not a direct indicator of low fidelity.
Revision: We removed all references to knowledge of previous layer parameters and model gradients, and we removed end-to-end from future work. We updated Section 5.2 and Table 3 with per-layer results for each recovered model.
Reviewer RK3A - Part 3/3
R3-Q2: Multi-run aggregates
Response: We thank the reviewer for identifying this inconsistency. The statement that each model was evaluated over 10 runs was inadvertently retained from an earlier version of the manuscript and has now been corrected. The results reported in the main tables correspond to individual extraction runs, rather than averages across multiple runs. Therefore, the reported coverage and dual-point budgets are measurements from those individual runs, not aggregated statistics. To further examine variability, we have expanded our evaluation to include additional training configurations (Table 7 varies the seeds, Table 8 compares trained versus untrained models) of Folded and 2max-0fc, with the corresponding results reported in the Appendix. These additional experiments provide further insight into how extraction coverage and dual-point requirements vary across training configurations.
Table 7: Varied training seeds.
| Model | Seed | Duals | Dual Time (s) | Queries |
|---|---|---|---|---|
| Folded | 42 | 50K | 9531 | |
| 64 | 10K | 105 | ||
| 23 | 10K | 118 | ||
| 17 | 10K | 120 | ||
| 83 | 10K | 136 | ||
| 2max-0fc | 42 | 60K | 5928 | |
| 64 | 20K | 859 | ||
| 23 | 20K | 387 | ||
| 17 | 30K | 2371 | ||
| 83 | 450K | 57534 |
The training seed occasionally affects the number of dual points and the time required to recover each model. For 2max-0fc, seed 83 required 450K duals to fully recover the second Conv+Pool layer. While extracting this specific layer, one kernel contained a low rate of dual RPCPs, which are required for a bias solution. After 450K dual points, this kernel contained two dual RPCPs, which is the minimum for a valid cluster. This indicates that neuron placement of trained models affects attack efficiency, especially in Conv+Pool layers, but not necessarily attack success.
Table 8: Trained versus untrained models.
| Model | Trained? | Fidelity | Duals | Dual Time (s) | Queries |
|---|---|---|---|---|---|
| Folded | No | 99.66% | 50K | 9531 | |
| Yes | 100% | 10K | 105 | ||
| 2max-0fc | No | 99.92% | 60K | 5928 | |
| Yes | 100% | 20K | 859 |
Table 8 reports a similar trend. The randomized neuron placement in the untrained models required more dual points to recover than the trained models. Additionally, the untrained models were extracted with lower fidelity, solely due to errors in the model head. Since these errors are within our fidelity threshold (>99%), we consider each model successfully extracted.
Revision: We added Appendix F and Table 7 with experiments showing varied training seeds. We added Appendix G and Table 8 with experiments showing trained versus untrained model extraction.
Thanks
Dear Reviewer,
With the discussion period nearing its conclusion, we would like to express our sincere appreciation for your time and thoughtful feedback. We would greatly appreciate hearing whether our responses and additional experiments have adequately addressed your concerns. Should any questions or concerns remain, we would be happy to provide further clarification.
Thank you again for your valuable time and consideration.
The Authors
Reviewer RK3A - Part 1/3
Thank you for your thoughtful feedback and constructive suggestions. We appreciate your insights and address your questions and concerns below.
R3-W1: Not full black-box end-to-end extraction
Response: Inspired by your insight, we have refined our attack to allow for full black-box end-to-end extraction in the hard-label setting. Our experiments no longer require perfect knowledge of previous layers, model gradients, or ground-truth neuron signs. We updated our evaluation on six representative models: 1D (2 Conv layers), Non-Square (2 Conv layers), Folded (L0: Conv, L1: AvgPool->Conv->BatchNorm), 2max-0fc (2 Conv->MaxPool layers), 2max-1fc (2 Conv->MaxPool followed by 1 FC), and LeNet-5 (2 Conv->MaxPool followed by 2 FC). All models are trained on MNIST with an input size of (1x28x28), except for 1D (3x80, normal distribution). We successfully extracted each model end-to-end to high fidelity without any white-box knowledge.
Table 4: End-to-end extraction results.
All models were recovered with high fidelity (>99%), with 5/6 models recovered with 100% fidelity. The missing 0.16% fidelity in LeNet-5 is caused by missing near-dead neurons in its fully connected layers.
Revision: We added Section 5.3 and Table 4 with end-to-end extraction results for each model. We removed all references to knowledge of previous layer parameters and model gradients, and we removed end-to-end from future work.
R3-W2: Larger models
Response: To evaluate our approach on more complex CNNs and beyond image classification, we added experiments with variants of the HiDDeN decoder (HiDDeN-4, HiDDeN-8), a CNN designed to recover hidden messages from images, trained on CIFAR-10. These models have nine layers (eight hidden Conv->BN->ReLU layers followed by the model head), almost twice the depth of LeNet-5, and larger inputs (3×32×32 vs. 1×28×28).
Table 5: CNN Extraction Beyond LeNet-5.
We successfully extracted all nine layers of HiDDeN-4 using approximately 500K dual points and the first layer of HiDDeN-8 using approximately 57K dual points within the past few days. For the three wider variants (HiDDeN-16, HiDDeN-32, and HiDDeN), our method also identified dual points with usable ASVs, with retention rates ranging from 41% to 63%. Given the computational and time cost of dual-point collection, we prioritized completing the extraction of HiDDeN-4 and extending layer recovery to HiDDeN-8, while evaluating dual-point discovery across all five variants. These results extend our demonstrated extraction capability beyond LeNet-scale image classifiers and provide initial evidence of its applicability to CNNs in other domains. The experiments across increasingly wider HiDDeN variants further establish a concrete direction for scaling our approach to more complex architectures. Although scaling to significantly more complex architectures remains an open challenge to the community, we believe that our approach lays a solid foundation. We have incorporated new experimental results and carefully calibrated our claims regarding model size and complexity to align with the evidence.
Revision: We updated Appendix B with the details of each HiDDeN variant. We added Section 5.4 and Table 5 with extraction of larger model sizes.