acceptodds
Under review as a conference paper at ICLR 2027

Accelerating End-to-End Cryptanalytic Extraction via Geometric Critical Point Screening

Abstract

Cryptanalytic model extraction recovers neural-network parameters from prediction queries, exposing information that model owners seek to protect in prediction systems. Layerwise methods often spend many queries on critical points from unrelated layers, while parameter errors propagate into subsequent recovery stages. To address these bottlenecks, we propose Geometric Critical Point Screening (GCPS) within an end-to-end framework for layerwise parameter recovery. GCPS checks geometric relations among critical points before measuring shallow-layer signatures, reducing the number of candidates requiring costly directional measurements. Numerical refinement and coordinate alignment prepare these shallow estimates for subsequent layers, where the full-prefix Jacobian identifies reachable measurement directions. Guarded low-dimensional measurements reduce directional queries, while native input-space repair, cross-region clustering, and completion handle unreliable checks and partial observations. Experiments compare independent true-prefix recovery and end-to-end extraction on a public FP64 ReLU checkpoint, supplemented by shallow MNIST and CIFAR comparisons. Both protocols use truth-assisted signs and native auxiliary truth, with end-to-end costs summed over selected completed or resumed stages. Relative to the reference implementation, our end-to-end procedure achieves 1.94× query and 2.00× recorded layer-time speedups under this accounting. Both end-to-end methods verify 6593 of 6643 effective weights. Independent true-prefix recovery achieves a 1.85× query speedup and verifies 6607 effective weights, compared with 6603 for the reference under the same evaluation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Cryptanalytic Extraction, Neural Network Model Extraction, Geometric Critical Point Screening, End-to-End Recovery, Query Efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XkQjOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

In this paper, a new method is proposed called Geometric Critical Point Screening (GCPS). My understanding is the purpose of this method is that it is a form of model extraction. Weights from a network are recovered layer by layer using the proposed technique. Empirically the method is tested on a public checkpoint, MNIST and CIFAR-10 models. The main strength of this method is that it requires 2x less queries and operates in less time.

Critical strengths

I like the way the introduction spaces the development of this work chronologically into paragraph with corresponding years given. It makes the intro easy to follow.

The experimental results are presented in tables clearly.

Critical weaknesses

There are two major issues with the work:

Issue #1: As this work deals with model extraction, I would consider it a subfield of adversarial machine learning. The authors are free to correct me if this assumption is wrong. If this assumption is correct, a major flaw in the paper is that setup is extremely poorly defined. There is no adversarial threat model to speak of. This is unacceptable for any kind of machine learning security paper. Specifically, section 2 does not clearly define the powers of the attackers or the defender. This is critical because if this is a model extraction attack, right out of the gate the practical scenario is unclear.

#Issue 2: Experimental Issues. In this paper, really only one measurement is important in my opinion, the number of queries. The time it takes to do one query is entirely hardware dependent because a query is based on the model, whose inference time will be based on the GPU being utilized. Therefore, it does not seem to be very honest to report a 1.85x query speed up because the results will change when the hardware changes. Query count on the other hand is hardware independent so it will not change. I would prefer the authors DO NOT falsely advertise query time speed up in the abstract and move these results to the supplementary material.

A second issue is that the authors only compare with ONE other method (called Liu or Liu’s). Maybe I overlooked it in the writing, but I am not really sure what the justification is for only including ONE other model extraction technique. I am not an expert in this domain, but as there are an abundance of ML techniques and algorithms available, it is suspicious (e.g. hints at cherry picking) when only one other technique is used. It would be great if the authors could address this or if time permits, possibly include comparisons to other techniques.

Minor points:

=I do not understand why a random assortment of words is italicized in proposition 1.

=There is no figure that overviews the setup or illustrates your method. It makes thing much harder to follow. I would strongly suggest including an overview figure at the start of the paper to make a nice snapshot available to the reader. Remember not everyone who read your work (this reviewer included) are experts in your domain.

=I was very confused by section 4.5 concepts. Is it possible for you to elaborate more on that section, or possibly point the readers to an appendix section where you give more information? The current description is completely inadequate.

Questions for authors
  1. Can you improve the writing to include an adversarial threat model, put in a figure that overviews the setup and fix the setup writing?

  2. Can you either have a better analysis or justification about why only one other technique is compared to yours? Can you also improve the experiments according to the comments I made in the weaknesses section?

1 comment
AuthorsOpenReview ↗

Response to Reviewer XkQj

Q1. Are the threat model and the information available during the experiments clearly specified? Section 2 and Appendix A.3.1 define the threat model. The model owner retains the parameters and exposes raw FP64 outputs through a prediction interface. The attacker knows the fully connected ReLU architecture, can query the model with arbitrary real-valued input vectors, and seeks to recover its parameters. Section 2 also specifies the use of ground truth for sign recovery, the auxiliary ground-truth information used by the native procedure, and the two prefix protocols. Under the ground-truth-prefix protocol, the correct prefix is supplied at each layer. Under the end-to-end protocol, each stage receives the parameters recovered in preceding stages. The query-efficiency results correspond to the assistance protocol specified for each experiment. Q2. Does the reported factor of 1.85 conflate query savings with runtime speedup? Table 2 in Section 4.3 reports query count and local runtime separately for the two methods under consistent experimental settings. For each metric, the ratio is the baseline value divided by the value for our method. In Table 2, the query-count ratio is 244,026,401 / 132,197,083 ≈ 1.85, and the corresponding local runtime ratio is 1.04. In Table 3 of Section 4.4, the query-count ratio is 1.94 and the local runtime ratio is 2.00. Thus, the factors of 1.85 and 1.94 quantify query savings, while 1.04× and 2.00× are the corresponding observed runtime speedups. As specified in Section 4.1 and Appendix D.1, query count is the primary efficiency metric. Local runtime is a secondary metric that includes both prediction and recovery computations. Q3. Why did you choose Liu et al. as the direct experimental baseline? The framework of Liu et al. directly matches our experimental setting in its deep ReLU architecture, raw-output interface, layerwise parameter recovery objective, and eight-hidden-layer configuration. Section 1 describes how their framework handles rank deficiency in deeper layers, cross-layer interference, merging, and precision refinement. Section 4.1 and Appendix D.1 identify the public checkpoint and research code used in our experiments. We retain their cross-region clustering and completion procedures while modifying candidate screening and local measurement. This baseline therefore allows us to measure the query savings from those changes within the existing deep-network extraction framework. Tables 2 and 3 report the comparisons under the ground-truth-prefix and recovered-prefix protocols, respectively. Q4. Where can readers find an overview of the method and an explanation of Section 4.5? Figure 1 on page 3 illustrates the L1 midpoint test and the L2 collinearity test. Section 3 presents the full pipeline, covering screening, refinement and coordinate alignment, local measurement using the prefix, repair and completion, and output-layer fitting. Section 4.5 explains the relationship between query cost and local runtime and directs readers to the supporting records. Appendix D.1 defines the accounting rules. Appendix D.5 explains how to regenerate the tables and audit the recovered parameters. Appendix D.8 documents the execution branches taken and the stages from which recovery resumed. Together, these records allow readers to trace the reported query savings to the procedures actually executed. Q5. What do the italics in Proposition 1 signify? The italics are the typesetting convention used for the body of a proposition. The mathematical conditions are stated in the hypotheses of Proposition 1 and in Equation (2) of Section 3.1. Under those conditions, the points at which a common hyperplane intersects equally spaced parallel sampling lines satisfy the midpoint identity.

Reviewer_d8s2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the extraction of network parameters from fully connected ReLU networks by analyzing the raw FP64 numerical outputs. Building upon the layer-by-layer extraction framework proposed by Liu et al., the authors introduce a technique called "Geometric Critical Point Screening" (GCPS). This method leverages midpoint and collinearity relationships to filter critical points in shallower layers before performing computationally expensive feature measurements. Furthermore, they refine the reconstructed hyperplanes by incorporating additional local roots. For deeper layers, they utilize the Jacobian matrices of the previously fully reconstructed layers to conduct measurements within a low-dimensional subspace; this process encompasses feature reuse, analysis of independently reachable directions, result verification, and direct correction within the input space. The clustering and completion procedures employed herein follow the methodology established in the aforementioned framework.

Critical strengths

This approach targets a meaningful query bottleneck and reports substantial savings. Screening candidates before measuring full normals is well motivated when the input dimension greatly exceeds the hidden width.

Critical weaknesses
  1. A clearer technical comparison with the geometric layer-identification ideas in Rolnick and Kording (ICML 2020), already cited by the paper, would also clarify the specific advance of screening before normal measurement.

  2. The empirical evidence does not adequately isolate the proposed contributions or establish robust performance. The reported procedure combines GCPS, refinement, compact measurements, signature reuse (A), independent-direction measurements (B), and native repair. Section 4.4 acknowledges the absence of comparisons with matched prefixes and search schedules. The deep evaluation also uses one checkpoint and one seed. Furthermore, the selected L7/L8 stages disable A/B and use whole-layer native measurements (Appendix D.8); their query counts are almost unchanged between methods, while their reported times differ by approximately 1.51-fold and 2.06-fold. Thus, the total improvements do not identify the separate contributions or establish the stability of the runtime claim.

  3. The recovery chain selected in the paper offers very limited evidence for achieving a complete, independently reproducible, end-to-end extraction. The auxiliary measures disclosed in the paper constitute explicitly defined experimental constraints rather than covert flaws. Nevertheless, the primary result is, in effect, a composite of several completed or recovered stages, which also incorporates "target-assisted correction" applied to the preserved L7-layer orientation. Consequently, the result does not demonstrate a complete, uninterrupted execution and recovery process carried out under a fixed, consistent workflow.

Questions for authors
  1. How were the stages in Table 5 selected? Which repair and continuation decisions were fixed before observing outcomes, and which were made afterward? Please clarify the decision and accounting for the L7 orientation correction and the subsequent L8/L9 continuation.

  2. Do existing logs separate the queries spent on candidate collection, geometric screening, normal measurement, validation, refinement, and native repair? If available, a concise breakdown would help assess how the savings relate to the proposed mechanisms.

  3. What metric, evaluation inputs, sample size, and numerical value underlie the held-out output-fidelity shortfall mentioned in Section 4.4? Are comparable measurements already available for both recovered models?

  4. Were hardware, numerical-library threading, code versions, and timing boundaries matched for L7/L8? What explains their time differences despite nearly equal query counts? Which currently available artifacts permit independent verification of Table 5?

2 comments
AuthorsOpenReview ↗

Response to Reviewer d8s2

Q1. Advance over Rolnick and Kording Rolnick and Kording use boundary geometry for reconstruction. We screen critical-point locations before querying full normals (§§3.1, 3.3). L1 uses a midpoint identity for parallel-line intersections. L2 tests collinearity after a common lifting where all recovered L1 units are active. We measure and verify only retained signatures. Eq. (5) gives net query savings. Q2. Modules and robustness Baseline/ours query ratios are 6.84, 15.97, 3.98, 5.24, 4.35 and 1.58 for L1–L6, and ≈1 for L7–L8 with native measurement (Table 5, §4.4). Savings concentrate where screening and compact measurement apply. For L1–L2 combined, the ratio is 11.58 with 6,328 effective weights verified per method. Tables 8–9 (D.8) trace settings, calls and fallbacks for GCPS, full-prefix measurement, signature reuse (A), independent directional measurement (B) and native repair. The deep comparison fixes the checkpoint and seed 42 (§4.1). Table 1 varies shallow-network input dimension and hidden width. Q3. Recovery chain Each layer uses the recovered prefix, including L2 using recovered L1 (§2). Table 8 (D.8) gives stages and version hashes for L1–L6 (R1), L7 (R2) and L8–L9 (R3). After an L7 sign-hook error, we applied the stated ground-truth-assisted sign protocol. We negated column 0 of the saved L7 matrix before recovering L8–L9 and recorded pre- and post-correction hashes. Each retained layer and selected repair is charged once (D.1, Table 5 note). Q4. Query accounting Each target input row costs one query, so a batch of b costs b (D.1). Table 5 counts acquisition, screening, measurement, verification, refinement, repairs, failed attempts and fallbacks per layer within selected stages. Table 9 traces overlapping module calls, successes and fallbacks. Per-layer records determine costs. §4.4 separates shallow and deep totals. Q5. Fidelity and parameter verification In our Liu et al. reproduction, 0% of 2,048 standard normal inputs and 7.4% of 10,000 uniform inputs on [-1000,1000]^784 met the relative output error threshold of 2.96 × 10^-5. The respective 75th percentiles were 0.0709997 and 0.00157673 (§4.4, D.4). Against the same checkpoint, both methods verified 6,593/6,643 effective weights and 54/65 biases, giving a baseline/ours query ratio of 1.94 at matched counts (Table 5). Q6. L7/L8 settings and timing Both methods use the same checkpoint, FP64 and seed 42 (§4.1, D.1). D.8 lists stage code versions. A and B were off for L7/L8. Stage metadata give their environments. D.6–D.7 detail hardware and thread controls for paired shallow runs. The 1.51× and 2.06× stage speedups measure total local runtime (prediction and recovery). The 1.94× ratio measures target queries. §4.5 details calibration, completion and linear algebra costs. D.5 reproduces costs from frozen CSVs, source hashes and a deterministic table script. Saved parameters, target checkpoint and audit script allow independent parameter checks.

I thank the authors for their detailed response. Their reply addresses my primary concerns, and I will maintain my rating.

Reviewer_vAhsOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper is focused on weight extraction/weight stealing. Experiments here focus soley on fully-connected ReLU nets (on order of.60k parameters) and assumes the architecture is know to the attacker. The work primarily builds off Liu et. al. 2026, and demonstrates notable improvements both in terms of number of queries and wall-time to extract the network weights.

Critical strengths
  • A notable improvement over preceeding work (Liu et. al 2026) is demonstrated, with 1.94x fewer queries required.
  • Extraction of network weights is a relevant and interesting area of research, and the motivation is clear.
  • Writing is clear, and the paper is complete - I felt no details of the approach were missing.
Critical weaknesses
  • Focusing on ReLU-only FC layers is a very limiting choice as there are no such networks that are valuable enough to motivate extraction, hence making the work highly limited in it's applicability.
  • Focusing on small networks only further limits practicality. If there was an established scaling law linking cost to model size, this would be alleviated.

Overall due to the above, despite the paper being clear and well written, I find the applicability so limited that I recommend reject.

Questions for authors

Did you ever examine transformer-like architectures for this setting? Presumably, the lack of discontinuity the relu's provide would make this approach non applicable, however I am curious if this is not the case.

1 comment
AuthorsOpenReview ↗

Response to Reviewer vAhs

Q1. Why study fully connected ReLU networks? The piecewise-affine structure of fully connected ReLU networks allows us to analyze, layer by layer, how activation boundaries relate to parameter recovery and query cost. We exploit this structure to reduce the number of queries required to extract the parameters of deep networks. Section 1 describes the risk of parameter leakage through prediction interfaces. Table 3 in Section 4.4 quantifies the query savings. Under the stated assistance protocol, 6,593 of 6,643 effective weights pass verification for each method. Our method reduces the query count from 164,584,812 to 84,645,021, a reduction of 48.6%. Q2. How does extraction cost depend on model size? Extraction cost depends jointly on the input dimension, the rank of the network prefix, the cost of verification, and the proportion of cases requiring the native repair procedure. Equation (17) in Section 3.5 relates these quantities to the cost of the measurement stage. Equation (5) in Section 3.1 accounts for the cost of geometric screening when calculating net query savings. Table 1 in Section 4.2 compares input dimensions of 784 and 3,072 and hidden-layer widths of 8, 16, and 32. Section 4.4 examines a recovery chain spanning eight hidden layers. Together, these experiments and the measurement-cost analysis explain where the query savings arise. The method is particularly well suited to settings in which the input dimension substantially exceeds the prefix rank. Q3. How does the method relate to Transformers? Our contribution is an algorithmic improvement to an existing parameter extraction framework for deep, fully connected ReLU networks. Candidate screening and local measurement reduce query cost and accelerate extraction. Section 2 and Appendix A.3.1 specify the scope of the method. As Equation (4) in Section 3.1 shows, the method exploits the gradient jump across an activation boundary. The ReLU activation is continuous in value, while its gradient can differ on the two sides of the boundary. Extending parameter extraction to Transformers would require deriving local identification conditions for attention and the other nonlinear operations in that architecture.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.