- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces ProoFit, a zero-knowledge system for auditing model overfitting without revealing model weights or proving the full training process. It compares a candidate model with an auditor-calibrated reference model using normalized prediction margins and a spectral-complexity measure, with paired statistical tests intended to determine whether the candidate meets the reference’s overfitting threshold. The system proves that the reported margins and complexity were computed from the committed model, using upper-bound certificates for spectral norms and randomized checks for matrix multiplication. The authors evaluate ProoFit on five datasets and six model–dataset settings, reporting true-positive and true-negative rates across independently trained models. They also report proof-generation times of 2.28–161.52 seconds and speedups of approximately 10^3–10^7 over projected zero-knowledge proof-of-training baselines.
Critical strengths
-
Practical and well-motivated problem setting. This paper addresses a practically important problem: auditing model overfitting when model weights and training data are confidential. It clearly explains why proving the full training trajectory is prohibitively expensive, quantifying this challenge with the ResNet18 baseline in Section 3 and Table 1. The three-stage workflow (model commitment, certificate generation, and audit verification) clearly presents the system’s procedure and facilitates evaluation of its potential use cases.
-
Targeted combination of statistical and cryptographic techniques. The proposed method combines complexity-normalized prediction margins with a reference model to construct a relative overfitting test (Sections 4.1-4.2). It uses paired permutation and bootstrap procedures on the same audit examples, while spectral-norm upper-bound certificates and Freivalds’ checks reduce the cost of zero-knowledge verification (Section 4.4). This combination gives a coherent explanation for how the system avoids proving the entire training process, although the statistical implementation requires clarification.
Critical weaknesses
- ProoFit provides a relative audit: it assesses whether the candidate model is no more overfitted than the reference model, rather than directly establishing that the candidate is not overfitted. The audit is therefore only meaningful if the reference model is itself calibrated to an acceptable level of generalization. However, the paper does not sufficiently explain how the reference model is trained. Without the detail, it is difficult to determine whether the reference model provides a reliable acceptance threshold.
- ProoFit audits a trained model by proving computations on audit examples, including its prediction margins and spectral complexity. In contrast, the baseline proves the model’s training process, and its runtime is projected over 100 epochs (Section 5 and Table 1, pp. 8–9; Appendix A.12.2, p. 28). This comparison demonstrates the cost of proof-of-training, but it does not establish a like-for-like speedup over an alternative method for auditing overfitting. The authors should clarify the intended role of this baseline and explain what audit guarantee it provides relative to ProoFit. If the paper claims a direct comparison with conventional overfitting audits, it should include a baseline that performs the same audit task under comparable assumptions and accounts for the cost of proving the relevant computations.
Questions for authors
The paper uses zero-knowledge proofs to prove and verify prediction margins and spectral complexity without revealing private information, and describes optimizations for each. However, complex CNNs, such as ResNet18 in the experiments, involve many nonlinear operations, including ReLU and max pooling. In addition, the output layer of a fully connected network or CNN may use activation functions such as Softmax. Since these operations are nonlinear, they cannot be directly represented as arithmetic circuits during model inference. How does the paper handle these nonlinear operations, and what optimizations does it apply?
Addressing Reviewer Feedback
We thank the reviewers and revise the paper (color blue)
Motivation for auditing Overfitting
Several regulatory and governmental frameworks explicitly recognize generalization as a required audit metric. These include the NIST AI Risk Management Framework, the EU AI Act, European Banking Authority’s guidance and the FDA’s Good Machine Learning Practice principles. Beyond generalization, overfitting also serves as a proxy for privacy leakage, i.e., whether a model reveals information about its training data. The SOTA approach for measuring privacy leakage uses membership inference (MI) tests, which face two limitations in our setting. First, an MI test requires the specific data point x to test -- precisely what our confidentiality constraints prohibit. Second, MI tests can be unreliable in practice [ZDK+24, DSM+24, DZT25, KCC23, MHL+25, AGP25]. Overfitting is a compelling alternative, and prior work establishes a strong connection between memorization and privacy leakage. UK ICO’s AI audit toolkit explicitly recognizes this connection -- ‘implement measures to ensure that AI models are not vulnerable to privacy attacks through overfitting’. Overfitting is also a fundamental measure of a model’s quality and our test can be used where the model itself cannot be shared. A prominent example is an ML model marketplace. A buyer wants evidence that a model generalizes before purchasing it, but the seller is unwilling to reveal the model’s weights or training data.
Baseline
Why PoT? Proof-of-training (PoT) is a natural and necessary baseline in our setting. The conventional approach to auditing overfitting is to empirically measure the generalization gap. However, because the training dataset is hidden from the auditor, a dishonest model owner could substitute an arbitrary dataset and pass the audit, even if the model is severely overfitted. To prevent this, the auditor must verify that the submitted dataset is the actual training set, without compromising confidentiality. PoT is the only known way to establish this guarantee. We now add two new baselines.
New baseline 1: Directional sharpness (Tan et al). This is concurrent work found after our submission. First, this approach is not sound - a dishonest owner can pass the test with a heavily overfitted model. We prove this theoretically (Theorem 4) and confirm it empirically: we get false positive rates of 0.94-0.99 and models with generalization gaps of up to 0.40 still pass (Table 5). In contrast, we formally prove ProoFit's soundness (Lemma 3) and validate it empirically. Second, ProoFit is 7−10× faster, and directional sharpness cannot even run on ResNet-18 (out of memory).
New baseline 2: Unoptimized ProoFit To isolate the effect of the two optimizations introduced in Sec.4.4 for efficient ZK proof generation, we also evaluate an unoptimized variant of ProoFit. Without optimization 1, the naive way of computing the spectral norm requires a ZKP circuit that grows exponentially resulting in 1074× higher cost than our optimized version.
Furthermore, we added performance benchmarks of two more larger image models VGG11 and ResNet50 for a more rigorous testing.
Test Reliability
We analyzed the test errors and made two observations.
Misclassifications are concentrated almost entirely at the decision boundary - candidate models whose generalization gap is very close to that of the reference model. The mean generalization gap of misclassified models is within 0.007 of the reference model’s gap. At this resolution, a model’s training regime (well-fit or overfit) is a weaker indicator of generalization than the gap itself, so some errors are unavoidable. Two reasons make this more pronounced. First, in our original submission, we deliberately kept Daudit small to reduce the cost of proof-of-inference, which limits statistical power for very small differences. Second, our spectral-norm optimization (Sec.~4.4) certifies an upper bound on RW, which makes the test conservative--it errs on the side of flagging a well-fit model as overfitted rather than letting an overfitted one pass. This lowers TPR near the boundary. Moreover, the remaining errors are largely benign. The rejected well-fit models are early, underfit checkpoints whose gap is near zero only because their accuracy is low (test accuracy 0.39-0.72). These are not good-quality models anyway. Conversely, the accepted overfitted models have test accuracy close to that of well-fit models, and in most cases, they outperform the reference model despite their higher spectral complexity. In other words, the test rarely accepts a model that is meaningfully worse than the reference.
A larger audit set substantially improves reliability. We reran the experiments with a larger Daudit. TPR/TNR now ranges from 80% to 100% across all models (Table 3, App. A.12.3).
We welcome any further suggestions for improving the exposition of the paper.