acceptodds
Under review as a conference paper at ICLR 2027

ProoFit: Confidential Auditing of Model Overfitting

Abstract

As machine learning models are increasingly deployed in high-stakes settings, auditing their quality becomes essential. A critical property of interest is whether the model is overfitted. Yet direct audits are often impossible: model owners cannot reveal model weights without risking exposure of proprietary IP. While zero-knowledge proofs, a cryptographic primitive, offer a compelling theoretical solution, naive approaches require proving the correctness of entire training trajectories–an operation that is prohibitively expensive for modern models. In this paper, we introduce ProoFit, a practical system for auditing overfitting without violating model confidentiality. Rather than proving properties of the training process, ProoFit employs novel tests that depend only on trained model weights, leveraging theoretical connections between generalization and model complexity. Our experiments demonstrate that ProoFit significantly outperforms baseline cryptographic approaches, achieving a to improvement in runtime.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Test of overfitting, Model auditing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_hQAcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ProoFit, a zero-knowledge system for auditing model overfitting without revealing model weights or proving the full training process. It compares a candidate model with an auditor-calibrated reference model using normalized prediction margins and a spectral-complexity measure, with paired statistical tests intended to determine whether the candidate meets the reference’s overfitting threshold. The system proves that the reported margins and complexity were computed from the committed model, using upper-bound certificates for spectral norms and randomized checks for matrix multiplication. The authors evaluate ProoFit on five datasets and six model–dataset settings, reporting true-positive and true-negative rates across independently trained models. They also report proof-generation times of 2.28–161.52 seconds and speedups of approximately 10^3–10^7 over projected zero-knowledge proof-of-training baselines.

Critical strengths
  1. Practical and well-motivated problem setting. This paper addresses a practically important problem: auditing model overfitting when model weights and training data are confidential. It clearly explains why proving the full training trajectory is prohibitively expensive, quantifying this challenge with the ResNet18 baseline in Section 3 and Table 1. The three-stage workflow (model commitment, certificate generation, and audit verification) clearly presents the system’s procedure and facilitates evaluation of its potential use cases.

  2. Targeted combination of statistical and cryptographic techniques. The proposed method combines complexity-normalized prediction margins with a reference model to construct a relative overfitting test (Sections 4.1-4.2). It uses paired permutation and bootstrap procedures on the same audit examples, while spectral-norm upper-bound certificates and Freivalds’ checks reduce the cost of zero-knowledge verification (Section 4.4). This combination gives a coherent explanation for how the system avoids proving the entire training process, although the statistical implementation requires clarification.

Critical weaknesses
  1. ProoFit provides a relative audit: it assesses whether the candidate model is no more overfitted than the reference model, rather than directly establishing that the candidate is not overfitted. The audit is therefore only meaningful if the reference model is itself calibrated to an acceptable level of generalization. However, the paper does not sufficiently explain how the reference model is trained. Without the detail, it is difficult to determine whether the reference model provides a reliable acceptance threshold.
  2. ProoFit audits a trained model by proving computations on audit examples, including its prediction margins and spectral complexity. In contrast, the baseline proves the model’s training process, and its runtime is projected over 100 epochs (Section 5 and Table 1, pp. 8–9; Appendix A.12.2, p. 28). This comparison demonstrates the cost of proof-of-training, but it does not establish a like-for-like speedup over an alternative method for auditing overfitting. The authors should clarify the intended role of this baseline and explain what audit guarantee it provides relative to ProoFit. If the paper claims a direct comparison with conventional overfitting audits, it should include a baseline that performs the same audit task under comparable assumptions and accounts for the cost of proving the relevant computations.
Questions for authors

The paper uses zero-knowledge proofs to prove and verify prediction margins and spectral complexity without revealing private information, and describes optimizations for each. However, complex CNNs, such as ResNet18 in the experiments, involve many nonlinear operations, including ReLU and max pooling. In addition, the output layer of a fully connected network or CNN may use activation functions such as Softmax. Since these operations are nonlinear, they cannot be directly represented as arithmetic circuits during model inference. How does the paper handle these nonlinear operations, and what optimizations does it apply?

Reviewer_nnXcOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ProoFit, a framework for auditing model overfitting without the need of model weights or training data. The main motivation is that traditional zero-knowledge proofs of the entire training process are computationally expensive and difficult to scale. Experiments on several datasets and different architectures demonstrate reductions in proof generation time compared with proof-of-training baselines.

Critical strengths
  1. Clear motivation and practical relevance. The paper addresses a practical problem: how to audit proprietary models without sensitive information. The motivation is intuitive, and the idea of avoiding full training verification is interesting.

  2. Promising computational efficiency. The reported efficiency improvements are significant. ProoFit generates a proof for ResNet-18 on CIFAR-10 in under three minutes, while the cost of proving the full training process is much higher. The implementation across different architectures provides useful evidence.

Critical weaknesses
  1. Table 3 shows relatively limited performance in several settings. For example, the true negative rate is only 60% for ResNet-18 on CIFAR-10 and 62.5% for SimpleCNN on FMNIST. Since identifying overfitted models is the main goal, these results raise questions about how consistently the test can be applied in practice.
Questions for authors
  1. Table 3 shows different detection rates across datasets and architectures. Could the authors discuss what might explain the relatively low true negative rates on FMNIST and ResNet-18? Are there particular settings in which the proposed test is expected to be more reliable?

  2. I found some mathematical expressions in Section 4.2 a little difficult to follow. Some formulas appear quite small and densely formatted, making them harder to read. Could the authors improve the formatting and provide a slightly clearer explanation?

Reviewer_SiSvOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on auditing the overfitting of ML models and proposes ProoFit, a practical system for auditing overfitting without violating model confidentiality. Existing membership inference tests require access to private training data. In comparison, ProoFit uses zero-knowledge proof and reduces the computational cost by formulating a relative test. Experimental results on CIFAR-10 indicate the effectiveness of ProoFit

Critical strengths
  • This paper studies an important problem of auditing deep neural networks in terms of overfitting. This paper is the first to audit the overfitting of CNNs and provides a detailed discussion of prior work.
  • This paper provides clear problem setups and definitions. The authors clearly state their ideas and insights for deriving the ProoFit framework. The provided theoretical results improve the soundness of the proposed test.
  • The proposed relative test largely reduces the computational cost of auditing the overfitting of CNNs, as demonstrated in the experimental results.
Critical weaknesses
  • The motivation for auditing the overfitting of the target model is vague. The user focuses more on the performance of the downstream task, and the regulation mostly requires more general auditing of the model training process. It would be helpful to provide additional discussion on the scenarios where the generalization gap is specifically informative.
  • Directly comparing ProoFit and baselines in terms of efficiency seems to be unfair, as they audit different things. Comparing complete audit pipelines in terms of both computational cost and false acceptance/rejection rates would make the practical benefits clearer.
  • The models used in the experiments are still far from practical scale. The reported TPR and TNR for SimpleCNN decrease distinctly compared with the 2-layer NN. It is still unclear about the robustness across different scales.
Questions for authors

Please see weaknesses

1 comment
AuthorsOpenReview ↗

Addressing Reviewer Feedback

We thank the reviewers and revise the paper (color blue)

Motivation for auditing Overfitting

Several regulatory and governmental frameworks explicitly recognize generalization as a required audit metric. These include the NIST AI Risk Management Framework, the EU AI Act, European Banking Authority’s guidance and the FDA’s Good Machine Learning Practice principles. Beyond generalization, overfitting also serves as a proxy for privacy leakage, i.e., whether a model reveals information about its training data. The SOTA approach for measuring privacy leakage uses membership inference (MI) tests, which face two limitations in our setting. First, an MI test requires the specific data point x to test -- precisely what our confidentiality constraints prohibit. Second, MI tests can be unreliable in practice [ZDK+24, DSM+24, DZT25, KCC23, MHL+25, AGP25]. Overfitting is a compelling alternative, and prior work establishes a strong connection between memorization and privacy leakage. UK ICO’s AI audit toolkit explicitly recognizes this connection -- ‘implement measures to ensure that AI models are not vulnerable to privacy attacks through overfitting’. Overfitting is also a fundamental measure of a model’s quality and our test can be used where the model itself cannot be shared. A prominent example is an ML model marketplace. A buyer wants evidence that a model generalizes before purchasing it, but the seller is unwilling to reveal the model’s weights or training data.

Baseline

Why PoT? Proof-of-training (PoT) is a natural and necessary baseline in our setting. The conventional approach to auditing overfitting is to empirically measure the generalization gap. However, because the training dataset is hidden from the auditor, a dishonest model owner could substitute an arbitrary dataset and pass the audit, even if the model is severely overfitted. To prevent this, the auditor must verify that the submitted dataset is the actual training set, without compromising confidentiality. PoT is the only known way to establish this guarantee. We now add two new baselines.

New baseline 1: Directional sharpness (Tan et al). This is concurrent work found after our submission. First, this approach is not sound - a dishonest owner can pass the test with a heavily overfitted model. We prove this theoretically (Theorem 4) and confirm it empirically: we get false positive rates of 0.94-0.99 and models with generalization gaps of up to 0.40 still pass (Table 5). In contrast, we formally prove ProoFit's soundness (Lemma 3) and validate it empirically. Second, ProoFit is 7−10×7-10\times faster, and directional sharpness cannot even run on ResNet-18 (out of memory).

New baseline 2: Unoptimized ProoFit To isolate the effect of the two optimizations introduced in Sec.4.4 for efficient ZK proof generation, we also evaluate an unoptimized variant of ProoFit. Without optimization 1, the naive way of computing the spectral norm requires a ZKP circuit that grows exponentially resulting in 1074×10^{74}\times higher cost than our optimized version.

Furthermore, we added performance benchmarks of two more larger image models VGG11 and ResNet50 for a more rigorous testing.

Test Reliability

We analyzed the test errors and made two observations.

  1. Misclassifications are concentrated almost entirely at the decision boundary - candidate models whose generalization gap is very close to that of the reference model. The mean generalization gap of misclassified models is within 0.007 of the reference model’s gap. At this resolution, a model’s training regime (well-fit or overfit) is a weaker indicator of generalization than the gap itself, so some errors are unavoidable. Two reasons make this more pronounced. First, in our original submission, we deliberately kept DauditD_{audit} small to reduce the cost of proof-of-inference, which limits statistical power for very small differences. Second, our spectral-norm optimization (Sec.~4.4) certifies an upper bound on RWR_W, which makes the test conservative--it errs on the side of flagging a well-fit model as overfitted rather than letting an overfitted one pass. This lowers TPR near the boundary. Moreover, the remaining errors are largely benign. The rejected well-fit models are early, underfit checkpoints whose gap is near zero only because their accuracy is low (test accuracy 0.39-0.72). These are not good-quality models anyway. Conversely, the accepted overfitted models have test accuracy close to that of well-fit models, and in most cases, they outperform the reference model despite their higher spectral complexity. In other words, the test rarely accepts a model that is meaningfully worse than the reference.

  2. A larger audit set substantially improves reliability. We reran the experiments with a larger DauditD_{audit}. TPR/TNR now ranges from 80% to 100% across all models (Table 3, App. A.12.3).

We welcome any further suggestions for improving the exposition of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.