acceptodds
Under review as a conference paper at ICLR 2027

Training Witnesses: Trusting the Training without Trusting the Trainer

Abstract

Progress in machine learning cannot outpace our ability to verify it. With an explosion in papers today, every scientific claim rests initially on trust in the trainer, leading to uneven evaluation, baselines, and forestalling of reliable progress. Traditionally, the burden of verification falls on the reader, who must reproduce expensive training runs. This strategy is impractical due to an explosion in slop contributions, diversity of methods, and the sheer compute required. We put the burden of proof where it belongs, on the trainer, and in the process also cut the overall cost of verification significantly. We introduce **Witnesses**, a method for certifying training data usage and evaluation in a neural network training run. We leverage the insight that fast behavioral fingerprints with occasional replay challenges is sufficient for auditing neural network training. Our method is applicable at scale with low overhead to the trainer, is cheap for the verifier, rejects bad training runs with amplifiable probability, and allows for exact queries of both data inclusion and exclusion. We test our method through baseline language model training runs ranging over 100M to 2B scales, across DDP and FSDP, and demonstrate minimal overhead. We also introduce a self-regulating leaderboard of training runs that enables shared baselines and progress. We invite the community to participate in the leaderboard to improve reproducibility in machine learning.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
reproducibility, proof of training data, parameter sketching, feature hashing, technical governance, cryptographic certificates, software package

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YXsnOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Training Witnesses, a framework for verifying neural network training without reproducing the entire process. It uses lightweight parameter fingerprints to record training states. The authors evaluate the system on models ranging from 100M to 2B parameters. The results show strong detection rates under the tested attacks, with relatively low training overhead.

Critical strengths
  1. Clear motivation. Reproducing large-scale training is expensive. The paper addresses this problem with a useful approach that could make training claims easier to verify.

  2. Promising experimental results. The proposed fingerprints and selective replay is interesting. The experiments demonstrate its feasibility at billion-parameter scale, with only about 1.19× runtime overhead in the 2B model setting.

Critical weaknesses

None

Questions for authors

How effective is the method against attacks involving only a few targeted training updates? Could the authors clarify how much protection parameter sketching provides beyond random replay in such cases?

2 comments
AuthorsOpenReview ↗

Targeted training updates

For the case of a few targeted updates, consider the following casework. One of three things could occur:

  1. An update that shows up on sketch: a transition whose difference is larger than expected would generate a sketch that is also dramatically different; beyond heuristic measures since every sketch is committed, challenged steps will also catch this difference by simply checking the sketches.
  2. An update that does not show up on sketch:
    1. …because it is too small: in figure 14, we see that small, up to single-parameter cheats (upper left) are reliably detected since the magnitude of difference when sketched is much larger than the single-parameter would affect (due to the final hash’s avalanche effect); when the change shows up in a small subspace to avoid the sketch (rank one case in figure 14), it is either reliably detected or does not meaningfully affect the model outputs
    2. …because it's adversarially optimized: in figure 8, we see that active optimization against our sketch cannot evade it by meaningfully shifting the training distribution. By committing the logits of training samples, we can measure and have the verifier catch this shift directly.

To operationalize that these cases are indeed exhaustive, we refer to [1] and [2]. In particular, [1] highlights that a small, yet constant number of tokens are required for poisoning, a regime we reliably detect. [2] highlights that LoRA fine-tunes of poisoning occupy many singular directions, meaning an orthogonal projection type method can be more effective in detecting it. Even most recent works on post-training edits [3] requires the spiking activation of a few neurons which we detect with near certainty in Figure 14.

If the reviewer believes there's a form of attack missed, we'd love to discuss it.

[1] Souly, et al., “Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples.” arXiv preprint, 2510.07192, (2025).

[2] Chillimuntha, et al., “Associative vs. Distributional: Two Regimes of Backdoor Learning in LoRA-Adapted Code-Generation Models.” J. Cybersecurity Priv, (2026).

[3] Papailiopoulos, "Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents" (2026).

I appreciate the authors' revisions and efforts. I have no further concerns.

Reviewer_nqrgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a system for verifying neural network training without reproducing the entire training run. It records compact fingerprints of model parameters, optimiser states, and training data at each step, then randomly replays a small fraction of updates to detect inconsistencies. This allows third parties to check training-data usage and whether a model follows a declared training procedure. The main contribution is a practical combination of efficient parameter fingerprints, probabilistic verification, and an implementation designed for large-scale training. Experiments on 100M–2B parameter models report a 19% training-time overhead for the 2B sharded model, with promising detection rates against simulated data manipulation.

Critical strengths
  1. Reducing the cost of independently verifying training runs could improve reproducibility and confidence in reported results.
  2. The combination of compact parameter fingerprints, occasional replay, and efficient state management is well motivated. The 2B experiments provide encouraging evidence that verification can be integrated into training at moderate cost.
  3. The authors compare against relevant baselines, study data addition and removal, and test attacks specifically designed to evade their parameter fingerprints. The theoretical analysis also provides a useful framework for understanding detection probability.
Critical weaknesses
  1. The practical system uses an obfuscated verification component that the attacker is assumed to be unable to compromise, despite controlling the training environment. This assumption is central to the entire method, but its security is not established. It is unclear if this assumption is realistic.
  2. Most data-manipulation experiments distribute corruption uniformly across training. At a 1% checking rate, a small number of deliberately corrupted steps can easily escape replay. The theoretical guarantees can also be weak for such attacks. The adversarial fingerprint experiments are encouraging but limited to a small model.
  3. The 19% overhead is measured on a 2B model using two GPUs. The 1B DDP experiment incurs approximately 46% overhead. Larger training configurations remain untested, and the growing certificate size raises questions about the cost of verifying full pretraining runs.
Questions for authors

Can this approach be adapted to verify in a pipeline-parallel training setting considered in [1]?

  • [1] Dolatabadi, Hadi Mohaghegh, et al. "SENTINEL: Stagewise Integrity Verification for Pipeline Parallel Decentralized Training." arXiv preprint arXiv:2603.03592 (2026).
7 comments
AuthorsOpenReview ↗

Security Assumption

In our system, the practical guarantees and formal ones are separate (sec 3.1). For the practical guarantees, we assume that the attacker cannot tamper with the white-box cryptographic virtual machine. This is a commonly used technique in technologies for ensuring copyright, such as DRM, and widely established in works like [1].

Our central contribution can be understood as a way to reduce the trust in expansive pretraining operations into a much cheaper and smaller trust boundary of trusting easy-to-compute hashes that does not require a large working GPU/CPU memory or evolving state. DRM / Whitebox cryptography is only one way to establish the final small root of trust.

[1] Chow, et al., “A White-Box DES Implementation for DRM Applications.” ACM-DRM, (2002).

AuthorsOpenReview ↗

Sparse, targeted attacks beyond replay

First, note that when the model scales, the rate of checking is considered as a function of parameters and in fact communication overheads dominate. So our method becomes more efficient despite checking more.

Second, regarding a small check rate, consider the following casework. One of three things could occur:

  1. An update that shows up on sketch: a transition whose difference is larger than expected would generate a sketch that is also dramatically different due to the linearity of the sketch the sketch. Although the prover indeed do not take into account the sketch magnitudes for sampling exact replays, a verifier can use the fact that a sketch is dramatically out of distribution compared to other transitions (since every transition is sketched) to challenge the authenticity of tapes. In Figure 14, we see that even smaller (single parameter cheats) can show up reliably on sketch much before output distributions of the model change; thus, a large cheat would show up with even higher magnitude due to almost surely linear independence of random Gaussian matrices. This would be easy for a verifier to identify because regular training gradient magnitudes decay and stay constant after training; a large change would cause a relatively dramatic difference in the sketch.
  2. An update that does not show up on sketch:
    1. …because it is too small: in figure 14, we see that small, up to single-parameter cheats (upper left) are reliably detected since the magnitude of difference when sketched is much larger than the single-parameter would affect (due to the final hash’s avalanche effect); when the change shows up in a small subspace to avoid the sketch (rank one case in figure 14), it is either reliably detected or does not meaningfully affect the model outputs
    2. …because it's adversarially optimized: in figure 8, we see that active optimization against our sketch cannot evade it by meaningfully shifting the training distribution. By committing the logits of training samples, we can measure and have the verifier catch this shift directly.
AuthorsOpenReview ↗

Training Overheads and Scalability

High DDP Overheads. Note that DDP beyond 7B or above is really an unrealistic setup. For most larger models, some form of communication-bound sharding more in the lineage of FSDP or Pipeline Parallelism is the norm. This setup allows us to be more efficient, as you noted.

Cost of verification. In Figure 10, right, we note that the Resident Set Size (RSS) of a certificate verification procedure is constant during scaling. Furthermore, per link, the certificate transmits a constant number of bytes. This means that a verification procedure can be streamed to the verifier (who we trust) and then the certificate discarded. To also provide trust to another party independently, we can stream and store the certificate, and stream them again to another verifier.

Reviewer_W4s7OpenReview ↗
Rating
1
Confidence
4(+1)
Informativeness
1
Presentation
2
Soundness
1(−1)
Read review
Paper summary

The manuscript introduces "Witnesses," a framework designed to cryptographically certify the training process, data usage, and evaluation of neural networks. To address the growing crisis of reproducibility and the high computational cost of verification, the authors propose shifting the verification burden from the evaluator to the trainer. The core mechanism leverages fast, low-rank geometric parameter sketching (behavioral fingerprints) combined with probabilistic replay challenges to audit training runs. By recording update functions, optimizer states, and data batches into a cryptographically signed, append-only chain, the system enables third-party verifiers to independently check data inclusion and exclusion queries with minimal computational cost. The authors demonstrate that this method scales efficiently, incurring minimal speed (≤1.19×\le 1.19\times) and memory (≤1.11×\le 1.11\times) overheads on a 2.02B parameter model.

Critical strengths

Practical Scalability and Novel Cryptographic Application: The integration of a low-rank, full-spark orthonormal projection for parameter commitments (geometric sketching) provides a highly efficient behavioral fingerprint. This effectively restricts the degrees of freedom for an attacker, ensuring that any undeclared malicious updates must be non-sparse to evade detection. Because this sketching is computationally cheap enough to execute at every gradient update, it successfully bridges the gap between theoretically sound cryptographic proofs (like zk-SNARKs, which are practically intractable at scale) and fast but unreliable statistical heuristics. This creates a highly practical, verifiable certificate of trust for large-scale models.

Critical weaknesses

Hardware-Induced Nondeterminism and Bit-Level Reproducibility: The protocol's reliance on exact replay challenges assumes that environments will yield deterministic floating-point operations. The manuscript claims to use stateless accelerator serializations to ensure identical replays without side effects. However, bit-level reproducibility is notoriously difficult to guarantee across different hardware architectures (e.g., varying GPU microarchitectures or non-deterministic atomic operations). If a verifier utilizes a different hardware setup than the trainer, accumulated floating-point discrepancies could cause false rejections of benign training transitions, undermining the practical deployability of the certificate in open-hardware ecosystems.

Vulnerability to Sparse, High-Impact Malicious Updates: The system relies on a probabilistic replay check, challenging transitions with a probability pcheckp_{check} (e.g., 1%). While the authors mathematically bound the overall success of continuous attacks, a critical vulnerability remains regarding highly sparse, targeted poisoning attacks (e.g., inserting a specific backdoor trigger in a single batch). If an attacker sneaks a malicious update into the 99% of unchecked transitions, and the subsequent updates follow the declared benign function, the malicious payload might survive unnoticed. The geometric sketch bounds the sparsity of the attack, but it does not definitively eliminate the risk of rare but high-impact targeted poisoning that evades the 1% challenge rate. Examining threat models around isolated vulnerabilities is a vital critique for securing this architecture.

Questions for authors

How does the system handle floating-point nondeterminism when the verification environment (e.g., hardware architecture, driver versions) differs from the training environment? While the adversarial optimization section mentions floating-point calibration (rtol=1×10−5r_{tol}=1\times 10^{-5}, atol=1×10−8a_{tol}=1\times 10^{-8}), are similar tolerances formally built into the verification hashing protocol to prevent false rejections due to hardware-specific rounding errors?

Regarding the probabilistic replay mechanism where pcheck=1%p_{check} = 1\%, how can the protocol guarantee robustness against a highly targeted, single-batch poisoning attack? If a malicious update occurs in an unchecked step, and all subsequent updates are strictly honest, how does the geometric sketch guarantee the eventual detection of this persistent, low-frequency backdoor?

2 comments
AuthorsOpenReview ↗

Hardware nondeterminism and replay

How does the system handle floating-point nondeterminism when the verification environment (e.g., hardware architecture, driver versions) differs from the training environment?

Please note that the verifier and the prover are two different systems. The prover runs on the same hardware co-located with the trainer, creating, challenging, and replaying hashes.

The verifier indeed runs on diverse hardware, but one benefit of our orthonormal linear geometric sketch design (Figure 4) is that floating point tolerances are passed through linearly from before to after the sketch. This means that users can choose what level of tolerances they believe based on information regarding what hardware transfers happened, and large deltas in the model being measured will result in no smaller sketch differences (and vise versa.)

For an exploration for this exact consideration of the relative rates of differences in the sketch versus actual influence on model behavior relating to float tolerances, see Figure 14 in the Appendix.

The same Torch-default tolerances (rtol 1.6e-2, atol 1e-5) are used everywhere whenever “equality” is claimed.

AuthorsOpenReview ↗

Sparse, targeted attacks

Regarding the probabilistic replay mechanism where , how can the protocol guarantee robustness against a highly targeted, single-batch poisoning attack?

Thank you for this great question! Let’s walk through each scenario.

Cheating only on unchecked transitions. This is essentially not possible (Figure 6). The reason why this is the case is because hashes and cheating challenges for future steps is a function of previous step sketch outputs. We sketch on every step, and sample randomness after, so trainers who cheat on one particular step (i.e., change their outputs) will result in a totally new sampling pattern for future steps which they cannot anticipate. Put another way, there are no “unchecked transitions” that trainers can know where it lies a priori without first committing to their transition.

Cheating on a single large update. Thank you very much for pointing this out, and we should improve the clarity of our work in this case. One of three things could occur, let’s work through each of them.

  1. An update that shows up on sketch: a transition whose difference is larger than expected would generate a sketch that is also dramatically different due to the linearity of the sketch the sketch. Although the prover indeed do not take into account the sketch magnitudes for sampling exact replays, a verifier can use the fact that a sketch is dramatically out of distribution compared to other transitions (since every transition is sketched) to challenge the authenticity of tapes. In Figure 14, we see that even smaller (single parameter cheats) can show up reliably on sketch much before output distributions of the model change; thus, a large cheat would show up with even higher magnitude due to almost surely linear independence of random Gaussian matrices. This would be easy for a verifier to identify because regular training gradient magnitudes decay and stay constant after training; a large change would cause a relatively dramatic difference in the sketch.
  2. An update that does not show up on sketch:
    1. …because it is too small: in figure 14, we see that small, up to single-parameter cheats (upper left) are reliably detected since the magnitude of difference when sketched is much larger than the single-parameter would affect (due to the final hash’s avalanche effect); when the change shows up in a small subspace to avoid the sketch (rank one case in figure 14), it is either reliably detected or does not meaningfully affect the model outputs
    2. …because it's adversarially optimized: in figure 8, we see that active optimization against our sketch cannot evade it by meaningfully shifting the training distribution. By committing the logits of training samples, we can measure and have the verifier catch this shift directly.

To operationalize that these cases are indeed exhaustive, we refer to [1] and [2]. In particular, [1] highlights that a small, yet constant number of tokens are required for poisoning, a regime we reliably detect. [2] highlights that LoRA fine-tunes of poisoning occupy many singular directions, meaning an orthogonal projection type method can be more effective in detecting it. Even most recent works on post-training edits [3] requires the spiking activation of a few neurons which we detect with near certainty in Figure 14.

If the reviewer believes there's a form of attack missed, we'd love to discuss it.

[1] Souly, et al., “Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples.” arXiv preprint, 2510.07192, (2025).

[2] Chillimuntha, et al., “Associative vs. Distributional: Two Regimes of Backdoor Learning in LoRA-Adapted Code-Generation Models.” J. Cybersecurity Priv, (2026).

[3] Papailiopoulos, "Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents" (2026).

Comments on the paper

1 comment

Thank you to the reviewers for their very nice questions. As the review period is coming to an end, we'd love to request the reviewers to consider updating their scores, or respond if there's any additional questions we can help answer. Thank you again for your time and service!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.