acceptodds
Under review as a conference paper at ICLR 2027

GreenCert: Signed-Response Verification of Neural Training Trajectories

Abstract

We introduce GreenCert, a matrix-free verifier for certifying neural-network training trajectories over thousands of optimizer updates from a fixed checkpoint. GreenCert propagates the signed response to a proposed path’s one-step defects, preserving cancellation before bounding the remaining nonlinear error. We derive a self-consistency condition guaranteeing an enclosure of the exact-real optimizer trajectory. Output-margin bounds then certify persistent-event times, later reversals, or the absence of a target within the verified window on a fixed evaluation set. When the required conditions cannot be established, GreenCert abstains. The verifier uses explicit derivative and arithmetic-error bounds, together with an ideal-Gaussian assumption for randomized operator estimates. On classification and modular-arithmetic tasks, the signed construction closes in cases where its matched unsigned counterpart fails. In the tested small-network long-window studies, GreenCert completes full-trajectory guarantees within budgets exhausted by the sequential comparators. The cost advantage depends on the problem and horizon: direct continuation remains cheaper on the larger digits task through 1,024 updates. In a separate short-window experiment, GreenCert certifies a persistent-event onset to a single optimizer update in a million-parameter LayerNorm Transformer, matching the onset subsequently observed in a separately executed float64 continuation.

Primary area:
optimization
Keywords:
grokking, training dynamics, neural network verification, validated numerics, matrix-free methods, trajectory certification, rigorous numerics

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_CwgbOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper "GreenCert: Signed-Response Verification of Neural Network Training Trajectories" proposes a verification method for optimiser trajectories, i.e., which outputs may be reached when the actual optimisation trajectory deviates from the proposed one. For this, GreenCert propagates signed errors through the optimiser dynamics (thereby allowing for subsequent cancellation) and then infers the reachable output margins of resulting networks. Evaluations against sequential baselines show superiority on some benchmarks, but on larger models GreenCert does not perform well.

Critical strengths
  • the framework appears to be sound and correct in its formulations.
Critical weaknesses
  • the presentation and motivation of the paper are questionable. It remains unclear why a certificate for training trajectories is needed, when it induces costs exceeding the execution of the optimisation by orders of magnitude. Also, when a reference trajectory exists, future optimiser dynamics are already evaluated, so it remains unclear why certificates are needed.
  • The paper is extremely hard to read. In particular, there are several words and sentences that appear to be artifacts of heavy LLM assistance in writing the paper. Examples "The optional fallback has no fresh rescue", "Post-seal fixed-radius transformer audit", "amended budet clocks", "a metadata admission bridge", "frozen errors are unchanged". Appendix F is a particularly bad example, containing direct comments of an LLM used to prepare the code appendix. This is a direct quote from Appendix F: "The anonymous supplement is distinct from the manuscript source. Its top-level README and manifests identify the original inputs, numerical sources, results and logs for each study. The source package also includes ARTIFACT_INDEX.md and a relocation map linking every moved topic to its appendix and original record. Full pre-revision source passages are retained as noncompiled artifact documentation; none is experimental data generated by this edit." Thus, overall in terms of presentation and polish this paper is far from being ready to be published at a venue like ICLR.
  • GreenCert does not outperform the baselines in all experiments, so overall it appears like a weak improvement over simple baselines.
Questions for authors
  • What concrete practical question/problem requires a certificate obtained through GreenCert?
  • What are the advantages of GreenCert on the experiments where it performs worse than the baselines?
Reviewer_rGuBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This submission computes an enclosure of further training iterations using Hessian-vector products. The likely target is proof-of-training, although this isn't elaborated in the submission. The proposed method is evaluated in two settings. One setting studies long-horizon training certificates. The other setting studies a shorter horizon setting with a larger model.

Critical strengths

Assuming the goal is proof-of-training, this is a timely topic.

The submission is refreshingly up-front about its limitations.

The submission is also very concerned with numerical details, which is laudable in principle.

Unfortunately, I was unable to follow the paper beyong this.

Critical weaknesses

The presentation is severely lacking. For example, I could not extract the purpose of the proposed method or how the method operates.

Additionally, the paper fails to meet scientific standards, as no references are provided for the majority of the content.

While some proofs are provided in the appendix, I could not relate these proofs to Theorem 1.

Questions for authors
  • In plain language, what is the goal of this submission?
  • In plain language, what are your contributions?
  • In plain language, how does your method compare to existing work?
Reviewer_D8VoOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Summary: The paper proposes an approach to compute an error bound between a proposed training path and the exact-arithmetic trajectory of the optimizer, which is turned into proven statements about accuracy.

Critical strengths

Strengths:

  • The bounding errors are tracked as signed vectors so they can cancel before norms are taken.
  • The approach does not require building Hessian matrices (only Hessian-vector products).
Critical weaknesses

Weaknesses: -- The paper is very hard to read, with main-text claims scattered across ~25 appendix pages.

  • The approach has significant computational cost, and the paper does not provide a clear use case or practical motivation.
  • The scope is limited, i.e., full-batch and deterministic momentum only, with no minibatch, dropout or Adam.
  • The paper contains AI-written artifacts, e.g. Appendix F: "none is experimental data generated by this edit."; Appendix D.9 is self-referencing.
Questions for authors

Questions:

  1. What is a practical use case for the proposed approach?
  2. What is the advantage compared to just continuing training, which is much cheaper?
  3. The error bounds are only w.r.t. to the idealized training (real arithemtic). Could the error bounds be extended to the actual executed training trajectory?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.