acceptodds
Under review as a conference paper at ICLR 2027

PRISM: Probing and Repairing Internal States in Language Model Pretraining

Abstract

Internal monitors reveal changes that aggregate training statistics obscure, but deciding when to intervene requires understanding their consequences. We present PRISM, a framework that connects component-level measurements to counterfactual probes and matched interventions in language model pretraining. Across dense and mixture-of-experts models, these comparisons reveal an intervention gap: suppressing an activation tail can leave training worse, and restoring expert opportunities can coexist with higher observed loss. Conversely, a small aggregate error in optimizer state can precede a catastrophic update. To recover from this failure, we introduce protected-summary recovery, which reconstructs damaged Adam second moments from one independently maintained total per tensor and intact momentum. A constrained reconstruction combines this total with a known moment bound; an equal-information mean reconstruction isolates the role of coordinate allocation. Both rules contain update transients and achieve lower early-horizon negative log-likelihood than moment reset in all five matched execution blocks across two dense settings. PRISM turns internal observations into experimentally testable intervention choices, with compact optimizer-state recovery as a concrete application.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Pretraining, Training Dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_1PXWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that internal training signals alone cannot tell you whether to intervene. So it proposes PRISM: fork matched continuations from one checkpoint under different interventions, then compare mechanism, quality, and cost. It also proposes protected-summary recovery, which rebuilds damaged Adam second moments from one stored scalar per tensor and intact momentum.

Critical strengths

The recovery method is easy and cheap:

Critical weaknesses

It is unclear how short controlled continuations represent later training. The reported gains are tiny, about 1e-3 NLL, and mostly fade as training continues. The conclusions also do not transfer, so the comparison must be trained for each model, even stage, and fault. This overhead from my view is too high.

Questions for authors

How are fork points and candidate interventions chosen in a real run?

Reviewer_xrEtOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the effectiveness of internal monitoring signals and of intervention actions during language model pretraining. The authors propose PRISM, a framework that first identifies the component on which a monitoring signal is computed, then tests the relation between its statistic and function through function-preserving transformations, and finally measures the effect of an intervention action on mechanism, quality and cost through forked comparison experiments. On 284M/378M dense settings and a mixture-of-experts setting, the experiments reveal an "intervention gap": interventions aimed at monitoring signals are not necessarily effective. For example, suppressing the activation tail improves the monitored statistic, while increasing the training loss. In addition, for the update explosion and training collapse caused when vv in the Adam optimizer is damaged in a precision round-trip conversion, the paper proposes protected-summary recovery, an algorithm that does not significantly increase storage or compute cost while allowing training to continue.

Critical strengths
  1. This paper proposes PRISM, a systematic framework for evaluating intervention actions: starting from the same parent model and controlling the data and the horizon, it compares three conditions (a target action, no action and an alternative action), yielding the mechanism, quality and cost consequences of an intervention. With this framework, the authors identify two intervention gaps: both suppressing the activation tail and correcting expert imbalance improve the monitored statistic while slightly increasing the loss. The framework and these findings have diagnostic value for language model pretraining.
  2. This paper proposes protected-summary recovery, a low-cost solution to the problem that the Adam second moment vv is largely rounded to zero after a precision narrowing, which causes training to collapse. The method maintains one additional full-precision scalar per tensor, namely the second-moment total (the sum of the coordinate-wise second moments), and reconstructs vv with the coordinate-wise lower bounds given by the first moment mm, so that training continues.
Critical weaknesses
  1. The diagnostic conclusions of the PRISM framework are based on controlled runs built by the authors: two dense models of 284M/378M and one mixture-of-experts model, with an internal pretraining corpus and its tokenizer; several mechanism studies rely on a single parent model. The generality of the "intervention gap" across model scales and across fault distributions is therefore not demonstrated.
  2. When protected-summary recovery reconstructs vv, the residual mass beyond the conservative lower bound is allocated uniformly, and this uniform term relies only on maximum entropy: it uses neither the difference in scale between coordinates nor the distribution of the first moment. Because the lower bound is itself conservative, the residual mass can be considerable, and the reconstructed value is dominated by this uniform term. The paper does not evaluate whether a finer allocation rule (for example, weighting by the distribution of mim_i) affects the early NLL.
Questions for authors
  1. Can the intervention gap be reproduced beyond the current settings? For example, with a second parent model or independently trained seeds, a different pretraining corpus, or a different fault distribution.
  2. If protected-summary recovery used a finer allocation rule instead of uniform allocation, how large would the difference in early NLL be?
Reviewer_55jmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces PRISM, a framework for diagnosing internal training states and evaluating interventions during language model pretraining. It combines component-level monitoring, counterfactual probes, and matched training continuations to distinguish changes in internal statistics from actual training benefits. The paper further proposes protected-summary recovery, which reconstructs corrupted Adam second moments using a per-tensor summary and intact momentum. Experiments on dense and MoE models demonstrate several diagnostic findings and improved early-stage recovery compared with optimizer-state reset.

Critical strengths
  1. Relevant and practical problem. The paper addresses training diagnosis and recovery, which are important challenges in large-scale LLM pretraining. Distinguishing internal monitoring signals from actual intervention benefits is a valuable perspective.

  2. Comprehensive diagnostic investigations. The study covers multiple training mechanisms, including activation outliers, MoE expert utilization, residual connections, optimizer states, and coordinate transformations. The matched interventions provide useful empirical observations.

  3. Lightweight optimizer-state recovery. The proposed protected-summary reconstruction is technically interesting, requiring only one additional scalar per tensor. The experiments demonstrate improved early-horizon NLL and reduced update transients compared with moment reset.

Critical weaknesses
  1. Limited novelty and unclear central contribution. PRISM primarily organizes existing diagnostic techniques, counterfactual analysis, and controlled interventions into a common framework. It does not provide a clearly specified, generalizable algorithm for selecting optimal interventions. The optimizer-state recovery method is more concrete, but its relationship to existing compressed-state and fault-recovery techniques needs clearer differentiation.

  2. Fragmented empirical validation. The diagnostic experiments investigate largely independent mechanisms under different settings, without demonstrating how PRISM systematically improves training decisions. Several experiments report negative or context-dependent intervention outcomes, and the paper does not establish that the proposed framework consistently outperforms simpler monitoring or intervention strategies.

  3. Restrictive assumptions and limited applicability of the recovery method. Protected-summary recovery assumes that only Adam's second moments are corrupted, while weights, momentum, clocks, gradients, and the independently maintained summaries remain intact. The experiments primarily use artificially introduced FP16 corruption, leaving its effectiveness under more realistic and diverse training failures unclear.

  4. Limited scalability and long-term validation. The principal experiments use relatively small models (284M–774M parameters), and repeated executions do not constitute independent pretraining runs. Moreover, the recovery advantage over reset becomes less consistent over longer training horizons, while the two reconstruction rules exhibit different performance across settings.

  5. Unclear practical cost-effectiveness. Maintaining protected summaries introduces approximately 4% additional training overhead in the tested dense settings. Without evaluating realistic failure frequencies and end-to-end recovery costs against checkpoint-based approaches, the practical benefit for large-scale LLM pretraining remains uncertain.

Questions for authors

Q1. What is the concrete decision-making procedure provided by PRISM beyond conducting matched diagnostic experiments? Given that several proposed signals fail to generalize across stages or contexts, how would PRISM select interventions in a previously unseen training run without requiring expensive additional continuations?

Q2. The paper shows that both internal support monitoring and a prospective-update guard can detect the optimizer corruption. What additional practical benefit does the proposed internal monitoring provide over simpler global or update-based safeguards? Could the authors quantify improvements in detection accuracy, false-positive rates, or intervention timing?

Q3. How does protected-summary recovery perform under more realistic fault scenarios, such as partial tensor corruption, nonzero perturbations, or simultaneous damage to optimizer states? Since the protected summary is maintained in memory, how would the method handle correlated failures affecting both the optimizer state and its summary?

Q4. In Table 2, the constrained and mean reconstruction rules exhibit different rankings across GQA and MLA, while the advantage over reset weakens at longer horizons. What determines the preferred reconstruction rule, and under what conditions does preserving the summary provide a sustained improvement rather than only a temporary recovery benefit?

Q5. Given the approximately 4% maintenance overhead, what fault frequency would make protected-summary recovery more cost-effective than conventional checkpointing and rollback? Could the authors provide an illustrative end-to-end cost comparison using measured training and recovery times?

Reviewer_rTMVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies a gap in monitoring and intervention of LLM pretraining, where intervening on poor "training health" statistics (e.g. outlier activations, routing collapse) does not fix the underlying issue causing training to fail. The paper proposes PRISM, a framework to evaluate the effectiveness of interventions by using counterfactual probes and matched training continuations.

Critical strengths

The paper considers a broad view of pretraining monitoring and interventions, which can be a much more efficient alternative than re-running pretraining with alternative learning algortihms or loss functions.

Critical weaknesses
  1. Limited evaluation. In order to properly motivate the approach, the authors should use examples of training failure that arise in real-world setups rather than creating failure modes by hand.

  2. End-to-end benefit is not clear. The paper mainly tests varying types of interventions against each pretraining failure, but does not evaluate how this analysis can speed up real-world pretraining. For example, a natural baseline for a practitioner is to roll back a failing run, test an intervention, and compare losses. If PRISM is used in an end-to-end manner to identify and skip interventions that don't help, overall training quality may not be significantly better.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.