acceptodds
Under review as a conference paper at ICLR 2027

Mechanism-Aligned Observability for Training Instability of Large Language Models

Abstract

Frontier large language model training consumes massive accelerator fleets over long wall-clock times, making stability failures extremely costly. However, global symptoms such as loss divergences often emerge only after the underlying fault has already affected the training dynamics, further increasing the sunk compute. In which internal representation can a training fault become persistently and interpretably observable before it appears in global training metrics? We propose a mechanism-aligned principle for training observability: identify the internal state into which a fault is persistently written, derive the structure induced by its mechanism, and construct monitors in representations aligned with the computation of the affected module. For FlashAttention, we show that biased backward errors accumulate into a persistent low-dimensional structure in weight updates, producing spectral concentration. For MoE routers, we characterize routing dynamics through per-token routing behavior rather than parameter similarity, which may change without altering the routing function. Experiments show that the two monitors respond selectively to changes in their corresponding modules, before degradation becomes visible in global training metrics. ur code is available at https://anonymous.4open.science/r/mindspeed-observability-F605.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM Training, Training Instability Detection, Low-Precision Training, Mixture-of-Experts, Spectral Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VDTUOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a monitor that detects training failures. This monitor has two applications: FlashAttention, and MoE routers. This is a three-step process: first, identify the persistent affect of the fault; second, derive the structural change; third, construct a monitor.

Critical strengths

The paper addresses the important problem of error diagnosis based on internal states. We need failure diagnosis methods beyond downstream task evaluation metrics.

FlashAttention and MoE models are widely used in nowadays AI/LLM systems.

Critical weaknesses
  1. Limited experiment scale, and lack of experiment details: MoE experiments only use Qwen3 8B-A1B and no other models. I did not see any detailed experiment setup for the FlashAttention part.

  2. A unified method is unclear/missing: I did not see any shared diagnosis method/pipeline proposed. FA and MoE are separate problems that require separate attention. What is the connection between them? Is the method proposed only to solve these problems? The paper looks more like two independent case studies than a unified method.

  3. Limited comparison/discussion with prior diagnosis approaches. What is the unique contribution of this paper, compared to existing approaches?

  4. I fail to see any concrete empirical validations (e.g., improved training performance, measured by task metrics) that can show why the proposed method is superior, or should be used.

Questions for authors

Please kindly refer to weaknesses.

7 comments

Thank you for the insightful comments! We are sorry that our paper presentation did not meet your satisfaction. We have revised the paper based on your suggestions and those of other reviewers, hoping that it and the following discussion of the weaknesses will address your concerns.

Weakness 1a: Model Coverage and Architecture Choice

Thank you for asking about model coverage. We clarify the rationale for the primary architecture and the additional evidence for the FA and Router studies.

We chose Qwen3 8B-A1B MoE for the main training experiments because its 28 layers, 128 routed experts, and top-6 sparse selection per token retain the key structures examined in our analysis: competition between expert scores, concentration of routing probabilities, and load-balancing feedback. These structures correspond directly to the mechanism analysis in Section 3.2. Varying LR, GBS, and AUX within a fixed architecture allows us to compare their effects on routing dynamics while reducing confounding changes in model size, expert count, and routing scheme. Compared with the GPT-2-scale models used in much of the related work, our 8B-A1B MoE setting is closer to realistic LLM training.

For additional evidence, we have added FA training experiments on a 10B-A1.6B model, with results included in Appendix F.4 of the revision. The router study also includes analyses of other MoE models. Table 1 and Fig. 5 of our paper compare weight similarity and actual routing distributions across models to examine whether parameter similarity consistently characterizes routing behavior.

Weakness 1b: FA Experimental Setup

Sections 4.1–4.2 and Appendix C of our paper already specify the main training configuration, fault-injection procedure, and monitoring settings. Rebuttal Table 1 summarizes these details. In particular, Eqs. (14)–(16) in Appendix C provide the bit-level operations, a concrete example, and the error expansion. We will consolidate this information in the revision for easier reference.

Rebuttal Table 1: Setup of the main FA experiments

ItemSetup in our paper
Model and training configurationQwen3 8B-A1B MoE, 28 layers, 128 routed experts, top-6 dispatch; LR = 5e-4, GBS = 384, AUX = 0.001
ConditionsHealthy Baseline and three numerical perturbation levels: FB2, FB1, and FB0
Fault injectionZeroing the least significant mantissa bits of FP32 OO and BF16 dOdO in the FA backward pass; FB2, FB1, and FB0 retain 2, 1, and 0 explicit mantissa fraction bits, respectively
Monitoring500-step windows; comparison of spectral statistics for raw weights, parameter updates, and QK functional updates

Weakness 2: Unified Method and Its Two Instances

Thank you for asking about the unified method and the connection between the two instances. Section 3 presents a method for constructing monitoring representations from fault-propagation mechanisms; FA and MoE instantiate this method in different modules. Given a target mechanism and the affected module, the method identifies the internal states that are persistently affected, derives the expected structure and constructs a monitoring representation based on the module's computation. Both instances follow this shared process, as detailed in Rebuttal Table 2.

Rebuttal Table 2: The shared mechanism-aligned design process for FA and the MoE router

Shared design stepFAMoE router
Identify the target mechanismBiased errors from low-precision backward computation repeatedly enter Q/K updatesLR, GBS, and AUX affect routing dynamics through relative expert-score margins
Locate the persistently affected stateRecent Q/K parameter updatesRouter parameters and the relative expert-score margins they induce
Derive the expected structureUnder conditions including a low-rank mean and controlled fluctuations, accumulated errors can concentrate the update spectrumFeedback that reinforces the current winning experts can drive routing probabilities toward concentration and saturation
Identify the mismatch in existing representationsThe historical component in raw weights can obscure recent update structureChanges in router weight similarity need not correspond to changes in the routing function
Construct a function-aligned representationMove from WW to windowed updates ΔW\Delta W, then use the bilinear QK computation to construct the functional increment ΔF\Delta FMove from parameter similarity to per-token routing entropy to track probability concentration and saturation

The mechanism analysis directly determines the representation in each instance. This unified method can also be applied to other training modules whose fault-propagation paths and computational structure can be analyzed. The monitoring quantity is derived from the target module's mechanism, rather than prescribing a single metric for every module. We will clarify this in the revision.

Weakness 3: Comparison with Prior Diagnostic Approaches

Our paper already includes related-work discussion and targeted comparisons of monitoring representations. Our distinct contribution is to derive these representations from mechanism analysis and evaluate the resulting choices in actual training. Rebuttal Table 3 summarizes our specific contributions relative to these diagnostic approaches and mechanism studies.

Rebuttal Table 3: Comparison with existing diagnostic approaches and our contributions

Existing workOur specific contribution
Global or parameter-level statistics, such as loss, gradient norm, and raw weightsAnalyze why these representations can obscure mechanism-related changes, and compare windowed parameter updates with functional updates to demonstrate how representation choice affects fault observability
Qiu & Yao's analysis of biased errors in low-precision FAFurther analyze error accumulation within update windows, derive the predicted spectral concentration under the corresponding conditions, and construct the functional increment ΔF\Delta F and its decomposition aligned with the bilinear QK computation
Behavior-level monitoring, such as attention logits and max-logitProvide a functional-update representation based on periodic Q/K parameter snapshots, without modifying the fused FA kernel or collecting additional attention activations; compute the nonzero singular spectrum exactly using a small core obtained by QR factorization
Router weight-similarity monitoringUse functional invariance to show that parameter similarity does not uniquely characterize the routing function, then derive the choice of per-token routing entropy from relative expert-score margins to monitor probability concentration and saturation
Reviewer_bYfTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors provide a mechanism-aligned method for monitoring training instability in LLMs. While other methods might rely only on global symptoms such as loss divergence, this method can identify representations in which some faults become visible earlier. For low-precision FlashAttention, this method examines the spectral structure of recent parameter updates and changes in the join QK operator. For Mixture of Experts (MoE) routers, instead of relying on router-weight similarity, it monitors per-token routing entropy. Overall, the paper combines mathematical analysis, controlled training experiments that include fault injection and hyperparameter variation, and an analysis of routing statistics across checkpoints of open-weighted models.

Critical strengths
  1. The paper is well written. The figures are clean and clear, and I found the related-work section really helpful. In particular, I appreciated the explanations that motivated the shift from raw weights to recent updates and from router-weight similarity to routing behavior.
  2. The experimental comparisons are aligned with the proposed monitoring choices. For instance, the attention experiments compare raw weights, recent parameter updates, and QK-functional updates, while the MoE router experiments vary learning rate, batch size, and auxiliary balancing. Also, the cross-module comparisons examine whether the monitors are responding in a selective way to the perturbations. The cross-family checkpoint analysis also shows why router-weight similarity alone might be misleading and not sufficient.
Critical weaknesses
  1. The scope is narrower than the general motivation. Scope boundaries should be stated more explicitly.

    The motivation acknowledges two forms of monitoring mismatches. I think there should be a brief, explicit clarification that these might not cover all possible mismatches and are more illustrative.

    More importantly, the controlled training experiments use one Qwen3-like MoE architecture and a specific controlled FlashAttention fault-injection procedure.

    Also, the analysis of checkpoints across model families, while providing helpful complementary evidence, does not establish early-warning performance across those model families.

    So the paper might benefit from separating which conclusions are shown in these specific cases from those that are proposed as a broader monitor design rule.

  2. The experiments leave some questions about the calibration and reliability

    The spectral analysis is compared to a healthy baseline. The paper notes that healthy updates can also be low rank, which is correct. But the paper does not clearly establish how an operator would set a threshold in a new training run to separate normal training progress from abnormal concentration and assess missed failures or false alarms.

    The sensitivity to the monitoring window is also important.

    Also, the plotted bands show variability across layers rather than reproducibility across independent training runs.

    Appendix E discusses the efficiency of the small-core spectral computation, but does not cover the complete training-time and memory overhead of the monitoring system.

Questions for authors
  1. Are the two mismatches mentioned as representative examples? Can you clarify the scope of the design principle and mention which failure mechanisms the two proposed monitors are not intended to detect?
  2. How should the thresholds be calibrated when a matched healthy training trajectory is not available? Can you clarify the sensitivity to the 500-step update window and provide any evidence on false positives, missed failures, and variability in warning lead time across multiple independent runs?
  3. What is the end to end overhead of collecting and computing the monitors at the evaluation frequency used, including additional memory and synchronization costs?
6 comments

Thank you for the insightful comments!

Weakness 1a: Experimental Coverage and Controlled Faults

Our choice of monitoring representations is supported jointly by model-independent structural results and empirical tests during actual training. Invariance of the routing function under a common-vector shift and the bilinear decomposition of joint QK changes provide the structural basis for these choices. The 8B-A1B model used in our main experiments has 28 layers, 128 routed experts, and top-6 dispatch, allowing both FA and router modules to be examined in complete end-to-end training. Different monitoring representations share the same training states and fault conditions, enabling a direct comparison of how representation choice affects fault observability. Compared with the GPT-2-scale models used in much of the related work, our 8B-A1B MoE setting is closer to realistic LLM training.

We have added FA training experiments on another 10B-A1.6B MoE model in Appendix F.4 of the revision, extending the training evidence to a different model configuration. In addition, our early experiments were repeated with three different random seeds, and the standard deviations across seeds were below 1% for almost all results, supporting the reproducibility of our main empirical observations.

Our monitor targets biased errors that are persistently introduced by an implementation or numerical computation and repeatedly propagated into Q/K updates through backpropagation. Applicability to a different implementation depends on whether its errors produce persistent update structure along this propagation path. An occasional perturbation that is absorbed by model redundancy or subsequent training dynamics does not produce persistent accumulation; a perturbation that immediately triggers INF is already exposed by global metrics. Windowed spectral monitoring focuses on the stage when persistent errors accumulate in parameter updates before global metrics deteriorate noticeably.

Our supplementary experiments provide O-only and dO-only ablations. In the FB1 comparison under the same training configuration, both runs have a loss of 2.328 at step 10k, while their QK functional-update spectra have already separated clearly. The O-only and dO-only loss trajectories subsequently diverge as well. Thus, spectral monitoring reveals differences in update structure caused by different error entry points while global loss has yet to distinguish the two runs. Appendix F.1 of the revision provides the detailed experimental results.

Weaknesses 1b & Question 1: Scope and Monitor Design

We propose a mechanism-aligned principle for designing monitoring representations: (1) start from the fault propagation mechanism, identify the internal states that it persistently affects, (2) derive the resulting structural changes, and (3) select a representation that corresponds to the module's computation. Section 3 of our paper presents this shared design process, while FA and the MoE router provide the corresponding mechanism analyses, monitor implementations, and empirical validation. FA and MoE are key components of LLM training, involving the accumulation of persistent numerical errors and the dynamics of routing decisions, respectively. These two examples demonstrate how the same design process yields different monitoring representations for different mechanisms.

The two monitoring mismatches describe complementary discrepancies between changes in the target and the monitor's response: a relevant target change that the monitor fails to reflect, and a monitor change despite unchanged target functionality, as shown in Rebuttal Table 1. This classification concerns the forms of mismatch; multiple mechanisms can give rise to each form, and FA and MoE are the two representative cases analyzed in our paper. We have clarified this relationship in Section 2 of the revision. Rebuttal Table 2 summarizes how the shared design principle and its validation apply to both cases.

Rebuttal Table 1: Two forms of mismatch between target changes and monitoring responses

Target \ MonitorUnchangedChange
UnchangedMismatch II: Insufficient functional fidelity
ChangedMismatch I: Insufficient fault sensitivity

Rebuttal Table 2: The shared mechanism-aligned design process for FA and the MoE router

Shared design stepFAMoE router
Identify the target mechanismBiased errors from low-precision backward computation repeatedly enter Q/K updatesLR, GBS, and AUX affect routing dynamics through relative expert-score margins
Locate the persistently affected stateRecent Q/K parameter updatesRouter parameters and the relative expert-score margins they induce
Derive the expected structureUnder conditions including a low-rank mean and controlled fluctuations, accumulated errors can concentrate the update spectrumFeedback that reinforces the current winning experts can drive routing probabilities toward concentration and saturation
Identify the mismatch in existing representationsThe historical component in raw weights can obscure recent update structureChanges in router weight similarity need not correspond to changes in the routing function
Construct a function-aligned representationMove from WW to windowed updates ΔW\Delta W, then use the bilinear QK computation to construct the functional increment ΔF\Delta FMove from parameter similarity to per-token routing entropy to track probability concentration and saturation

The scope of each monitor is determined by its target mechanism. The FA functional-update spectrum tracks structured numerical abnormalities that persistently enter Q/K updates; routing entropy tracks the concentration and saturation of routing probabilities. For faults that are not reflected in Q/K updates, or damage to expert outputs that leaves the routing distribution unchanged, the monitor should be constructed around the internal state actually affected. This is how the mechanism-aligned principle is applied. The same process can be applied to other modules to obtain monitors with an explicit mechanistic interpretation.

Weakness 1c: Cross-Model Checkpoint Analysis

The cross-model analysis examines the relationship between parameter similarity and routing functionality. We do not currently have access to the complete training trajectories of these external models, so this analysis focuses on the correspondence between their parameters and routing functions. Table 1 and Fig. 5 of our paper show that models can have similar normalized routing entropy but substantially different router weight similarity. These results show why parameter similarity alone is insufficient to characterize routing behavior.

Weakness 2a & Question 2a: Operational Warning System

Our monitoring representations support two modes of use: (1) comparison with a healthy reference trajectory, and (2) trend-based early warning within a single training run. When a training system moves to a different hardware platform, such as from NVIDIA to AMD, adopts a new FA implementation, or changes numerical precision, a validated healthy run provides a reference for identifying abnormal structural changes. When no matched healthy trajectory is available, online trend monitoring can instead use the history already observed in the current run.

Our paper has already used a trend criterion to identify the warning step. Update-spectrum entropy in healthy training tends to increase and then plateau. A fixed threshold on the raw spectral value can be insufficient to distinguish normal progress from abnormal concentration. Our implementation computes MACD using observations available up to the current step and triggers an alarm after 5 consecutive decreases in the resulting sequence.

We add an online sequential CUSUM warning procedure. We use the current run's early trajectory through 10k steps to estimate the background fluctuation scale and alarm threshold, as warnings during this initial stage have limited practical value. We then freeze these calibration parameters. Each new observation contributes evidence of a sustained decline in a spectral metric or a sustained increase in loss, and an alarm is confirmed after 2 consecutive threshold exceedances. Both calibration and subsequent sequential decisions use only the data available at the time, yielding explicit alarm without a healthy reference. Rebuttal Table 3 reports the results for each monitoring signal. In FB1 and FB0, the detector based on ΔFlin\Delta F_{\text{lin}} raises alarms 9k and 5.5k steps before the loss detector, and 6k and 2.5k steps before the detectors based on the QKV and QK-only update spectra. The healthy baseline trajectory triggers no alarms throughout the evaluation interval ending at 68k steps.

Rebuttal Table 3: CUSUM alarm counts and first alarm times. Entries are alarm count (first alarm step); -- indicates no alarm before the end of training.

Monitoring signalBaselineFB2FB1FB0
ΔFlin\Delta F_{\mathrm{lin}}0 (--)1 (54.5k)5 (29k)7 (16k)
QKV update spectrum0 (--)0 (--)4 (35k)5 (18.5k)
QK-only update spectrum0 (--)0 (--)4 (35k)5 (18.5k)
Loss0 (--)0 (--)3 (38k)2 (21.5k)

Gradient norm does not exhibit a sustained abnormal trend suited to the present one-sided trend rule in this experiment, so it is not included in the quantitative comparison using the same CUSUM procedure. Fig. 1(c) of our paper shows that pronounced gradient-norm abnormalities occur at a similar training stage to the loss upturn and HellaSwag accuracy decline. Spectral changes thus provide an earlier structural signal.

Weakness 2b & Question 2b: Update-Window Sensitivity

The window-sensitivity experiments are provided in Appendix F.3 of the revision. As analyzed in Sec 3.1, under conditions including a low-rank mean and controlled stochastic fluctuations, persistent biased errors accumulate within the update window. A moderately longer window can therefore make the resulting spectral structure more pronounced. In our implementation, the update-window length and the interval between weight snapshots share the parameter τ\tau. Shorter windows provide more frequent observations but are more affected by short-term fluctuations and incur higher collection overhead. Longer windows accumulate persistent changes while reducing the sampling frequency. The window choice therefore balances signal accumulation, temporal resolution, and monitoring cost.

In the supplementary experiments, we compare τ=10,50,200,500\tau=10,50,200,500 using the Baseline, FB2, FB1, and FB0 runs. For each window, we compare ΔFlin\Delta F_{\text{lin}} against the healthy Baseline with the same window. Rebuttal Table 4 reports the results for FB2, FB1, and FB0 at step 15k. For all three perturbations, the spectral deviations are larger with 200- and 500-step windows than with 10- and 50-step windows. These results are consistent with the accumulation of persistent errors within a window. They also show that the 500-step window captures persistent structural changes while balancing temporal resolution and end-to-end overhead.

Rebuttal Table 4: Deviations in the functional-update spectrum across update-window lengths

Update window τ\tau (steps)FB2FB1FB0
101.79%6.41%23.12%
503.95%8.88%29.70%
2007.56%15.94%42.63%
5008.40%19.62%48.24%
Reviewer_Nx8FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a mechanism-aligned approach to monitoring training instability in large language models. It investigates two scenarios: (1) low-precision FlashAttention (FA), where biased backward errors produce spectral concentration in accumulated weight updates, and (2) Mixture-of-Experts (MoE) routing, where per-token routing entropy characterizes routing concentration more directly than router-weight similarity. The authors develop spectral monitors based on QK functional updates and analyze the effects of learning rate, batch size, and auxiliary balancing on routing dynamics. Experiments on an 8B-parameter MoE model demonstrate that the proposed indicators can reveal structural changes before certain global training metrics exhibit degradation.

Critical strengths
  1. Relevant and practical problem. The paper addresses early detection and diagnosis of LLM training instability, an important challenge in large-scale model pretraining.

  2. Interesting mechanism-aligned perspective. Connecting numerical errors to spectral changes in attention updates and analyzing routing behavior through per-token entropy provides useful insights beyond conventional global training statistics.

  3. Meaningful theoretical and empirical analyses. The QK bilinear decomposition provides a principled functional representation, while experiments on an 8B MoE model demonstrate promising early-warning signals under controlled training perturbations.

Critical weaknesses
  1. Limited methodological novelty. The main approach combines existing techniques, including spectral analysis of weight updates and routing entropy monitoring. The FA error mechanism builds heavily on prior work, while the proposed low-rank accumulation analysis relies on assumptions about coherent errors. The contribution beyond existing training diagnostics and monitoring methods is not sufficiently established.

  2. Lack of quantitative early-warning evaluation. Although the paper shows that spectral changes can precede loss degradation, it does not establish an operational detection criterion or systematically evaluate detection precision, false-positive rates, or warning lead times. The results are largely based on visual comparisons with healthy trajectories, leaving practical deployment effectiveness unclear.

  3. Insufficient validation of routing entropy as an instability indicator. Low routing entropy does not necessarily imply harmful expert collapse, since confident expert specialization may be beneficial. The paper primarily correlates entropy with training hyperparameters without sufficiently establishing its relationship to expert utilization, prediction quality, or eventual training failure.

  4. Limited experimental generality and robustness. The experiments focus on one 8B MoE architecture and controlled low-precision FA perturbations. The absence of independent training runs and diverse architectures or naturally occurring failures limits confidence in the generality of the findings. Moreover, comparisons across batch sizes may be affected by differences in the number of training tokens processed at a given step.

  5. Unclear practical monitoring overhead. Although Appendix E demonstrates efficient low-rank spectral computation, the paper does not report end-to-end monitoring overhead during training, including state collection, synchronization, and monitoring frequency. Therefore, its scalability to production-scale LLM training remains uncertain.

Questions for authors

Q1. How would the proposed monitors operate as practical early-warning detectors rather than retrospective diagnostic indicators? Could the authors specify detection thresholds and report false-positive rates, detection latency, and warning lead times against loss-, gradient-, and update-based baselines?

Q2. In Section 3.1, the low-rank accumulation argument assumes that the expected biased update has low rank and that fluctuations are sufficiently controlled. How well do these assumptions hold in the actual training experiments? Could the authors empirically characterize the rank and temporal coherence of the fault-induced updates?

Q3. How can harmful routing collapse be distinguished from beneficial expert specialization? Could the authors examine whether low routing entropy predicts subsequent degradation in expert utilization, training loss, or downstream performance, rather than merely reflecting changes in routing confidence?

Q4. In the GBS experiments, different batch sizes process substantially different numbers of tokens at the same training step. Would the observed routing-entropy trends remain consistent under comparisons matched by total training tokens or compute budget? This distinction is important for attributing the results specifically to batch-size effects.

Q5. How does the proposed spectral monitor perform under naturally occurring numerical instability or alternative FA implementations, rather than deliberately injected precision truncation? Additionally, what is the end-to-end runtime overhead of collecting and computing these statistics at practical monitoring intervals?

6 comments

Thank you for the insightful comments!

Weakness 1: Limited Novelty

We propose a mechanism-aligned principle for designing monitoring representations: (1) start from the fault propagation mechanism, identify the internal states that are persistently affected, (2) derive the structure expected to emerge in those states, and (3) construct a representation aligned with the module's computation. This principle turns mechanism analysis into concrete monitoring quantities that can be collected, computed, and validated. It yields two concrete contributions for FA and MoE routers.

  1. Functional-update monitoring for FA. Building on the established mechanism of low-precision FA errors, we analyze how fault contributions accumulate within an update window and develop QK functional-update monitoring aligned with the bilinear computation of attention operator.
  2. Routing-behavior monitoring for MoE. Through invariance analysis under a common-vector shift and experiments on checkpoints from multiple models, we show that router weight similarity does not uniquely characterize routing behavior. We then analyze how training conditions affect routing dynamics through the relative expert-score margins that determine routing, and select per-token routing entropy to monitor probability concentration.

These representations also offer practical implementation benefits. FA monitoring requires only periodic Q/K parameter snapshots, without modifying the fused attention kernel, collecting attention logits, or performing additional downstream evaluations; it can run asynchronously or offline. Appendix E already shows that the singular spectrum can be computed from a small core of dimension at most 2dk2d_k. MoE routing entropy can be computed online directly from the routing probabilities already available.

Weakness 2 & Question 1: Operational Warning System

Our monitoring representations support two modes of use: (1) comparison with a healthy reference trajectory, and (2) trend-based early warning within a single training run. When a training system moves to a different hardware platform, such as from NVIDIA to AMD, adopts a new FA implementation, or changes numerical precision, a validated healthy run provides a reference for identifying abnormal structural changes. When no matched healthy trajectory is available, online trend monitoring can instead use the history already observed in the current run.

Our paper has already used a trend criterion to identify the warning step. Update-spectrum entropy in healthy training tends to increase and then plateau. A fixed threshold on the raw spectral value can be insufficient to distinguish normal progress from abnormal concentration. Our implementation computes MACD using observations available up to the current step and triggers an alarm after 5 consecutive decreases in the resulting sequence.

We add an online sequential CUSUM warning procedure. We use the current run's early trajectory through 10k steps to estimate the background fluctuation scale and alarm threshold, as warnings during this initial stage have limited practical value. We then freeze these calibration parameters. Each new observation contributes evidence of a sustained decline in a spectral metric or a sustained increase in loss, and an alarm is confirmed after 2 consecutive threshold exceedances. Both calibration and subsequent sequential decisions use only the data available at the time, yielding explicit alarm without a healthy reference. Rebuttal Table 1 reports the results for each monitoring signal. In FB1 and FB0, the detector based on ΔFlin\Delta F_{\text{lin}} raises alarms 9k and 5.5k steps before the loss detector, and 6k and 2.5k steps before the detectors based on the QKV and QK-only update spectra. The healthy baseline trajectory triggers no alarms throughout the evaluation interval ending at 68k steps.

Rebuttal Table 1: CUSUM alarm counts and first alarm times. Entries are alarm count (first alarm step); -- indicates no alarm before the end of training.

Monitoring signalBaselineFB2FB1FB0
ΔFlin\Delta F_{\mathrm{lin}}0 (--)1 (54.5k)5 (29k)7 (16k)
QKV update spectrum0 (--)0 (--)4 (35k)5 (18.5k)
QK-only update spectrum0 (--)0 (--)4 (35k)5 (18.5k)
Loss0 (--)0 (--)3 (38k)2 (21.5k)

Gradient norm does not exhibit a sustained abnormal trend suited to the present one-sided trend rule in this experiment, so it is not included in the quantitative comparison using the same CUSUM procedure. Fig. 1(c) of our paper shows that pronounced gradient-norm abnormalities occur at a similar training stage to the loss upturn and HellaSwag accuracy decline. Spectral changes thus provide an earlier structural signal.

Question 2: Empirical Verification of Low-Rank Hypothesis

We understand the question to concern whether low-rank bias and controlled fluctuations are consistent with actual training, and whether the rank and temporal coherence of the updates support the accumulation explanation. We therefore examine two directly relevant observable properties: (1) whether update energy is concentrated in a few directions within each window, and (2) whether these dominant directions persist across windows.

Spectral concentration within a window. Rebuttal Tables 2 and 3 report Stable Rank and Singular Spectrum of the training runs in Fig. 1. All three perturbation conditions have lower values than the healthy Baseline, with the largest decrease in FB0, indicating concentrated update energy.

Rebuttal Table 2: Stable Rank of windowed updates at different training stages

StepBaselineFB2FB1FB0
20k185.70142.19115.1636.58
30k149.41145.9897.6413.59
40k199.37159.6338.882.63
50k175.47133.5750.963.94
60k194.26137.0723.952.29

Rebuttal Table 3: Singular Spectrum of windowed updates at different training stages

StepBaselineFB2FB1FB0
20k739.70732.84711.90444.84
30k751.57748.05689.71218.83
40k769.67751.97495.0537.63
50k770.09709.85502.6089.86
60k778.19698.07352.2810.37

Subspace overlap between adjacent windows. This metric tests whether the dominant input directions of an update in one window persist in the next. At a fixed subspace dimension rr, greater overlap in a fault condition than in the healthy Baseline indicates that adjacent updates more consistently occupy similar dominant input subspaces. In the supplementary experiment, we compare two adjacent, non-overlapping 500-step updates, At=Wt−500−Wt−1000A_t=W_{t-500}-W_{t-1000} and Bt=Wt−Wt−500B_t=W_t-W_{t-500}. For each Q/K head, we take the leading rr right singular vectors of each update and compute their mean projection overlap, SR=∥VA,r⊤VB,r∥F2/rS_R=\lVert V_{A,r}^{\top}V_{B,r}\rVert_F^2/r. Rebuttal Table 4 reports the results over 15k–20k steps with r=4r=4.

Rebuttal Table 4: Q/K update subspace overlap between adjacent windows

ConditionQ update SRS_RK update SRS_R
Baseline0.0550.057
FB20.1780.199
FB10.2740.341
FB00.4240.472

These results support concentration into low-dimensional structure and its persistence across windows. In the main experiment, update spectra become more concentrated as perturbation strength increases. In the supplementary experiment, dominant input-subspace overlap also increases with perturbation strength, while the energy fraction captured by the leading four singular directions of the Q update rises from 28.33% in the Baseline to 85.11% in FB0. These measurements provide support for this mechanism in actual training.

Weakness 3 & Question 3: Routing Concentration and Beneficial Specialization

Our supplementary experiments distinguish normal routing concentration from persistent saturation by examining actual gating participation and subsequent prediction quality. We conduct an LR × AUX comparison. All four runs branch from the same 500-step checkpoint and train to 20k steps. We evaluate pure LM NLL on the same held-out set to directly compare prediction quality. Rebuttal Table 5 summarizes routing behavior and prediction results at 20k steps.

Rebuttal Table 5: Effects of LR and AUX on routing behavior and held-out prediction quality

Config (LR, AUX)Normalized entropyNeffN_{\mathrm{eff}}qmax⁡q_{\max}MaxVioLM NLL ↓
A (0.001, 0)0.5782.7530.6043.8753.94981
B (0.001, 0.001)0.6702.9370.5731.4843.95488
C (0.01, 0)0.1151.3880.8794.3034.54274
D (0.01, 0.001)0.1541.4810.8642.0864.54156

Here, qq denotes the normalized top-k gating weights actually used in execution, Neff=1/∑e∈Sqe2N_{\mathrm{eff}}=1/\sum_{e\in S}q_e^2, and qmax⁡=max⁡e∈Sqeq_{\max}=\max_{e\in S}q_e. NeffN_{\mathrm{eff}} measures effective participation as reflected in the gating weights; MaxVio measures imbalance in the aggregate dispatch load. Routing metrics are averaged over MoE layers.

The training trajectories reveal two distinct regimes. In A/B, routing entropy recovers after its initial decline while NLL continues to improve. In C/D, entropy remains low and NLL plateaus earlier at a higher level. By 20k steps, holding AUX fixed, C versus A and D versus B both have a held-out NLL disadvantage of approximately 0.59. These comparisons distinguish recoverable initial concentration from persistent saturation accompanied by poorer subsequent prediction quality.

The gating weights clarify what low entropy means for expert participation. C/D have a maximum gating weight of approximately 0.87, and their effective gating participation is approximately 1.4, compared with 2.75–2.94 in A/B. Persistent low entropy thus reflects sustained concentration in the executed top-k gating mixture. The AUX comparisons further test whether aggregate load balance adequately captures these states. Adding AUX to C reduces MaxVio from 4.303 to 2.086 in D, yet NLL remains approximately 4.54 and per-token gating remains highly concentrated. D has better aggregate load balance than A (MaxVio: 2.086/3.875), but worse prediction quality (NLL: 4.54/3.95), together with lower routing entropy and effective gating participation. Improved aggregate load balance therefore does not imply that the quality deficit has disappeared. The proposed per-token routing entropy reveals this, which aggregate load statistics do not fully capture.

Weakness 4a: Experimental Coverage and Reproducibility

Our choice of monitoring representations is supported jointly by model-independent structural results and empirical tests during actual training. Invariance of the routing function under a common-vector shift and the bilinear decomposition of joint QK changes provide the structural basis for these choices. The 8B-A1B model used in our main experiments allows both FA and router modules to be examined in complete end-to-end training. Different monitoring representations share the same training states and fault conditions, enabling a direct comparison of how representation choice affects fault observability. Compared with the GPT-2-scale models used in much of the related work, our 8B-A1B MoE setting is closer to realistic LLM training.

We have added FA training experiments on another 10B-A1.6B MoE model in Appendix F.4 of the revision, extending the training evidence to a different model configuration. In addition, our early experiments were repeated with three different random seeds, and the standard deviations across seeds were below 1% for almost all results, supporting the reproducibility of our main empirical observations.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear reviewers,

Thank you for your careful reading and constructive feedback. We appreciate the your recognition of the following strengths of our work:

  • The importance and practical relevance of diagnosing training instability through internal states, beyond global training statistics or downstream evaluation metrics (Reviewers Nx8F, VDTU).
  • Useful mechanistic insights and well-motivated monitoring representations, including recent QK updates and per-token routing behavior (Reviewers Nx8F, bYfT).
  • Meaningful empirical analyses and targeted comparisons examining the proposed monitoring choices (Reviewers Nx8F, bYfT).
  • Clear writing and presentation (Reviewers Nx8F, bYfT).

During the discussion period, we provided detailed responses, additional experiments, and clarifications addressing the reviewers' concerns. We have also uploaded a revision with changes highlighted in red. Below, we summarize the main updates to each review.

To Reviewer Nx8F: We clarified our contribution. To make early warning operational and quantitatively assessable, we added a sequential CUSUM procedure calibrated using the current run’s early observations. We also provided measurements of update spectral concentration and the persistence of dominant subspaces across windows to support the accumulation mechanism. For the router, we added LR × AUX experiments distinguishing recoverable early concentration from persistent training degradation.

To Reviewer bYfT: We clarified the scope of the two monitors, the shared design principle, and what the cross-model checkpoint analysis establishes. We explained calibration without a matched healthy trajectory and added update-window sensitivity experiments covering 10, 50, 200, and 500 steps (Appendix F.3), together with O-only/dO-only ablations examining different error entry points (Appendix F.1). To assess practical cost, we measured end-to-end training overhead increase (Appendix F.2).

To Reviewer VDTU: We explained the shared mechanism-aligned design process connecting the FA and MoE instances, clarified the contributions relative to prior diagnostic approaches, and summarized the FA experimental setup. We expanded the FA training evaluation to an additional 10B-A1.6B MoE model (Appendix F.4). We also added WinoGrande evaluations (Appendix F.5), complementing the existing HellaSwag results and further relating early internal signals to subsequent degradation in task performance.

We would greatly appreciate feedback from each reviewer on whether these clarifications address their concerns, and which issues, if any, remain unresolved. We would be happy to discuss any remaining questions and provide further clarification during the discussion period.

Best regards,
Authors of submission 6817

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.