acceptodds
Under review as a conference paper at ICLR 2027

Doing More with Less: Monitoring Generalization During Pretraining via Virtual MoE

Abstract

How to monitor the generalization of dense LLMs as "generalist solvers" during their pretraining? A typical strategy is to evaluate each intermediate checkpoint on benchmarks of downstream tasks, which is costly and offers limited insights into the memorization-to-generalization transition. We propose to track an LLM's generalization by its capability of "doing more with less", i.e., generating diverse, rich outputs by fewer, highly reusable programs. We further discover that such a property can be captured by the pretraining dynamics of model internal states without additional finetuning or benchmark evaluation. Recent studies reveal a strong correlation between the generalization of Mixture-of-Experts (MoE) LLMs and its pathway (expert choices across layers) reusability across samples. Motivated by this, we develop a "virtual MoE" view of layers in dense LLMs by training a CrossCoder as a routing proxy, which enables us to compute and compare pathways between samples. Across the pretraining checkpoints of Pythia (160M-1B), OLMo-1B, and OLMo2-1B, we observe that (1) the edit distance between "virtual" pathways of relevant samples consistently decreases, which indicates their improved reusability; (2) within shared "virtual" pathways, latent representations become more discriminatively structured, suggesting that reusable pathways preserve input-specific distinctions needed for downstream generalization. Moreover, these two trends strongly correlate with generalization performance observed on downstream tasks. Hence, the two metrics form an efficient monitor, which tracks generalization capability and provides a mechanistic interpretation of pretraining.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, LLM pretraining, generalization, mechanistic interpretability, mixture-of-experts

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cKLiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a virtual MoE framework for monitoring generalization during dense LLM pretraining. The authors use a CrossCoder to construct virtual experts and computational pathways, and introduce two metrics, pathway edit distance and conditional state diversity, to measure pathway reuse and representation diversity. These metrics are further combined into an Adaptive Reuse Ratio. Experiments across Pythia, OLMo, and OLMo2 checkpoints show that the proposed metrics correlate with downstream generalization performance. The combined metric also achieves better benchmark performance prediction than loss-based baselines, suggesting its potential for monitoring pretraining dynamics.

Critical strengths

The paper addresses a meaningful problem of monitoring generalization during LLM pretraining and proposes an interesting and creative approach by introducing a virtual MoE perspective for analyzing dense models without modifying their architectures. Combining pathway reuse and conditional state diversity offers a novel and potentially useful way to understand pretraining dynamics. Additionally, the experiments cover multiple model families and data domains, showing consistent correlations with downstream performance. The proposed metrics also outperform loss-based baselines in the reported prediction experiment, providing preliminary evidence of their practical value.

Critical weaknesses

The proposed connection between reusable computation and generalization remains largely heuristic. The paper assumes that better generalization is associated with increased pathway reuse and richer representations within shared pathways. Although the experiments demonstrate correlations with downstream performance, they do not establish a causal relationship. Moreover, the theoretical analysis relies on additional assumptions connecting the proposed empirical metrics to generalization. Therefore, the claimed mechanistic interpretation is not sufficiently supported.

The effectiveness of the proposed framework for practical online monitoring is insufficiently validated. The main experiments train the CrossCoder using checkpoints sampled throughout the entire pretraining trajectory, requiring access to future checkpoints. Although Appendix D.2 evaluates CrossCoders trained only on early checkpoints, it reports reconstruction errors rather than generalization monitoring performance. Since accurate reconstruction does not necessarily guarantee consistent virtual routing patterns, it remains unclear whether the proposed metrics retain their predictive effectiveness in a realistic online setting.

The computational efficiency of the proposed monitoring framework is not sufficiently evaluated. Although the paper aims to reduce the cost of monitoring generalization during pretraining, training the CrossCoder requires collecting intermediate activations from multiple checkpoints and introduces additional computational overhead. However, the paper does not provide a detailed analysis of the end-to-end computational cost or compare it with directly evaluating downstream benchmarks. Therefore, it remains unclear whether the proposed framework offers a practical efficiency advantage over conventional evaluation methods.

Questions for authors

Q1: Could the authors provide stronger theoretical or experimental evidence supporting the connection between pathway reuse, conditional state diversity, and generalization, beyond the observed correlations?

Q2: Could the authors evaluate whether CrossCoders trained only on early checkpoints can reliably monitor and predict generalization performance at later checkpoints, rather than only reporting reconstruction errors?

Q3: Could the authors report the computational cost of training the CrossCoder and the overall monitoring pipeline, and compare it with the cost of directly evaluating downstream benchmarks?

3 comments
AuthorsOpenReview ↗

Response to Reviewer cKLi

We thank the reviewer for recognizing the value of generalization monitoring, the virtual-MoE framework, and the proposed metrics. We address the three questions below.

Q1. Connection between reusable computation and generalization

We provide complementary theoretical and predictive evidence supporting the connection between pathway reuse, conditional state diversity, and generalization.

Theoretically, Eq. (1) and Appendix B establish a conditional generalization bound involving pathway description complexity and task-relevant representation separability. Greater pathway reuse can reduce the description-complexity penalty, while preserved separability reduces downstream readout complexity. This provides a theoretical motivation for jointly monitoring pathway reuse and conditional state diversity, rather than pathway reuse alone.

Empirically, Table 2 evaluates benchmark-score prediction using pathway statistics and historical evaluations, without using the current checkpoint's benchmark score. The predictor using AtA_t achieves 72.9% direction accuracy, compared with 65.7% for loss-based controls. This provides predictive evidence that the joint structural signal captures information about downstream generalization beyond training loss.

Q2. Early-only CrossCoders and future-checkpoint monitoring

We further evaluate future-checkpoint monitoring by freezing CrossCoders trained on the first four (Early-4) or six (Early-6) checkpoints of Pythia-410M. We then compute PtP_t, RtR_t, and AtA_t on the remaining checkpoints, respectively, without retraining, using the full-trajectory CrossCoder (Full-10) as a reference.

Table R2. Pearson correlations between monitoring metrics and downstream accuracy.

CrossCoder settingr(Pt,Acc)r(P_t,\mathrm{Acc})r(Rt,Acc)r(R_t,\mathrm{Acc})r(At,Acc)r(A_t,\mathrm{Acc})
Full-10-0.47+0.68+0.62
Early-4-0.50+0.49+0.59
Early-6+0.26+0.77+0.79

Both early-only CrossCoders retain informative generalization-monitoring signals at later checkpoints. Early-4 achieves a correlation of +0.59 for AtA_t, while Early-6 achieves +0.77 and +0.79 for RtR_t and AtA_t, respectively. These results provide direct evidence of future-checkpoint monitoring with frozen early-trained CrossCoders, extending the reconstruction-transfer findings in Appendix D.1.

Q3. Computational cost of training the CrossCoder and the overall monitoring pipeline

Our primary goal is to monitor and interpret how generalization develops during pretraining, rather than primarily to reduce the cost of downstream benchmark evaluation. We distinguish one-time CrossCoder training from recurring per-checkpoint pathway analysis. The CrossCoder can be trained on early checkpoints and reused for later checkpoints, amortizing its training cost. Moreover, pathway analysis operates on input-side activations without autoregressive decoding.

For Pythia-410M, training the CrossCoder on the first four checkpoints costs an estimated 1.53 GPU-hours on one RTX A6000. For a fair runtime comparison, we evaluated pathway analysis and autoregressive generation on the same set of prompts. A complete pathway analysis for a later checkpoint averaged 79 s, compared with 149 s and 300 s for generating up to 128 and 256 tokens, respectively. Notably, CrossCoder encoding itself took only 5.25 s. The full per-checkpoint cost is broken down below:

ComponentTime
Model and CrossCoder loading20.06 s
Activation processing and CrossCoder encoding (including 5.25 s for encoding)33.85 s
Expert clustering16.84 s
Pathway metric computation8.32 s
Total79.07 s

These results demonstrate that our framework provides structural insights into generalization dynamics with an amortizable training cost and lower recurring overhead than the measured generation-based evaluations.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

We would like to kindly follow up on our rebuttal, where we have provided additional experiments on early-checkpoint monitoring and computational efficiency, along with further clarification of the connection to generalization.

We would greatly appreciate any further feedback you may have during the discussion period.

Thank you for your time and consideration!

Thank you for the clarifications and additional results. The early-only CrossCoder experiments and cost breakdown alleviate my concerns about online feasibility and monitoring overhead.

However, the link between conditional diversity and task-relevant separability still depends on the alignment assumption. Whether Table 2’s predictive gains persist with an early-only CrossCoder also remains unclear. The total cost advantage over actual benchmark evaluation is not yet established.

I appreciate these improvements, but the remaining concerns lead me to maintain my original score.

Reviewer_emRHOpenReview ↗
Rating
2(+1)
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces two metrics to characterize pathway reuse and discriminative structure within shared computational pathways, and uses them to explain the evolution of generalization during model training.

Critical strengths
  1. The experiments cover two model families across multiple model scales.
  2. The reported results do show associations between the proposed metrics and downstream task performance, suggesting their potential as predictive indicators.
Critical weaknesses
  1. The models used in the experiments are outdated: both Pythia and OLMo were publicly available before 2025. There is no evidence that the conclusions extend to frontier models.
  2. The paper is not sufficiently self-contained: key methodological details are not clearly explained, and some figures supporting the experimental conclusions appear only in the appendix.
  3. The central hypothesis that shared pathways preserve task-relevant discriminative representations lacks direct empirical validation. The interpretation of Rₜ as a proxy for task-relevant separability relies on an untested task-alignment assumption, weakening the evidence for the functional interpretation of “efficient compression.”
Questions for authors
  1. Some figures supporting the experimental conclusions are presented only in the appendix, such as those associated with Section 4.3. I am unsure whether this complies with the relevant guidelines.
  2. The description of the core methodological pipeline is insufficiently clear. Although the paper introduces CrossCoder to construct virtual experts, it does not explicitly explain how activations from different checkpoints and layers are organized as inputs, or how checkpoint- and layer-specific feature selections and routing decisions are obtained. Although the appendix provides hyperparameters and training procedures, it does not supply the missing architectural and data-flow definitions. Since the subsequent pathway-distance and generalization-monitoring metrics depend on this construction, these omissions make the procedure difficult to understand and reproduce unambiguously, and make it difficult to assess the validity of cross-checkpoint comparisons.
  3. The paper lacks direct validation of its central hypothesis that shared computational pathways preserve discriminative representations. Specifically, Rₜ in Section 3.3 measures only the normalized representation distance between pathway-similar samples; it cannot distinguish improvements in task-relevant separability from task-irrelevant within-class variation. Although Appendix B explicitly introduces a task-alignment assumption, the experiments do not test this assumption. Furthermore, because both the selected sample pairs and the normalization denominator change across checkpoints, an increase in Rₜ does not directly demonstrate that the same group of inputs retains more task-relevant information under shared computation. I suggest tracking a fixed group of samples across checkpoints and reporting the between-class and within-class structure, or task decodability, of representations conditioned on similar pathways. This would help validate the functional interpretation of “efficient compression,” rather than relying solely on correlations between the metrics and downstream scores.
3 comments
AuthorsOpenReview ↗

Response to Reviewer emRH

We thank the reviewer for the feedback on experimental scope, methodological clarity, and the task relevance of conditional state diversity.

W1. Model scale and experimental scope

We evaluate five public pretraining trajectories across three model families (Pythia, OLMo, and OLMo2), covering model sizes up to 1B parameters. Our analysis requires multiple intermediate checkpoints from the same training run, which are generally unavailable for frontier-scale models. These experiments establish the observed training dynamics across the evaluated architectures and scales, while extending the analysis to larger models remains an important future direction.

Q1. Placement of supporting figures

The ICLR 2027 guidelines permit supporting figures in the appendix. Due to the main-text page limit, we originally placed the full domain-wise trajectories (Figures 5–6) in Appendix D.1, with the key findings summarized in Section 4.3. To improve clarity, we have moved Figures 5–6 into the main text in the revised manuscript.

Q2. Cross-checkpoint activation organization and routing construction

We train a single CrossCoder on jointly organized activations from multiple checkpoints and transformer layers, establishing a shared latent feature space. Since the relevance of these features can differ across checkpoints, we construct checkpoint-specific virtual experts by grouping relevant latent features according to their co-activation patterns. We then use these experts to construct pathways and compare pathway-distance statistics computed within each checkpoint, rather than directly matching expert identities across checkpoints.

Feature extraction. After training, the frozen CrossCoder jointly encodes activations organized across checkpoints and layers and returns a shared token-level latent-selection mask. Max-pooling over valid tokens yields the sample-level indicator bi,kb_{i,k}, which is used with each checkpoint- and layer-specific expert set.

Virtual expert and pathway construction. For each checkpoint and layer, we cluster CrossCoder latent features based on their co-activation patterns across calibration inputs, after filtering out weak features. For each input, we count its activated features within each expert, select the ordered top-$K_{\mathrm{route}}$ experts at each layer, and concatenate these selections to form its virtual pathway.

Cross-checkpoint monitoring. At each checkpoint, we compute the average pathway edit distance between inputs using that checkpoint's virtual experts. The same frozen CrossCoder, input samples, and construction procedure are used throughout pretraining. Thus, we track how the checkpoint-specific organization of shared latent features changes pathway diversity, without requiring expert-ID alignment across checkpoints.

We have clarified these architectural and data-flow definitions in the revised manuscript.

Q3. Fixed-pair validation of efficient compression

To address the concern that sample-pair selection and normalization change across checkpoints, we conduct a fixed-pair analysis of the efficient-compression phase (64k–143k) identified in Figure 4 for Pythia-1B.

Specifically, we rank all pairs from 96 Pile samples by their pathway edit distance at 64k and select the most pathway-similar pairs at different percentile thresholds (5%–50%). We then track the same fixed pairs across subsequent checkpoints without reselecting them, measuring pathway distance and unnormalized representation distance to eliminate changes caused by pair selection and the normalization denominator.

Table R3. Fixed-pair analysis during the Pythia-1B efficient-compression phase (Δ=143k−64k\Delta = 143k - 64k).

Pathway similarity percentile at 64kFixed pairsΔ\Delta Pathway Distance (143k−64k)(143k - 64k)Δ\Delta Raw Representation Distance (143k−64k)(143k - 64k)
5%239-0.50+0.0323
10%456-2.86+0.0432
20%932-5.64+0.0583
30%1,414-6.49+0.0723
40%1,881-6.74+0.0860
50%2,312-6.74+0.0980

Across all thresholds, the same fixed pairs exhibit decreasing pathway distance and increasing raw representation distance. These results provide direct fixed-pair evidence for the net efficient-compression pattern in Figure 4, showing that increased pathway reuse coexists with preserved representation diversity, independent of changing pair selection or normalization.

Thank you for the additional experiments. While the task relevance of the observed representation differences still needs validation, these results partially address my concerns. I will therefore raise my rating.

Thank you for taking the time to consider our additional experiments and for updating your rating. We appreciate your constructive feedback!

Reviewer_yrd6OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper discusses a method for evaluating dense LLMs during pretraining, where performance evaluation is usually difficult. The basic idea is to apply the concept of pathways in MoE models to dense LLMs. To achieve this, the paper proposes to train a CrossCoder (a variant of SAEs) on multiple dense checkpoints, and to regard the top-k distribution of the latent representations extracted from the CrossCoder as virtual routing. The experimental results show that the complexity of pathways on the virtual routing behaves inversely to downstream performance (the higher the performance, the more unified the routing becomes) and the paper also notes that differentiation and integration of routing (discussed as "expansion-compression" in the text) occur during training.

Critical strengths
  • The idea of regarding features obtained by SAEs as routing is interesting. The motivation presented in Fig. 1 and Fig. 2 is also easy to understand. In particular, I think it is reasonable that the paper introduces not only the complexity of pathways but also the diversity of the resulting representations as a measure to capture the behavior of LLMs as "generalist solvers."
  • Within the range of models tested, the "expansion-compression" pattern is observed for the 1B-scale models, and it may be a training dynamic common to models of this scale. Although it would be difficult to directly use this result to argue anything, the existence of common training dynamics increases the possibility of reusing knowledge from past models when training new models, so I think it is an important finding.
Critical weaknesses
  • Even including the appendix, some parts of the CrossCoder training method and the method for extracting latent representations from the trained CrossCoder are unclear, and the experiments cannot be reproduced as they are. In addition, the paper describes the proposed method as "efficient" and "lightweight." This may be true in theory, but a numerical comparison is missing.

  • There are similar prior studies [1], [2] that are neither cited nor compared. I think it is difficult to claim novelty from the perspective of using CrossCoders to analyze the training dynamics of dense models. There is also a prior study [3] on "expansion-compression." Since no literature from the past year is cited, the survey of research after a certain point in time seems insufficient.

  • [1] Deniz Bayazit, Aaron Mueller, Antoine Bosselut. "Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretraining." In Proc. ACL, 2026.

  • [2] Xuyang Ge, Wentao Shu, Jiaxing Wu, Yunhua Zhou, Zhengfu He, Xipeng Qiu. "Evolution of Concepts in Language Model Pre-Training." In Proc. ICLR, 2026.

  • [3] Melody Zixuan Li, Kumar Krishna Agrawal, Arna Ghosh, Komal Kumar Teru, Adam Santoro, Guillaume Lajoie, Blake A. Richards. "Tracing the Representation Geometry of Language Models from Pretraining to Post-training." In Proc. HiLD at ICML, and NeurIPS, 2025.

Questions for authors
  • Please provide more concrete reproduction steps for the CrossCoder-related parts. In particular, please state explicitly which values (or sets of values) are input at each step and which hyperparameters are used.
  • The behavior and the value ranges of the benchmark scores seem to differ between Fig. 3 and Fig. 4, why is this?
  • Are the values of "Residual r" in Table 3 correct, especially their signs? Since the generality cannot be judged from the results of Pythia-160M alone, please also show the results for other models.
2 comments
AuthorsOpenReview ↗

Response to Reviewer yrd6

We thank the reviewer for the feedback on reproducibility, computational cost, related work, and result interpretation. We address each point below.

W1. Computational efficiency

The CrossCoder is lightweight in practice, with a one-time training cost amortized over subsequent monitoring. On Pythia-410M, training an Early-4 CrossCoder takes approximately 1.53 GPU-hours on a single RTX A6000, after which it can be reused across later checkpoints without retraining. For the same set of inputs, CrossCoder encoding takes only 5.25 seconds, while the complete pathway analysis takes 79.07 seconds. For comparison, autoregressive generation on these inputs takes 149 and 300 seconds with maximum budgets of 128 and 256 tokens, respectively.

W2. Related work and novelty

Our novelty lies in pathway-level generalization monitoring, rather than CrossCoder-based feature tracking or expansion–compression alone. Specifically, we construct discrete virtual pathways from CrossCoder features and jointly characterize pathway reuse and conditional state diversity to monitor generalization during pretraining.

The suggested studies address complementary questions: Bayazit et al. and Ge et al. examine feature and concept evolution during pretraining, while Li et al. investigate expansion–compression in representation geometry. We thank the reviewer for highlighting these relevant works and have incorporated them into the revised manuscript.

Q1. CrossCoder training and reproduction details

Appendix C.3 and Table 5 specify the CrossCoder training configuration, including checkpoint selection, activation inputs, dictionary sizes, latent top-kk, optimization hyperparameters, and the six-stage training procedure.

To clarify the data flow, a single CrossCoder jointly processes activations from multiple checkpoints and transformer layers. After training, the frozen CrossCoder extracts token-level active latent features, which are aggregated across valid tokens for each input. For each checkpoint and layer, we construct virtual experts by filtering the corresponding latent features using decoder-vector norms and clustering their co-activation patterns. We then count each input's active features within every expert and select the ordered top-KrouteK_{\mathrm{route}} experts at each layer to construct its pathway. These input-output relationships are now explicitly described in Sections 3.1–3.2 and Appendix C.3 of the revised manuscript.

Q2. Differences between Figures 3 and 4

Both figures use the same benchmark scores. Figure 3 shows smoothed trajectories to highlight the overall relationship between pathway complexity and downstream performance, while Figure 4 uses raw scores to preserve checkpoint-level training dynamics.

Q3. Residual correlation signs and generality

The signs in Table 3 are correct. After separately regressing PED and benchmark accuracy on a control variable, the residual correlation need not preserve the original sign. We focus on the absolute correlation, ∣r∣|r|, to assess the strength of the remaining linear association.

We extend the analysis to all five models, using 10 checkpoints and controlling separately for log(step) and training loss. For each control, Macro reports ∣r∣|r| computed from the 20-task averaged accuracy; Q50/Q70/Q90 report the corresponding quantiles of task-level ∣r∣|r|.

ModelControl log(step): Macro; Q50 / Q70 / Q90Control training loss: Macro; Q50 / Q70 / Q90
Pythia-160M0.925; 0.705 / 0.770 / 0.8230.760; 0.376 / 0.507 / 0.734
Pythia-410M0.897; 0.508 / 0.578 / 0.7860.219; 0.359 / 0.490 / 0.685
Pythia-1B0.775; 0.343 / 0.537 / 0.8190.018; 0.286 / 0.378 / 0.697
OLMo-1B0.185; 0.128 / 0.241 / 0.3790.051; 0.128 / 0.266 / 0.401
OLMo2-1B0.141; 0.264 / 0.367 / 0.6120.016; 0.290 / 0.347 / 0.613

These results show that training step and loss do not consistently explain the PED–accuracy association across models. Strong residual correlations remain in several models, while near-zero macro correlations can mask stronger task-level associations. Even when global controls capture overall performance trends, they do not directly characterize domain-specific computation reuse, which our pathway-based analysis captures.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

We would like to kindly follow up on our rebuttal and the revised manuscript, where we have clarified the methodological details and provided additional results addressing your concerns.

We would greatly appreciate any further feedback you may have during the discussion period.

Thank you for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.