- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a virtual MoE framework for monitoring generalization during dense LLM pretraining. The authors use a CrossCoder to construct virtual experts and computational pathways, and introduce two metrics, pathway edit distance and conditional state diversity, to measure pathway reuse and representation diversity. These metrics are further combined into an Adaptive Reuse Ratio. Experiments across Pythia, OLMo, and OLMo2 checkpoints show that the proposed metrics correlate with downstream generalization performance. The combined metric also achieves better benchmark performance prediction than loss-based baselines, suggesting its potential for monitoring pretraining dynamics.
Critical strengths
The paper addresses a meaningful problem of monitoring generalization during LLM pretraining and proposes an interesting and creative approach by introducing a virtual MoE perspective for analyzing dense models without modifying their architectures. Combining pathway reuse and conditional state diversity offers a novel and potentially useful way to understand pretraining dynamics. Additionally, the experiments cover multiple model families and data domains, showing consistent correlations with downstream performance. The proposed metrics also outperform loss-based baselines in the reported prediction experiment, providing preliminary evidence of their practical value.
Critical weaknesses
The proposed connection between reusable computation and generalization remains largely heuristic. The paper assumes that better generalization is associated with increased pathway reuse and richer representations within shared pathways. Although the experiments demonstrate correlations with downstream performance, they do not establish a causal relationship. Moreover, the theoretical analysis relies on additional assumptions connecting the proposed empirical metrics to generalization. Therefore, the claimed mechanistic interpretation is not sufficiently supported.
The effectiveness of the proposed framework for practical online monitoring is insufficiently validated. The main experiments train the CrossCoder using checkpoints sampled throughout the entire pretraining trajectory, requiring access to future checkpoints. Although Appendix D.2 evaluates CrossCoders trained only on early checkpoints, it reports reconstruction errors rather than generalization monitoring performance. Since accurate reconstruction does not necessarily guarantee consistent virtual routing patterns, it remains unclear whether the proposed metrics retain their predictive effectiveness in a realistic online setting.
The computational efficiency of the proposed monitoring framework is not sufficiently evaluated. Although the paper aims to reduce the cost of monitoring generalization during pretraining, training the CrossCoder requires collecting intermediate activations from multiple checkpoints and introduces additional computational overhead. However, the paper does not provide a detailed analysis of the end-to-end computational cost or compare it with directly evaluating downstream benchmarks. Therefore, it remains unclear whether the proposed framework offers a practical efficiency advantage over conventional evaluation methods.
Questions for authors
Q1: Could the authors provide stronger theoretical or experimental evidence supporting the connection between pathway reuse, conditional state diversity, and generalization, beyond the observed correlations?
Q2: Could the authors evaluate whether CrossCoders trained only on early checkpoints can reliably monitor and predict generalization performance at later checkpoints, rather than only reporting reconstruction errors?
Q3: Could the authors report the computational cost of training the CrossCoder and the overall monitoring pipeline, and compare it with the cost of directly evaluating downstream benchmarks?
3 comments
Follow-up on Our Rebuttal
Dear Reviewer,
We would like to kindly follow up on our rebuttal, where we have provided additional experiments on early-checkpoint monitoring and computational efficiency, along with further clarification of the connection to generalization.
We would greatly appreciate any further feedback you may have during the discussion period.
Thank you for your time and consideration!
Thank you for the clarifications and additional results. The early-only CrossCoder experiments and cost breakdown alleviate my concerns about online feasibility and monitoring overhead.
However, the link between conditional diversity and task-relevant separability still depends on the alignment assumption. Whether Table 2’s predictive gains persist with an early-only CrossCoder also remains unclear. The total cost advantage over actual benchmark evaluation is not yet established.
I appreciate these improvements, but the remaining concerns lead me to maintain my original score.
Response to Reviewer cKLi
We thank the reviewer for recognizing the value of generalization monitoring, the virtual-MoE framework, and the proposed metrics. We address the three questions below.
Q1. Connection between reusable computation and generalization
We provide complementary theoretical and predictive evidence supporting the connection between pathway reuse, conditional state diversity, and generalization.
Theoretically, Eq. (1) and Appendix B establish a conditional generalization bound involving pathway description complexity and task-relevant representation separability. Greater pathway reuse can reduce the description-complexity penalty, while preserved separability reduces downstream readout complexity. This provides a theoretical motivation for jointly monitoring pathway reuse and conditional state diversity, rather than pathway reuse alone.
Empirically, Table 2 evaluates benchmark-score prediction using pathway statistics and historical evaluations, without using the current checkpoint's benchmark score. The predictor using At achieves 72.9% direction accuracy, compared with 65.7% for loss-based controls. This provides predictive evidence that the joint structural signal captures information about downstream generalization beyond training loss.
Q2. Early-only CrossCoders and future-checkpoint monitoring
We further evaluate future-checkpoint monitoring by freezing CrossCoders trained on the first four (Early-4) or six (Early-6) checkpoints of Pythia-410M. We then compute Pt, Rt, and At on the remaining checkpoints, respectively, without retraining, using the full-trajectory CrossCoder (Full-10) as a reference.
Table R2. Pearson correlations between monitoring metrics and downstream accuracy.
Both early-only CrossCoders retain informative generalization-monitoring signals at later checkpoints. Early-4 achieves a correlation of +0.59 for At, while Early-6 achieves +0.77 and +0.79 for Rt and At, respectively. These results provide direct evidence of future-checkpoint monitoring with frozen early-trained CrossCoders, extending the reconstruction-transfer findings in Appendix D.1.
Q3. Computational cost of training the CrossCoder and the overall monitoring pipeline
Our primary goal is to monitor and interpret how generalization develops during pretraining, rather than primarily to reduce the cost of downstream benchmark evaluation. We distinguish one-time CrossCoder training from recurring per-checkpoint pathway analysis. The CrossCoder can be trained on early checkpoints and reused for later checkpoints, amortizing its training cost. Moreover, pathway analysis operates on input-side activations without autoregressive decoding.
For Pythia-410M, training the CrossCoder on the first four checkpoints costs an estimated 1.53 GPU-hours on one RTX A6000. For a fair runtime comparison, we evaluated pathway analysis and autoregressive generation on the same set of prompts. A complete pathway analysis for a later checkpoint averaged 79 s, compared with 149 s and 300 s for generating up to 128 and 256 tokens, respectively. Notably, CrossCoder encoding itself took only 5.25 s. The full per-checkpoint cost is broken down below:
These results demonstrate that our framework provides structural insights into generalization dynamics with an amortizable training cost and lower recurring overhead than the measured generation-based evaluations.