- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes L-State, a 25-dimensional representation combining five current capability scores with changes induced by four target-independent, short LoRA training probes. Ridge-regression readouts use these measurements to predict the effects of candidate training actions across language-model families. The authors evaluate response prediction and action selection on several model families, including held-out GLM and Granite.
Critical strengths
The paper investigates a relevant question: whether short, reusable interventions provide information about future training responses beyond static benchmark scores. The held-out-family evaluations and explicit cost accounting are useful aspects of the experimental design.
Critical weaknesses
-
Limited novelty and insufficient positioning against prior work. Predicting fine-tuning outcomes from probing or early-training signals has been studied extensively, including Predicting Fine-Tuning Performance with Probing (EMNLP 2022) and TuneAhead (ICML 2026). Related work such as LensLLM (ICML 2025) and COSMOS (2025) is not discussed. Given that the proposed readouts are essentially regression models over a small set of handcrafted training-response features, the substantive methodological advance and advantages over existing approaches remain unclear.
-
Limited external validity. The 135 evaluated checkpoint states originate from only five base model families, with each family's states constructed through short synthetic LoRA training trajectories. This controlled setup does not adequately represent diverse real-world pretraining or post-training checkpoints. Moreover, the main response targets are changes in clipped token-level negative log-likelihood rather than downstream task accuracy, limiting the strength of claims about actual capability improvements.
-
Unconvincing practical advantage over competitive alternatives. The headline improvements primarily use a static capability-only predictor as the reference. However, the paper's own cost comparison (Table 29) shows that direct eight-step target-specific screening achieves higher final utility at lower cost than the four-probe approach in three of the four evaluated settings. Thus, the practical benefit of acquiring reusable target-independent probes, relative to competitive cost-aware training-selection strategies, is not convincingly established.
-
Disproportionate theoretical complexity and weak presentation. Much of the theoretical treatment restates standard level-set non-identifiability, linear-system identification, and conditional error propagation. The cross-family transfer guarantee depends on assumptions about unseen-family deviations that are not prospectively established, while exact coordinate sharing is empirically rejected for several actions. The extensive formalism and terminology obscure an otherwise straightforward feature-construction and regression framework, reducing the paper's clarity and substantive theoretical insight.
Questions for authors
What evidence supports the effectiveness of L-State on independently obtained, heterogeneous post-training checkpoints and on actual downstream task performance, particularly under a cost-matched comparison with target-specific screening?