- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper develops a predictive framework for what each class of post training intervention (steering, retrieval and context injection, SFT, DPO, reward only RL) can write into a partially frozen language model. Each knowledge item is placed into one of three measurable layers via two scalars: the probability that temperature one sampling hits the gold answer, and the hit rate once a premise containing the answer is supplied. The three layers correspond respectively to items the base model knows but does not reliably produce, items it can verify but not sample, and items that lie outside the prior. The authors state an inheritance law: answers reachable after any intervention lie in the union of the budget widened base support and the content the protocol explicitly carries. A theorem proves, for on policy reward only RL under an item symmetric policy class, that items with zero sampling support receive zero gradient along the sample path, that the expected gradient scales with support, and that the asymptotic test accuracy ceiling is an inner product of training and test answer marginals that can be computed before training with no fitted parameters. Four channel propositions extend the picture to steering, content injection and gradient writing, each supported by empirical constants on a controlled synthetic bed. Validation combines the synthetic bed on Qwen3-8B and 14B, cross family replication on four additional models, and a new benchmark assembled from three public sources. A no context identity audit is introduced to distinguish genuine writes from distillation leakage, and a decision table operationalises the framework.
Critical strengths
1.A training free and parameter free upper bound on reward only RL. The paper derives the asymptotic test accuracy ceiling of on policy reward only RL as an inner product of two frequency tables known before training. The predicted number is matched empirically within about one percentage point and reproduced by rejection sampling fine tuning, which supports the authors' claim that the ceiling reflects the learning signal rather than any particular optimiser. In the current discussion of the capability boundary of RLVR, this is a rare pre training computable, falsifiable prediction.
2.A useful public benchmark. The paper assembles ReachBench from three existing public sources and, more importantly, labels each item by measurement (sampling support and verifiability under a fixed sampling budget) rather than by its source dataset. The resulting layer assignments are shown to replicate across five model families, which makes the benchmark immediately usable both for diagnosing where a given base model stands and for prospectively evaluating future post training methods against a common coordinate system. The release of per item records alongside the benchmark adds practical value.
Critical weaknesses
1.The paper is dense and heavy on terminology, which makes it difficult to read. Beyond readability, once the formalism is set aside, several non RL claims closely restate established findings from the steering, knowledge editing and retrieval literatures. The genuinely novel component appears to be the RL ceiling together with the measurement protocol; the authors should state more explicitly which claims are new and which are repositionings of existing results. On a presentational note, none of the figures in the submission are vector graphics, and the resulting raster rendering makes several of the quantitative curves difficult to read at normal zoom.
2.The unification is of notation rather than of proof strength. The theorem is rigorous only for the reward only RL case. The remaining three channel propositions are supported by empirical constants on a single base model under a short LoRA training schedule, without closed form reach bounds comparable to the RL ceiling. The framework is therefore predictive in a strong sense for one of its four quarters and observational for the other three, in tension with the paper's framing as a unified predictive theory.
Questions for authors
1.RLVR is now primarily applied to reasoning models, whose generations are long token sequences rather than short factual spans. How is the proposed training accuracy upper bound to be computed in that regime?
2.Is the proposed framework restricted to knowledge question answering with direct factual answers, or is it intended to apply more broadly, for instance to multi step reasoning, program synthesis, or agentic tasks?
We thank the reviewer for the careful reading and for the two questions. We have uploaded a revised version of the paper, and every change is marked in blue.
The paper starts from a definition. The knowledge it measures is knowledge that is written into the model and can be read back, either by sampling an answer or by accepting one that a premise supplies. Knowledge that a model derives by reasoning is outside this object. We exclude it for two reasons: a derived answer can be reached through a chain that does not establish what the model holds, and it becomes impossible to attribute what pretraining and post-training contributed, because the same stored fact can support different derivations in different models. Fixing the object is what makes the comparison possible at all. With the object fixed, the four channels are measured on one coordinate, and the proof differs by what the object admits. For reward-only training the object reduces to the answer marginal, which is known before training, so the ceiling is a closed-form number. For steering and content the effect depends on the model's representation and on how the content is supplied, so we measure the constants and label the statements descriptive. The unification is of the measured object rather than of notation, and the closed-form result is the case where the object is simple enough to admit one.
The revision's experiments test this object rather than the notation, and they test it in the way the reviewer identified as the contribution. The prediction is computed before training from the base model alone: for items a base does not answer, the acceptance measured before training ranks the items a later update moves, so the coordinate is prospective in the same sense as the reward-only ceiling. The six pairs place a released model and its update on one coordinate without modification, and they cover four families: Qwen3-8B-Base and its instruction-tuned model, Qwen3.5-9B-Base and its instruction-tuned model, OLMo-2-7B and OLMo-2-7B-Instruct, Llama-3.1-8B and Llama-3.1-8B-Instruct, GLM-5.2 and GLM-5.3, with a size-matched Qwen3-4B and Qwen3.5-4B control. Two of the pairs are full-parameter updates, the GLM pair is a same-base update with no new pretraining data, and the Qwen3.5 pair is a newer generation of the same family. GLM-5.3 is a near-frontier model, measured through the vendor API because the weights are unavailable, which is the case the black-box probes are designed for; a probe that reads internal states could not measure it. The revision measures nine models on the same 99 items, with 64 samples per item for support and 8 for acceptance, and re-measures 430 items at a larger budget for the reliability check. The validity control establishes what the label measures, since a wrong premise is accepted almost as readily as a true one and a matched two-alternative test is answered at chance. The null test places the one run above the bound within chance, so the bound is measured against a null. The benchmark ships the per-item labels, which is what lets another group place a new model on the same coordinate. The revision also reduces the terminology, states in Section 4 which findings are prior work, and replaces the main figures with vector versions.
The two questions follow from the same definition. A reasoning mode is not enabled by design: it would turn the object from a retrievable fact into a derivable conclusion, and the layer would record derivability rather than retrievability. For a model with long generations, the answer class is the final answer, so the formula applies, but the cost of estimating support rises with the length of the generation. On scope, the framework assumes an item with a checkable answer and a premise that can supply it; multi-step reasoning and agentic tasks introduce intermediate steps that are not answer classes and can reach a right answer through a wrong chain, so we state the scope as factual question answering with checkable answers.