acceptodds
Under review as a conference paper at ICLR 2027

Model-Aware Data Selection from In-and-Out Information Interplay

Abstract

LLMs are effective representations that assimilate vast amounts of knowledge during pretraining, but post-training is necessary for models to reliably access this knowledge and “know what they know.” We observe an interesting rank equilibrium between knowledge stored in the weights and the data stream passing through the model. Across all model layers, we find that the hidden states (data stream) follow a U-shaped pattern, showing substantial compression in early layers and a steep rise during the late-layer decoding phase. In contrast, the weight rank follows an inverted U-shaped pattern, with very low rank in the early and late layers and high rank in the middle. We interpret this as an in-and-out information interplay: intermediate activations do not need to carry content that the weights can supply later, so they primarily preserve what the weights cannot provide. Motivated by this observation, we propose a model-aware data selection method, CAP (Counterfactual Assimilation Profile), which can determine whether a data candidate contains information accessible to the current model by utilizing the divergence gap in early- and late-layer representations between model-generated and reference responses. Across math, code, and science domains, CAP delivers 35.4% greater average improvement over the base model than the strongest baseline under different selection budgets. With only 10% of the data pool, CAP surpasses or matches full-pool training on math and science. We further show that CAP transfers to multimodal data selection and is robust to response horizon and noise.

Primary area:
foundation or frontier models, including LLMs
Keywords:
data selection, supervised fine-tuning, post-training, large language models, model-internal signals, hidden representations, data efficiency

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AKUiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAP, a model-aware data selection method that compares reference responses with the model's own generations and measures representation divergence in shallow and deep layers. It then scores candidate examples using the residual obtained by regressing deep-layer divergence on shallow-layer divergence. Experiments on Marin-8B-Base across mathematics, code, and science tasks demonstrate the effectiveness of CAP, with particularly strong results on mathematical reasoning.

Critical strengths
  1. The paper identifies two interesting layer-wise patterns: activation stable rank follows a U-shaped profile, while the weight profile exhibits an approximately inverted-U shape. Based on these observations, the paper proposes CAP, a model-aware data selection method.
  2. The experimental results demonstrate the generality and robustness of CAP across multiple tasks, with particularly notable improvements in mathematical reasoning.
Critical weaknesses
  1. The comparison with recent methods is insufficient. The authors are encouraged to include more recent data selection methods from the past two years to better demonstrate the performance advantages of CAP.
  2. The organization could be improved. For example, the proposed method could be presented in a separate section to improve clarity and readability.
  3. The connection between the theoretical analysis and the proposed method needs further clarification. Although the paper identifies complementary layer-wise rank patterns, it remains unclear how these observations directly motivate the design of CAP, particularly the shallow/deep layer split and the regression-based scoring mechanism.
Questions for authors

Please see the content in Weaknesses.

1 comment

We thank the reviewer for the positive evaluation and the helpful suggestions on positioning and presentation.

W1. Insufficient comparison with recent methods.

Thank you for this suggestion. We would like to explain how the baselines were chosen, and we add a comparison that highlights what distinguishes CAP from recent methods.

How we chose baselines. Rather than comparing against every method, we cover each major family of selection signals with a well-established representative, all under one controlled protocol (same pool, budget, data order, seed and SFT recipe). Six baselines × three domains × four budgets, plus CAP, amount to 84 full-parameter SFT runs (96 with DSIR, added below). Recent methods are largely newer members of these families:

Signal familyRepresented in our experiments byRecent methods in the same familyTarget data?Backward pass?
HeuristicRandom—nono
External quality ratingQuRatingAlpaGasus, DEITAnono
Geometric coverage / diversityk-centerG-Vendi, Prismatic SynthesisnoG-Vendi: yes
Output likelihood / lossPPL, IFDMIWV, GRAPE, LALPnono
Gradient influence toward a targetLESSGRACELESS: yesyes
Matching a target distributionDSIR (added)RDS+, NAIT, MONAyesno
Counterfactual representation contrastCAP—nono

Within this landscape, CAP combines two properties: it is target-free, and it scores training value directly from the model's internal representations. Unlike gradient-based methods (LESS, G-Vendi, GRACE) it is inference-only, at roughly the FLOP cost of one forward pass per candidate.

Comparison with target-aware selection. LESS and DSIR are given the GSM8K training split as their math target. Every math checkpoint was also evaluated on MATH500, which is no method's target:

Table R1: Math at the 8k budget (Marin-8B-Base, 80k NuminaMath pool)

MethodTarget dataGSM8K (target-matched for LESS / DSIR)MATH500 (no method's target)
Random—69.3722.8
LESSGSM8K-train70.8125.0
DSIRGSM8K-train76.1224.0
CAPnone76.0427.6
Full pool (80k)—72.4828.2
  • Target-aware gains are concentrated on the target. DSIR improves over Random by +6.8 on GSM8K, its target, but only +1.2 on MATH500.
  • CAP's gains transfer. Without any target data, CAP matches DSIR on GSM8K and is the best selector on MATH500 (+4.8 over Random), reaching 98% of full-pool accuracy with 10% of the data.

We will add DSIR and the MATH500 results to the paper, and we will consider further extending the baseline set with recent methods such as MIWV and RDS+ in the revision.

W2. The organization could be improved; the method could be its own section.

The method is presented in Sec. 4 ("CAP: Counterfactual Assimilation Profile for Model-Aware Data Selection"), but we agree it can be made easier to find. We will (i) shorten Sec. 3 to the observation and its two tested predictions, moving the formal capacity model to the appendix; (ii) open Sec. 4 with an algorithm box (inputs → greedy counterfactual → per-layer divergence → depth split → pool-level regression → z-score → top-k); and (iii) give the experimental setup its own section.

W3. How do the rank observations motivate the shallow/deep split and the regression score?

Each design choice corresponds to one element of the analysis and is supported by data:

CAP design choiceMotivationSupporting evidence
Compare against the model's own greedy output y^\hat ythe gap is defined relative to what the model can already produceCAP is nearly uncorrelated with PPL (1–6% top-8k overlap)
Shallow block = first third of layerscompression precedes retrieval, so shallow layers mostly see surface mismatchearly-only selects almost nothing in common with CAP (≤0.1% overlap)
Deep block = last thirdretrieval is deferred to late layers (Thm 1(i))late layers re-expand during pretraining (Pythia table)
Regress deep on shallow, keep the residualremoves the surface-predictable part (Thm 3)selection changes almost completely (0.2–8.4% overlap with late-only); noise is ranked low (Fig. 5)

The resulting selections generalize beyond the benchmark used for evaluation: CAP is the best selector on the untargeted MATH500 (Table R1). The theory does not fix the exact split points (0.33 / 0.66) or the horizon RR. These are empirical defaults, and Table 5 shows CAP is robust to RR across nearly an order of magnitude.

Reviewer_zWxxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Counterfactual Assimilation Profile (CAP), a model-aware data selection method for supervised fine-tuning. The method is motivated by an empirical observation that weight and hidden-state stable ranks exhibit approximately complementary depthwise patterns, which the authors interpret as an interaction between stored knowledge and information carried by activations.

CAP compares the hidden representations of a reference response with those of the model's own generated response. It computes shallow- and deep-layer divergences and uses linear regression to remove the component of deep divergence predictable from shallow divergence. The resulting residual serves as a data selection score intended to identify useful information not yet accessible to the model.

The authors also introduce a memory-augmented capacity model to explain the observed rank profiles and motivate CAP's noise-rejection behavior.

Experiments on mathematical reasoning, code generation, and scientific knowledge report an average improvement over the base model that is 35.4% greater than the strongest baseline's improvement. In particular, CAP achieves 76.04% on GSM8K using 8k selected examples, exceeding the reported full-pool result of 72.48%. Additional evaluations cover multimodal, finance, and medical tasks, response-length sensitivity, noise robustness, and optimization trajectories.

Critical strengths

S1. Substantial empirical improvements and strong data efficiency, especially on mathematical reasoning.

The most convincing contribution is the improvement on GSM8K. CAP achieves 74.30% and 76.04% accuracy using 4k and 8k examples, exceeding the strongest reported baselines by 6.14 and 5.23 percentage points, respectively. Moreover, the 8k result surpasses the reported full-pool performance of 72.48%, suggesting that model-aware selection can be more effective than simply increasing the training data volume.

The reported 35.4% greater average improvement over the base model is also numerically consistent with Table 2. These results provide meaningful evidence for CAP's practical utility, although the gains are substantially more pronounced in mathematics than in code or science.

S2. A simple and potentially useful model-aware selection mechanism.

CAP combines counterfactual response comparison with layerwise representation analysis and residualization. Unlike gradient-based influence methods, it does not require backward passes or a downstream target set, making it attractive for selecting data under limited computational resources.

The method is conceptually interesting because it attempts to distinguish useful model-relative information from general response difficulty. The additional experiments on specialized domains, multimodal data, and response horizons provide supporting evidence that the scoring mechanism is not restricted to a single benchmark, although broader validation remains desirable.

Critical weaknesses

W1. The theoretical connection between the rank profiles, knowledge assimilation, and CAP is not sufficiently justified, and some theoretical claims require stronger assumptions.

The paper interprets stable rank as a proxy for information or knowledge capacity. However, stable rank measures the distribution of singular-value energy and does not directly quantify stored knowledge or information accessibility. Although Appendix A.10 acknowledges this limitation, the interpretation of the complementary rank profiles as evidence of knowledge retrieval remains stronger than the presented measurements directly support.

More importantly, Appendix A.2 states that the exclusion restriction F1 follows from the deferred-retrieval structure of Theorem 1. I do not find this implication established. A scalar capacity model specifying when knowledge is retrieved does not prove that early-layer representation divergence is independent of the assimilation gap. Even before retrieval, the minimum activation capacity g(K)g(K) already depends on retrievable knowledge. Therefore, F1 appears to require an additional assumption or direct empirical validation.

There is also a concrete issue in Corollary 2. Assumption F3 requires the noise terms to be mean-zero and uncorrelated with SS and GG, whereas the proof in Appendix A.8 effectively sets their conditional expectations given S=s,G=0S=s,G=0 to zero. Uncorrelatedness alone does not imply this conditional-mean property. A stronger assumption is required.

Additionally, Theorem 1(iii) appears to require stronger conditions for its strict matching maximum and interior-peaked full weight profile. Assumption 3 permits one matching factor to be identically zero, and Assumption 4 alone does not establish that the interior weight value exceeds the initial boundary value.

These concerns do not invalidate the empirical method, but they weaken the claimed theoretical explanation and noise-rejection guarantee. The mechanism would be more convincing with an explicit separation between modeling assumptions, proven results, and empirical observations.

W2. Experimental inconsistencies, baseline specification, and insufficient uncertainty reporting weaken confidence in the comparative results.

First, the HumanEval results reported at overlapping budgets in Tables 2/7 and Table 12 are inconsistent. For example, CAP achieves 29.88% versus 32.32% at 2k and 35.37% versus 32.93% at 8k. Similar discrepancies occur for Random and PPL. These differences require explanation, particularly because Appendix H.1 emphasizes a shared training configuration and fixed random seed.

Second, the baseline implementations need clarification. Appendix H.3 describes IFD using a ratio of response perplexities, whereas the original method uses a ratio of average cross-entropy losses. These quantities do not generally induce the same example ranking. The LESS configuration is described without gradient warmup and appears to differ from the original procedure. Furthermore, the documented LESS cost configuration uses all 164 HumanEval evaluation problems and their canonical solutions as its target set. It is unclear whether the same configuration produces the accuracy results or only the cost measurements.

Third, the reported "standard deviations" in Appendix C numerically match binomial accuracy standard errors calculated from the benchmark sizes. For example, the reported GSM8K uncertainty of approximately 1.18 percentage points at 76.04% accuracy matches the test-set standard error for 1,319 examples. The number of independent training runs is not clearly specified. Therefore, the reported uncertainty does not establish robustness across selection or training seeds.

These issues are especially important because CAP's advantage outside mathematics is relatively small. For example, at the 8k budget, CAP ties LESS on HumanEval and exceeds the strongest reported science baseline by only 0.54 percentage points. Clearer experimental protocols and uncertainty estimates are necessary to assess the generality of these gains.

Questions for authors

Q1. Experimental consistency and statistical reliability.

Could the authors explain the discrepancies in the overlapping HumanEval results between Tables 2/7 and Table 12, especially for CAP, Random, and PPL at 2k and 8k? Were different seeds, candidate pools, or evaluation configurations used?

Additionally, what exactly do the reported error bars represent, and how many independent training runs were performed? Repeated-run evidence for the main comparisons would help establish the reliability of the reported improvements.

Q2. Baseline implementation and comparison fairness.

Is the IFD perplexity-ratio definition in Appendix H.3 a documentation error, or was this ratio used in the implementation? Could the authors clarify how their LESS implementation relates to the original method, particularly regarding gradient warmup?

For LESS, does the HumanEval target-set configuration described in Appendix H.3 apply to the accuracy experiments, the cost comparison, or both? If it applies to accuracy, could the authors provide results using a target set disjoint from the evaluation benchmark?

Q3. Necessity of the CAP residualization mechanism.

Could the authors provide a targeted ablation comparing the full CAP score against:

  1. Late-layer divergence alone;
  2. Early-layer divergence alone;
  3. Unresidualized late-minus-early divergence?

This would help determine whether regression-based residualization contributes substantially beyond the counterfactual representation difference itself. A controlled experiment separating semantic differences from formatting or stylistic differences would further support the proposed interpretation.

Q4. Theoretical assumptions and proof validity.

Could the authors clarify why the deferred-retrieval theorem implies the exclusion restriction F1 for measured hidden-state divergence?

Corollary 2 appears to require conditional mean-zero errors, rather than only the uncorrelatedness specified in F3. Could the authors state and justify the additional condition?

Could the authors also revisit Theorem 1(iii), particularly whether Assumptions 3 and 4 are sufficient to guarantee the claimed strict matching maximum and interior peak of the full weight profile?

Q5. Evidence for the proposed rank-based interpretation and noise rejection.

The manuscript states that the complementary rank profiles emerge during pretraining, but the provided figures primarily show depthwise measurements from trained checkpoints. Could the authors provide the corresponding initialization and intermediate-checkpoint evidence, or clarify the scope of this claim?

For the noise experiment in Figure 5, does the reported 99.7% refer to the mean noise score or individual noise examples? Reporting the actual fraction of contaminated examples retained under different selection budgets would more directly demonstrate practical noise rejection.

Rebuttal consideration: My current recommendation is weak acceptance. I would consider increasing it if the authors convincingly reconcile the experimental discrepancies, clarify the baseline implementations, establish the contribution of residualization, and correct the theoretical assumptions and claims. These issues are potentially addressable and do not negate the promising empirical results, particularly on mathematical reasoning.

2 comments
AuthorsOpenReview ↗

Part 1/2: Theory (W1/Q4); HumanEval tables and error bars (W2/Q1)

We thank the reviewer for the careful and constructive review. Below we clarify the theoretical scope and make two technical conditions explicit, explain the HumanEval tables and the error bars, describe both baseline implementations, and give evidence for residualization and for the generality of the gains.

W1/Q4. Stable rank vs. knowledge; F1; Corollary 2; Theorem 1(iii).

  • Stable rank. We fully share this view, and App. A.10 states it ("stable rank is a surrogate for capacity, not capacity"). The knowledge reading rests on the predicted timing rather than on rank values: the profile is monotone at initialization and the trough and re-expansion appear during pretraining (table in our response to 2pCZ-W2). Since the main-text wording can read as stronger than intended, we will phrase it as "consistent with".
  • F1. In App. A.2, "inherited" refers to the capacity model, where the shallow block precedes the first retrieval layer, so nothing has been retrieved there. Moving from that to measured hidden-state divergence is a modeling step, as the main text indicates ("motivates an exclusion restriction"). The reviewer's remark about g(K)g(K) illustrates why it should be stated separately, and we will list F1 as an explicit assumption.
  • Corollary 2. Thank you for this careful reading. The step that takes expectations conditional on S=s,G=0S=s, G=0 uses the conditional form of F3, E[εe∣S,G]=E[εl∣S,G]=0\mathbb E[\varepsilon_e\mid S,G]=\mathbb E[\varepsilon_l\mid S,G]=0, which holds, e.g., when the measurement noise is independent of (S,G)(S,G). We will state this form explicitly for the corollary (F3′). The rest of the proof is unchanged, and Thm 3 itself only needs the uncorrelated form.
  • Theorem 1(iii). The boundary comparison with w1w_1 is derived in the proof (App. A.4) but was not carried into the statement. We will add it as Assumption 4′ and require μ,ν>0\mu,\nu>0 on (0,∞)(0,\infty) for the strict matching maximum (details in 2pCZ-W3).

Following the reviewer's suggestion, App. A will be organized into modeling assumptions, proven results, and empirical observations.

W2/Q1. Why do the overlapping HumanEval results (Tables 2/7 vs. Table 12) differ? What do the error bars represent, and how many training runs?

HumanEval. The budget-scaling study (Table 12) is a separate, later run on a rebuilt candidate pool with an additional deduplication pass. It does not extend the main-table checkpoints. A different pool gives a different subset at every budget, so its 2k / 8k numbers are not re-measurements of the main-table models; the recipe and seed are the same. App. H.1's "shared configuration" refers to comparisons within each table. Because Table 12 sits in a separate appendix section without naming its pool, the relationship was easy to miss, and we will name both pools explicitly. Within Table 12, CAP is at or above Random and PPL at every reported budget, reaching 45.73 at 60k (Random 42.07, PPL 43.90).

Error bars. To clarify: the "±" values are binomial standard errors over benchmark questions, and the main comparison uses one selection and training seed. We will label them as evaluation standard errors. As a robustness check, every math checkpoint was also re-evaluated with sampled decoding (T = 0.7) under four independent evaluation seeds:

Table R2: GSM8K under sampled decoding, mean ± std over 4 evaluation seeds (same checkpoints as Table 2)

Method1k2k4k8k
Random57.45 ± 0.8059.42 ± 0.9763.46 ± 0.8064.65 ± 1.00
PPL56.43 ± 1.4059.08 ± 0.4361.32 ± 1.0163.00 ± 0.68
IFD55.82 ± 0.6657.87 ± 1.1758.72 ± 1.7759.95 ± 2.00
QuRating56.25 ± 0.5358.17 ± 0.7960.18 ± 0.5262.41 ± 1.28
LESS57.24 ± 0.8760.69 ± 1.7663.89 ± 1.4865.33 ± 0.76
k-center57.22 ± 0.6558.23 ± 0.5461.58 ± 1.2363.78 ± 0.48
CAP58.81 ± 0.2262.68 ± 1.5469.14 ± 0.1969.43 ± 1.48

CAP's math advantage holds under a different decoding scheme and across evaluation seeds. At 4k it leads the best baseline by 5.3 points against standard deviations of 0.2–1.5.

AuthorsOpenReview ↗

Part 2/2: Gains beyond math (W2/Q1, continued); baselines (W2/Q2); residualization (Q3); pretraining and noise (Q5)

W2/Q1 (continued)

On the size of the gains outside math. We would like to offer a complementary reading. The comparison in Table 2 is not on equal footing for code and science: LESS is given benchmark-matched target data (MBPP-train for code, MMLU-validation for science), whereas CAP uses none. CAP ties LESS on HumanEval and leads it on MMLU-Pro without that information. More importantly, Table R1 (in our response to 2pCZ) shows that target-aware gains are concentrated on the target benchmark (DSIR: +6.8 on GSM8K but +1.2 on MATH500), while CAP's gains transfer: it is the best selector on the untargeted MATH500. We will make this information asymmetry explicit in Sec. 4.3. We will also note that the 35.4% average is largest in math, where reasoning-heavy data makes the gap between assimilated and unassimilated content most pronounced.

W2/Q2. Is the IFD perplexity ratio a documentation error? How does LESS relate to the original (warmup)? Which target set does LESS use for accuracy?

  • IFD. It is not a documentation error: App. H.3 accurately describes what we ran, the perplexity ratio exp⁡(Lcond−Luncond)\exp(L_{\text{cond}}-L_{\text{uncond}}). Li et al. (2024) define IFD with the loss ratio Lcond/LuncondL_{\text{cond}}/L_{\text{uncond}}. Both measure how much the instruction helps predict the response, from the same forward passes, but they can order examples differently. We will name the row "IFD (perplexity-ratio variant)" so the relationship is clear.
  • LESS. Our implementation follows LESS's gradient-matching principle in a lighter form: no LoRA warmup, and SGD (rather than Adam) gradient features projected to 8,192 dimensions. This keeps its selection cost comparable to the other scorers. We will describe these choices explicitly in App. H.3.
  • Target sets. For the accuracy experiments, LESS (and DSIR) use GSM8K-train for math, MBPP-train for code, and MMLU-validation for science, none of which overlaps the evaluation sets. The HumanEval mention in App. H.3 refers to an optional configuration of the cost benchmark, and we will separate the two descriptions. The cost is dominated by per-candidate gradients over the 80k pool, so the target-set size (164 vs. 2,000 examples) changes the reported GPU-hours only marginally. As Table R1 shows, benchmark-matched targets tend to favor the target-aware baseline on its own benchmark, so this setup is, if anything, conservative for CAP.

Q3. Does residualization contribute beyond the counterfactual difference itself?

Thank you for this suggestion. Using the per-example scores on the full 80k pools:

Table R3: Overlap of each score's top-8k with CAP's top-8k on the 80k pools (fitted slope β=1.19/1.28/1.34\beta=1.19/1.28/1.34)

ScoreMathCodeScience
Late-layer divergence only0.2%6.0%8.4%
Early-layer divergence only0.0%0.1%0.1%
Late − early, no regression (β=1\beta=1)72.3%37.1%16.8%
PPL1.2%3.7%6.2%
  • Residualization changes which data are selected. Late-only and CAP select almost disjoint subsets, and fixing β=1\beta=1 instead of fitting it still changes 28–83% of the selection. The main-table gains therefore cannot come from the late-layer signal alone.
  • CAP is not a disguised loss score. Its top-8k overlaps PPL's by only 1–6%. Together with Fig. 5, this shows residualization produces a signal that difficulty-based scores do not.

We will add this analysis to Sec. 5, and note the training-level variant comparison and the controlled semantic-vs-format contrast as natural follow-ups.

Q5. Do the profiles emerge during pretraining? What does 99.7% mean? What fraction of contaminants is retained?

  • Pretraining. Pythia-6.9B at 12 checkpoints, step 0 to 143k (table in 2pCZ-W2). At initialization the activation profile is monotone and the weight profile flat; the trough, late re-expansion and interior weight peak appear between 1k and 13k steps and persist. We will add this figure to the main text. The temporal claim refers to the Pythia trajectory, the suite with public intermediate checkpoints, while the cross-model survey uses final checkpoints.
  • 99.7%. It is the fraction of clean examples whose CAP exceeds the mean CAP of the noise examples, i.e., a summary of how the two distributions are separated. We will state this definition in the caption and phrase the claim accordingly: CAP shifts injected noise to the low end of the ranking, whereas perplexity places it above the clean distribution.
  • Retention under selection. Our budgets keep the top 1.25–10% of the pool (1k–8k of 80k), so every selection threshold lies at or above the 90th percentile of clean scores. The noise mean lies below the 0.3rd percentile, so the noise distribution is centred far below every threshold we use. We will state this explicitly alongside Fig. 5.
Reviewer_2pCZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Data selection for training, especially finetuning during post-training, ideally cannot be model-agnostic. When measuring stable rank (squared Frobenius norm divided by the squared spectral norm) across twelve language models, the authors observe that hidden state stable rank decreases during intermediate layers and increases near the output layer, while the weight's stable rank is usually higher during intermediate layers. The paper interprets this as activation compression followed by retrieval of information stored in the weights. A "capacity-minimization" model is used to explain this phenemonon and make use of this fact to select better data as described below.

The paper's method Counterfactual Assimilation Profile (CAP) compares a reference response with the model’s own greedy response to the same prompt. CAP measures the cosine divergence between hidden states at corresponding response positions (using the last 256 prompt tokens and the first 48 response tokens) and then averages the divergences over the first and final thirds layers, aligning their method with the empirical observation. Then across the candidate pool, the paper fits a regression of late divergence on early divergence. Finally, the algorithm selects data samples with the largest residuals, CAP=Dlate−(α^+β^Dearly)\mathrm{CAP}=D_{\mathrm{late}}-(\hat{\alpha}+\hat{\beta}D_{\mathrm{early}}). The intuition for CAP is that early divergence captures simple wording and formatting differences, while abnormally large late divergence captures structured content that the model does not know or cannot use. Scoring is cheap in terms of prefilll but requries some generation cost as CAP needs one greedy generation and two forward passes per sample.

The main experiments fine-tune Marin-8B-Base for one epoch on 1,000 to 8,000 examples selected from separate 80,000-example math, code, and science pools. At the 8,000-example budget, CAP data selection achieves 76.04% on GSM8K, compared with 70.81% for LESS and 72.48% for full-pool training and roughly equal performance as the strongest baseline on HumanEval and MMLU-Pro (Table 2). On a pretrained checkpoint of Molmo2-O-7B, selecting 8,000 of 20,000 document-understanding examples gives 46.15% accuracy, better than perplexity-based selection. CAP also achieves slightly better performance on FinanceBench and medican diagnosis compared to random and perplexity-based selection.

Critical strengths

CAP scores data using the model being fine-tuned and the candidate pool, so all it requires hidden states from reference and generated responses and a simple regression. No separate per-example gradient computation is needed. The same scoring method produces a gain on an image-conditioned model Molmo2-O-7B as well. It's neat that they used a pretrained checkpoint to avoid contaimination of the target task. CAP retains its default 48-token horizon and selects document-understanding examples without an image-specific scoring component. Its 46.15% accuracy at the 8,000-example budget exceeds perplexity selection at 41.76% and random selection at 40.66%.

Critical weaknesses
  1. Many related baselines and related work discussion are missing, especially understates existing model-aware selection. In particular, there is zero citations on works from post-2024 other than two citations for an LLM checkpoint and another citation for the medical benchmark.

The introduction states that “most existing selection signals largely ignore the underlying model,” but the paper does not adequately discuss existing model-dependent selection. The paper cites PPL and LESS, yet its introduction and related work discussion seem lacking (the only entries from those years are Marin, MedCaseReasoning, and Molmo2). Normally, I would be hesitant to mention related work as a critical weakness, but this not only affects the selection of baseline comparsions as well as the positioning of this work that may mislead readers.

References are listed below. MIWV ranks examples through target-model loss changes with and without retrieved demonstrations, while RDS+ and NAIT select through the model’s internal representations, using reference examples to define the target task. The reference-set requirement is a meaningful distinction from CAP, but CAP already includes the target-dependent LESS baseline. Thus, including one of these (just MIWV would suffice since that's the one I'm most familiar with) would greatly show the benefits of CAP.

Not directly comparable, but the following works warrant some discussion for the readers. If the authors can find a setting to compare these methods, that would be even better; this can be done by using CAP to the baseline paper's settings. G-Vendi measures diversity in model-induced gradient space and includes gradient-cluster-based subset sampling (this should qualify section 2’s statement that gradient methods require a target evaluation set). GRAPE selects supervision using target-model likelihood, while Local Naturalness/LALP scores reasoning steps under local student contexts to address failures of whole-response likelihood. Their per-prompt response-selection setting differs from CAP’s subset selection, but applying CAP to that setting would be nonetheless valuable.

  1. The theory does not establish anything meaningful.

The capacity model first minimizes total activation capacity and then minimizes weight capacity as a tiebreaker. The result describes an optimum of the stipulated accounting model. The theory here feels like it exists to appease the reader without actually providing much value or intuition. Either the theory is based on some post-hoc justification or the paper has failed to convey the intuitions and the takeaways with indiscriminate presentation. Overall, I believe theory should have one of two goals: to provide strong explanatory power (possibly predictive) or to convey theoretical analysis that simple intuition cannot carry. The math shown in this paper achieves neither.

First, Theorem 1(iii)'s claim that the full weight-capacity profile peaks in the interior does not follow from Assumptions 1–4.** Consider the following setting:

  • L = 4, H = 1.5, and R = K = c = B_t = B_r = σ = τ = 1.
  • The matching coefficient is σ_m = 3.
  • The addressing function is φ(u) = u/2, and the matching functions are μ(u) = ν(u) = u.

Assumptions 1–3 hold: the addressing function is concave with derivative below the content rate, there is sufficient depth, and both matching functions are increasing. Assumption 4 also holds because the matching cost at the activation trough is M = 3, exceeding the required threshold σ + τcB_r = 2.

The optimal retrieval schedule, activation profile, and weight profile are, respectively:

  • k = (0, 0, 0, 1)
  • r = (1.5, 0.5, 0.5, 0.5, 1)
  • w = (4, 3, 3, 1.5)

In particular, the first layer incurs a transport cost of 1 and a matching cost of 3, giving w₁ = 4. The second and third layers each incur a matching cost of 3, while the final layer incurs an injection cost of 1 and a transport cost of 0.5. In turn, the unique maximum is at the first layer, not in the interior, despite all four assumptions being satisfied. The proof overlooks this possibility: Assumption 4 bounds the trough's matching cost against the final-layer cost but does not ensure that it exceeds the matching plus transport cost at the first layer. Consequently, the claimed sufficient condition for an interior-peaked weight profile is incorrect.

Most importantly, the theory provides a possible interpretation of CAP rather than a substantive theoretical justification for why its ranking should identify valuable training examples. Theorem 3 does not establish the central premise that makes CAP a meaningful data-selection criterion. Assumption 5 posits that early-layer divergence contains surface mismatch but no assimilation-gap signal, whereas late-layer divergence contains both surface mismatch and the assimilation gap. Yet Theorem 1's abstract deferred-retrieval schedule does not imply this exclusion restriction for actual transformer hidden states: the absence of retrieved information in an early layer does not imply that the layer cannot encode semantic differences between a reference response and the model's generation. Given this assumed factor structure, Theorem 3 establishes an ordinary least-squares residual identity. It does not independently show that the residual measures useful unassimilated knowledge. In fact, the identified quantity is only the component of the assimilation gap orthogonal to surface mismatch; any potentially useful gap information correlated with surface mismatch is removed. The analysis also does not establish that examples with larger residuals produce greater downstream fine-tuning improvements.

References:

  1. NAIT: Chen, Xin, et al. “Neuron-Aware Data Selection in Instruction Tuning for Large Language Models.” International Conference on Learning Representations (ICLR), 2026.
  2. RDS+: Ivison, Hamish, et al. “Large-Scale Data Selection for Instruction Tuning.” arXiv, 2025, arXiv:2503.01807.
  3. MIWV: Jiang, Tingyu, et al. “Importance-Aware Data Selection for Efficient LLM Instruction Tuning.” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 37, 2026, pp. 31328–31336. https://doi.org/10.1609/aaai.v40i37.40396.
  4. G-Vendi and Prismatic Synthesis: Jung, Jaehun, et al. “Prismatic Synthesis: Gradient-Based Data Diversification Boosts Generalization in LLM Reasoning.” Advances in Neural Information Processing Systems, vol. 38, 2025.
  5. LALP: Just, Hoang Anh, et al. “The Signal Is in the Steps: Local Scoring for Reasoning Data Selection.” Proceedings of the 43rd International Conference on Machine Learning, edited by Tong Zhang et al., vol. 306, Proceedings of Machine Learning Research, 2026, pp. 55125–55155.
  6. MONA: Ma, Da, et al. “Task-Specific Data Selection for Instruction Tuning via Monosemantic Neuronal Activations.” Advances in Neural Information Processing Systems, vol. 38, 2025.
  7. GRACE: Panigrahi, Abhishek, et al. “In Good GRACEs: Principled Teacher Selection for Knowledge Distillation.” International Conference on Learning Representations (ICLR), 2026.
  8. GRAPE: Zhang, Dylan, et al. “The Best Instruction-Tuning Data Are Those That Fit.” Advances in Neural Information Processing Systems, vol. 38, 2025.
Questions for authors

My questions are embedded within the weaknesses. Addressing each point there would greatly help my understanding and help me re-evaluate the work. Thank you.

3 comments
AuthorsOpenReview ↗

Part 1/3: Baseline choices and positioning (W1)

We sincerely thank the reviewer for their exceptionally thorough and constructive review. Below we (1) clarify our baseline choices and CAP's positioning relative to recent model-aware methods; (2) explain what the theory is meant to do and which of its predictions we could check against data; and (3) make the boundary condition in Theorem 1(iii) explicit.

W1. Related work and baselines underrepresent existing model-aware selection; the introduction may mislead readers.

Thank you for the reading list. It is very helpful, and we will add all of these works. We would also like to clarify two sentences whose intended scope may not have come through.

  • The introductory sentence ("most existing selection signals largely ignore the underlying model") is followed directly by its qualification: loss-based scores do consider the model, but "reduce its response to a scalar that cannot distinguish knowledge I have not yet assimilated from text that is intrinsically difficult". The intended contrast is therefore about which part of the model's state a signal reads, not whether it reads the model at all. Taken on its own, the sentence can indeed be read more broadly than we meant. We will rephrase it as "most existing signals either score data independently of the model or summarize the model's response in a single scalar."
  • The Sec. 2 remark that "gradient methods require a target eval set" refers to the gradient-influence methods listed in the same sentence (LESS-style matching to a target). G-Vendi uses gradients for a different purpose, set-level diversity without a target. We will make that scope explicit and discuss G-Vendi and Prismatic Synthesis.

How we chose baselines. Our goal was not to compare against every method, but to cover each major family of selection signals with a well-established representative, under one controlled protocol (same pool, budget, data order, seed and SFT recipe). The six baselines × three domains × four budgets, plus CAP, already amount to 84 full-parameter SFT runs (96 with DSIR). Most of the recent methods the reviewer lists are newer members of families we already cover:

Signal familyRepresented in our experiments byRecent methods in the same familyTarget data?Backward pass?
HeuristicRandom—nono
External quality rating (model-independent)QuRatingAlpaGasus, DEITAnono
Geometric coverage / diversityk-centerG-Vendi, Prismatic Synthesis (diversity in gradient space)noG-Vendi: yes
Output likelihood / loss of the target modelPPL, IFDMIWV, GRAPE, LALPnono
Gradient influence toward a targetLESSGRACE (student gradients)LESS: yesyes
Matching a target distributionDSIR (added in this rebuttal)RDS+, NAIT, MONAyesno
Counterfactual representation contrastCAP—nono

Two points stand out:

  • CAP occupies a distinct point in this space: it is target-free, yet scores each example's training value directly from the target model's internal representations. RDS+, NAIT and MONA read internal representations to match a given target set, and embedding-based selectors use representations for coverage or diversity. CAP uses them to measure what the model has not yet assimilated.
  • Unlike the gradient family, target-aware (LESS) or target-free (G-Vendi, GRACE), CAP is inference-only. It needs no backward pass, no warmup training, and no per-example gradient storage.

On MIWV, GRAPE and LALP. We thank the reviewer for the detailed reading of these works. They fall into the output-likelihood family, which PPL and IFD cover in our experiments. MIWV compares the response loss with vs. without a retrieved demonstration, structurally the same contrast IFD makes with vs. without the instruction. GRAPE and LALP score responses or reasoning steps by their likelihood under the target model. The key difference from CAP is the signal: a scalar likelihood versus a depth-resolved contrast between hidden states on the reference and on the model's own continuation. This difference shows up in what gets selected, since CAP's top-8k overlaps PPL's by only 1–6% across the three pools. We agree that adding these methods directly would strengthen the comparison, and we will consider expanding the baseline set (e.g., to eight baselines including MIWV and a target-representation method such as RDS+) in the revision.

AuthorsOpenReview ↗

Part 2/3: Selection cost and target-free selection (W1, continued)

W1 (continued)

Per-candidate compute. In units of one token-forward (F≈2NF\approx2N FLOPs, with a backward pass costing about 2F2F per token), on our math pool (≈50 prompt + 290 response tokens on average):

MethodWhat is computed per candidate≈ Cost
PPL1 forward over the full example~340F
IFD2 forwards (with / without instruction)~630F
MIWV2 forwards (with a demo, ≈2× length; without) + retrieval~1,000F
LESS (our lightweight variant)forward + backward over the full example, gradient projection~1,000F
LESS (original)the above at each warmup checkpoint (4 in the original) + LoRA warmup training~4,000F + training
CAP48-token greedy continuation + 2 forwards over prompt + 48 tokens; no backward~300F

CAP's cost is also independent of response length, since it reads only the first R=48R=48 response tokens, whereas loss- and gradient-based scores grow with the full response. Because greedy decoding is sequential, wall-clock does not scale exactly with FLOPs. Measured end-to-end (Sec. 5.1), CAP takes 2.81 GPU-hours per domain, versus 3.18 for our already lightweight LESS and 2.26 for IFD.

Why target-free selection matters. As the reviewer notes, the reference-set requirement of RDS+, NAIT and MONA is a meaningful distinction. We compare CAP with two target-aware selectors, LESS and DSIR, under the identical pool / budget / SFT protocol. Both are given the GSM8K training split as their math target. Every math checkpoint was also evaluated on MATH500, which is no method's target. That separates gains from matching the target from gains that transfer:

Table R1: Math at the 8k budget, on the target-matched benchmark and on an untargeted one (Marin-8B-Base, 80k NuminaMath pool)

MethodTarget dataGSM8K (target-matched for LESS / DSIR)MATH500 (no method's target)
Random—69.3722.8
PPL—68.9223.8
IFD—66.3426.2
QuRating—66.1124.8
k-center—70.5820.8
LESSGSM8K-train70.8125.0
DSIRGSM8K-train76.1224.0
CAPnone76.0427.6
Full pool (80k)—72.4828.2
  • Target-aware gains are concentrated on the target. DSIR improves over Random by +6.8 on GSM8K, its target, but only +1.2 on MATH500. LESS shows the same pattern (+1.4 / +2.2).
  • CAP's gains transfer. Without any target data, CAP matches DSIR on GSM8K and is the best of all selectors on MATH500 (+4.8 over Random, +3.6 over DSIR). With 10% of the pool it reaches 98% of full-pool MATH500 accuracy.

In science, where LESS and DSIR target MMLU-validation, CAP also leads on MMLU-Pro at 8k (39.05 vs. 38.39 / 38.25). This is the practical case for a target-free, model-aware signal: it improves the model's capability rather than its fit to one benchmark. We will add DSIR and the MATH500 column to the paper.

We will discuss MIWV, GRAPE and LALP in detail in the related work, and note applying CAP's depth-split contrast at the response or step level (as in GRAPE / LALP) as a promising extension.

AuthorsOpenReview ↗

Part 3/3: Role of the theory (W2), Theorem 1(iii) (W3), F1 and Theorem 3 (W4)

W2. The theory does not establish anything meaningful.

We appreciate the reviewer's standard for theory, and we share it: a theory should either explain or predict. The intended role of our capacity model may not have been clear, because the main text leads with formal statements and leaves the scope discussion to App. A.10. That appendix states that the model is an accounting model that "predicts a sign and a mechanism, not effect sizes", and that "the empirical case for CAP rests on Section 4, not on the theory alone". Its value lies in predictions that are not built into the method and could have failed. Two of them can be checked against data:

Prediction 1 (Cor. 1). With no stored knowledge (K=0K=0), activation capacity is non-increasing in depth: no interior trough and no late re-expansion. These should appear only as knowledge accumulates. Pythia-6.9B (32 layers), hidden-state and weight stable rank across pretraining:

StepLayer of activation minimumLate re-expansion (last − min)Weight rank, interior / edge
0 – 25630–31 (output)0.000 – 0.0011.00
1k300.0030.78
4k100.0220.92
13k70.0281.34
44k70.0401.58
143k60.0371.52

At initialization the profile is monotone, exactly the K=0K=0 case. The interior trough, the late re-expansion and the interior weight peak all emerge together during pretraining.

Prediction 2 (Cor. 2). Structureless, surface-difficult noise should rank high under loss-based scores but low under the residualized score. Fig. 5 confirms this on Marin-8B: perplexity ranks the injected junk above the clean distribution, whereas CAP ranks it low.

The theory also motivates the design step that matters most. Residualizing deep on shallow divergence (Thm 3) makes CAP a different selector from "read the late layers": on the 80k pools, late-only and CAP share only 0.2% / 6.0% / 8.4% of their top-8k (math / code / science).

To make this role visible, we will restructure Sec. 3: the main text will lead with the observation, these two tested predictions, and the assumptions behind CAP, each clearly labelled. The formal capacity model moves to the appendix.

W3. Theorem 1(iii)'s interior peak of the full weight profile does not follow from Assumptions 1–4.

We thank the reviewer for the careful construction. It identifies a boundary condition that appears in our proof but was left out of the theorem statement. The proof (App. A.4) compares the trough value MM with the first-layer cost w1w_1 explicitly, and in the bilinear case gives M/w1=ℓd/(1+τH/(σmK))M/w_1=\ell_d/(1+\tau H/(\sigma_mK)), i.e., an interior peak requires the compression phase to span more than 1+τH/(σmK)1+\tau H/(\sigma_m K) layers. The remark after the proof also notes that we state (iii) primarily in terms of the matching component for this reason. The reviewer's instance has a single compression layer (ℓd=1<1.5\ell_d=1<1.5), which is precisely the regime this condition excludes. We will state the theorem as:

  • (iii-a) The matching component mℓm_\ell peaks exactly under the trough, at a value strictly increasing in KK, for μ,ν>0\mu,\nu>0 on (0,∞)(0,\infty). This is the part the CAP design uses; App. A.10 already calls the weight profile "the weakest link on the phenomenon side".
  • (iii-b) The full weight profile peaks in the interior when, in addition, M>w1M>w_1 (Assumption 4′, the condition derived in the proof). Assumption 4 already gives M>wLM>w_L.

W4. Thm 1 does not imply the exclusion restriction F1; Thm 3 is an OLS identity that removes surface-correlated gap and does not show larger residuals help fine-tuning.

Several of these points match scope statements already in Sec. 3.2 and App. A.10, which we will make more prominent.

  • F1. The main text says the deferred-retrieval structure "motivates an exclusion restriction". Within the capacity model nothing is retrieved before the first retrieval layer, and that is the sense of "inherited" in App. A.2. Carrying this over to measured hidden-state divergence is a modeling step, and we will label F1 as an assumption next to F2–F3 to avoid any ambiguity. Fig. 5 is an observable implication consistent with it.
  • Thm 3 identifies the surface-orthogonal gap G⊥G_\perp, by design. With two measurements, gap that co-varies with surface mismatch cannot be told apart from surface mismatch itself, so keeping it would also keep surface-difficult junk, the failure mode of PPL in Fig. 5.
  • Downstream utility. As App. A.10 states, Thm 3 "is an identification result conditional on [the factor model]", and the downstream case rests on experiments. Beyond Table 2, Table R1 above shows that the selected data help beyond the benchmark they were measured on: CAP is the best selector on the untargeted MATH500.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.