- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces Feature Density (FD), defined as the fraction of input units whose individual deletion increases a fixed pretrained model’s loss on the target sequence. The authors evaluate the metric using synthetic data with known feature locations and twelve real-world datasets, primarily with Qwen3-0.6B, supplemented by Qwen3.5-0.8B. They also investigate its relationship with fine-tuning losses under Adam, AdamW, SGD, and Muon. The main claim is that higher Feature Density reflects a greater proportion of task-relevant content and is associated with better fine-tuning performance.
Critical strengths
The proposed metric could offer a useful way to characterize how a model uses training inputs beyond dataset size and sequence length.
Critical weaknesses
My two critical concerns are as follows.
-
The central claim about fine-tuning performance is not supported by the real-world evaluation. Although the abstract claims lower test loss in both synthetic and real-world settings, the real-world experiments report only training losses. Furthermore, comparisons between different datasets within broad task categories do not control for difficulty, target format, sequence length, or the pretrained model’s initial performance. Several loss differences are already present at the start of training, and the association is not consistent: Figure 8 shows higher FD but higher training loss for SWE-Bench than APPS. Consequently, the experiments do not establish that FD predicts generalization or gains from fine-tuning. This directly undermines the paper’s main empirical contribution.
-
Feature Density is insufficiently validated as a measure of task-relevant content. In Figure 3, FD is approximately 0.544 when the true feature ratio is 0.1. Even assuming perfect recovery of genuine features, this implies that approximately 49% of noise tokens are classified as features. The real-world validation consists of three illustrative examples without aggregate annotation-agreement statistics. Moreover, any positive deletion effect is treated as relevance, although it may reflect model-specific sensitivity to perturbations. These gaps are consequential because the interpretation of FD as task-relevant density underpins the paper’s subsequent analyses; a correlation with loss cannot, by itself, validate that interpretation.
Questions for authors
- Can the authors provide real-world held-out losses and task-specific performance metrics, including improvements over the pretrained checkpoint?
- Does FD predict held-out fine-tuning gains after controlling for initial loss, input and target lengths, and dataset difficulty? Can this be tested using matched subsets of the same dataset?
- What are the feature-identification precision, recall, and false-positive rates across synthetic feature ratios? Why is FD approximately 0.544 when only 10% of tokens are genuine features?
- How sensitive are FD values and dataset rankings to the deletion threshold, unit granularity, scoring model, and alternative perturbation methods?
- How is Assumption 2 satisfied by the fixed-position synthetic construction, and what does Proposition 1 establish beyond concentration of dataset-average differences?
5 comments
Response (Part 2)
Response to Q2: Does FD predict held-out fine-tuning gains after controlling for initial loss, input and target lengths, and dataset difficulty? Can this be tested using matched subsets of the same dataset?
Thank you for the suggestion. We understand that you would like to establish whether selecting training data by FD still leads to greater improvements in test performance when the model’s starting point and evaluation conditions are the same. To address this, we added data-selection experiments within individual datasets to compare the gains from fine-tuning on subsets with different FD values.
Specifically, for each of six datasets, GSM8K, SST-2, OPUS Books, LongBench, SWE-Bench, and NIAH, we sampled equal numbers of examples from the same training pool. High-FD selects examples with FD in [0.6, 0.9], Low-FD selects examples with FD in [0.1, 0.4], and Random samples from the entire training pool. All three groups start from the same Qwen3-0.6B checkpoint, use Adam for three training epochs, and are evaluated on the same held-out set. The experimental settings are described under “Fine-Tuning Results with FD-Based Data Selection” in Appendix C.2, and the training loss, test loss, and token accuracy results are shown in Figure 15.
This design directly addresses your concern about initial loss. Before fine-tuning, all three groups evaluate the same pretrained model on the same test set, so they share the same initial test loss and accuracy. Epoch 0 in Figure 15 marks this common starting point. Comparing test performance in subsequent epochs directly shows how much each data-selection strategy improves over the pretrained model. Comparing subsets within the same dataset also fixes the data source and test distribution, reducing the influence of differences in task difficulty, target format, and other factors that arise when comparing different datasets.
For example, in the OPUS Books test loss panel in Figure 15 (second row, third column), all three groups start from a test loss of 2.5142. After three training epochs, the test losses for High-FD, Random, and Low-FD are 1.7821, 1.7926, and 1.9001, respectively. Because the starting point is the same, the lower final test loss for High-FD also corresponds to a larger fine-tuning gain. The LongBench results provide further support: over the three fine-tuning epochs, High-FD consistently achieves lower test loss and higher token accuracy than Random.
Response (Part 3)
Response to Q4: How sensitive are FD values and dataset rankings to the deletion threshold, unit granularity, scoring model, and alternative perturbation methods?
For the threshold, $R_i(u) =\mathcal{L}\left(Y_i,f(W,X_i\setminus u)\right) -\mathcal{L}\left(Y_i,f(W,X_i)\right)>0$ means that deletion increases loss for the same target, matching our definition of content that contributes positively to target prediction. A threshold of gives a meaningful definition for every possible deletion score and a consistent meaning across datasets.
To test sensitivity to near-zero positive scores, we fixed inputs, targets, and the scoring model for the same batch of 10 NIAH examples and deleted consecutive spans of 1, 2, 4, or 8 sentences. Let , where is the number of target tokens. We excluded units with from the numerator, keeping the denominator unchanged.
Results for the same NIAH examples at four deletion granularities
| Sentences per deletion unit | Mean FD | Mean FD after excluding near-zero positive scores | Decrease (percentage points) |
|---|---|---|---|
| 1 | 0.4920 | 0.4564 | 3.56 |
| 2 | 0.4537 | 0.4238 | 2.99 |
| 4 | 0.4888 | 0.4678 | 2.10 |
| 8 | 0.4917 | 0.4917 | 0.00 |
In this batch, mean FD ranges from 0.4537 to 0.4920 across four deletion granularities, a maximum difference of 3.83 percentage points. Excluding near-zero positive scores reduces mean FD by 0.00–3.56 percentage points. Within the tested range, changing deletion granularity and excluding small positive scores have relatively limited effects on mean FD, supporting our measurement rule.
For scoring models and dataset rankings, Figure 5 (Section 4.2) and Figure 11 (Appendix C.2) report mean dataset FD for Qwen3-0.6B and Qwen3.5-0.8B, respectively. Five of six within-task pairs retain their ordering: mathematical reasoning, sentiment classification, translation, coding, and long-text localization. For finetranslations and OPUS Books, FD is 0.814 and 0.769 under Qwen3-0.6B, and 0.889 and 0.879 under Qwen3.5-0.8B. The long-text summarization ordering changes. These results support some cross-model consistency in most within-task comparisons; specific values and rankings remain model-dependent.
We chose deletion because it directly tests whether removing input content makes the same target harder to predict. Prior work similarly removes content and observes prediction changes: ERASER’s comprehensiveness metric removes rationale spans and compares the predicted probability of the same class to assess reliance on those spans [3], providing a methodological basis for deletion.
Masking and replacement require specifying a mask token or replacement content. Covert et al. [4] likewise identify how features are removed as a key design choice in attribution methods. Scores must be interpreted according to their intervention; numerical consistency across perturbations requires separate validation. We use deletion throughout to compute and compare FD under one defined rule.
Response to Q5: How is Assumption 2 satisfied by the fixed-position synthetic construction, and what does Proposition 1 establish beyond concentration of dataset-average differences?
Placing the feature tokens first in Eq. (4) can be viewed as selecting one of the equally likely placements under uniform shuffling. After uniform shuffling in Eq. (5), any further independent permutation preserves the joint distribution of the sequence and target. Assumption 2 thus applies, giving in Proposition 1. Figure 9 (Appendix B.1) compares dataset-level FD under this fixed feature/noise placement with random placement, showing little effect of token order in our synthetic experiments.
We sincerely thank the reviewer again for the detailed and constructive comments, which have substantially improved the quality of our manuscript.
References
[1] Frank Nielsen. Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means. Pattern Recognition Letters, 2014.
[2] Visar Berisha et al. Empirically Estimable Classification Bounds Based on a Nonparametric Divergence Measure. IEEE Transactions on Signal Processing, 2016.
[3] Jay DeYoung et al. ERASER: A Benchmark to Evaluate Rationalized NLP Models. ACL, 2020.
[4] Ian Covert et al. Explaining by Removing: A Unified Framework for Model Explanation. JMLR, 2021.
Thank you for your valuable comments that help us improve manuscript.
We understand that you would like direct test-set evidence showing fine-tuning on higher-FD data improves performance on unseen examples. In the revised manuscript, we added evaluations of test loss and target-token accuracy on 12 real-world datasets. The results are shown in Figure 8 (Section 4.5) and Figures 12, 13, and 14 (Appendix C.2).
The test loss panels in Figure 8 and Figure 12 show an overall advantage for higher-FD datasets within the same task category. Comparing the last recorded point on each curve gives 24 comparisons between higher- and lower-FD datasets across six task categories and four optimizers: the higher-FD dataset has lower test loss in comparisons. In mathematical reasoning, sentiment classification, translation, long-text summarization, and coding, the higher-FD dataset has lower test loss at the curve endpoint under all four optimizers. For long-text localization, the differences are smaller and vary across optimizers. A similar association between higher FD and better fine-tuning performance is shown by the target-token accuracy results in Figures 13 and 14.
Thank you for pointing this out. A measured FD of approximately 0.54 at a Feature Ratio of 0.1 does require a clear explanation, particularly because it affects how we interpret the task-relevant content identified by FD. Feature Ratio is a hyperparameter specified when generating the synthetic data, whereas FD is computed from changes in a given model’s loss on the same target after deletion. We believe this numerical difference can be understood through the data-generation process and the properties of deletion scores.
The generation process is specified in Eq. (4) of Section 3.1: feature tokens are constructed from normalized Gaussian random vectors and carry task signals through the labels, while noise tokens follow a centered Gaussian distribution and are independent of the labels. The model must identify label-related contributions among these random vectors, while noise may also happen to favor the prediction for a particular example when the distributions of feature and noise responses overlap. This is consistent with the relationship between distributional separability and the minimum achievable classification error [1, 2]. A simple example explains why the fraction of noise tokens counted toward FD can be close to one half. Suppose the model’s response to noise is additive:
\xi_j\sim\mathcal{N}\left(0,\frac{\sigma^2}{D}I_D\right).$$ Here, the noise is independent of the label $y\in\{-1,+1\}$, $w\neq0$ is a fixed vector, and $\sigma>0$. In this example, we use $\mathcal{L}(y,f(X))=-yf(X)$ as a loss proxy. Deleting noise token $j$ leaves the signal component and the other noise terms unchanged, so $$f(X)-f(X_{\setminus j})=w^\top\xi_j.$$ The corresponding deletion score is $$\begin{aligned} R_j &=\mathcal{L}(y,f(X_{\setminus j}))-\mathcal{L}(y,f(X))\\ &=y\bigl[f(X)-f(X_{\setminus j})\bigr]\\ &=yw^\top\xi_j. \end{aligned}$$ By the symmetry of the centered Gaussian distribution, this score is equally likely to be positive or negative: $$\Pr(R_j>0)=\Pr(yw^\top\xi_j>0)=\frac12.$$ In other words, although noise is generated independently of the label, its random contribution on a particular example has approximately a one-half probability of supporting the target prediction and therefore receiving a positive deletion score. Here, $1/2$ is the false-positive probability for a noise token in this example. The precision and recall requested in Q3 need to be computed separately at each Feature Ratio using the known true feature labels. Thus, for a given Feature Ratio $\rho$, if most true features are identified, their contribution to FD is approximately $\rho$, while the expected contribution from noise is $(1-\rho)/2$, giving $$\mathbb{E}\!\left[\mathrm{FD}\right] \approx \rho+\frac{1-\rho}{2}.$$ Substituting $\rho=0.1$ gives $0.55$, close to the measured value of approximately $0.54$ in Figure 3. This example shows that the value of $0.54$ implies most feature tokens have been identified by FD, such that its value concentrates around $\rho+\frac{1-\rho}{2}$. Even when most features are identified, centered Gaussian noise can still bring FD close to this level through the mechanism described above.