- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies online data selection and reweighting for LLM fine-tuning. The authors formulate sample selection as subset-level gradient matching under an optimizer-induced geometry and propose Filter-then-Weight, which first constructs a subset using a greedy residual filter and then solves a non-negative least-squares problem to determine sample weights. To make per-example gradient comparisons practical at LLM scale, the method uses factorized gradient representations, random projections, and token-first contractions. Experiments on Llama and Qwen models compare the method against several gradient-based data-selection baselines on MMLU, TyDiQA, and additional downstream tasks.
Critical strengths
-
The paper addresses an important practical problem. Online data selection is more challenging than offline selection because sample utility changes with the model parameters, and accounting for the actual optimizer state is a meaningful consideration. The subset-level perspective is reasonable. Modeling interactions among jointly selected examples is potentially more expressive than assigning independent scores to individual samples, especially when candidate gradients are redundant.
-
The implementation is carefully engineered. The factorized representation and token-first contraction substantially reduce the memory cost of computing candidate-target and candidate-candidate gradient statistics, making the approach applicable to models up to 8B parameters. The paper contains extensive ablations. In particular, the authors separately study filtering, reweighting, optimizer-aware preconditioning, projection dimension, candidate-pool size, and the frozen-preconditioner approximation.
Critical weaknesses
-
The methodological novelty appears somewhat incremental. The main ingredients are gradient matching, greedy residual selection related to OMP, non-negative least squares, optimizer-aware gradient preconditioning, whitening/standardization, and random projection. The central algorithmic modification is to remove coefficient refitting from the greedy support-construction stage and postpone NNLS until after the support is fixed. This is intuitively reasonable and empirically useful, but I am not convinced that this constitutes a sufficiently substantial methodological contribution by itself.
-
The baseline comparison is difficult to interpret because several prior methods are substantially modified to fit the proposed online setting. In particular, TracIn and LESS normally exploit historical checkpoints or full-corpus information, but the paper removes these components and evaluates only their scoring rules using the current-step gradients. This creates useful controlled baselines, but they are no longer the original algorithms. The evaluation relies heavily on repeated access to a target/validation distribution, and the role of this target data needs clearer characterization. At every selection step, the method computes a target gradient from a validation mini-batch and explicitly chooses training samples to align with it. Thus, the method is not task-agnostic online selection but rather repeated target-supervised selection.
Questions for authors
-
Since Theorem C.1 does not establish equivalence between the alignment objective and squared-distance matching for nonlinear , what theoretical justification supports Eq. (2) for Adam and AdamW, which are used in all experiments? Can the authors provide a bound relating optimization of the practical surrogate to improvement of the original target loss?
-
The frozen second-moment approximation has high Spearman correlation with the candidate-specific version, but the reported relative errors are --. Why is ranking agreement sufficient for the subsequent NNLS weighting stage, which depends on the actual magnitudes of the Gram and alignment terms rather than only their ordering?
-
Is there any theoretical stability or approximation result showing that the proposed unit-weight residual filtering is more robust than coupled OMP under noisy/projected gradients? At present, the motivation from classical OMP instability does not directly imply that the proposed decoupling solves the problem.
-
Could the authors clarify the novelty of Filter-then-Weight relative to existing gradient matching plus post-selection NNLS/reweighting? Which component of the two-stage optimization procedure is fundamentally new beyond combining existing selection, preconditioning, and reweighting techniques?
-
TracIn and LESS are modified substantially to operate without their historical checkpoint information. Could the authors discuss how much performance these modifications remove from the original methods? More importantly, could the authors compare against additional methods that are natively designed for online and optimizer-aware data selection?
Does the evidence isolate Filter-then-Weight’s contribution?
The revision strengthens the empirical case by reporting fixed-budget results for all seven main configurations. The long-context memory measurements are also valuable. My main concern is whether the reported gains can be attributed to the proposed selection mechanism and whether that mechanism reliably improves the update it is designed to shape.
1. The contribution of residual filtering remains unresolved. The decisive comparison is top-k on exactly the same standardized alignment vector, followed by the same weight solver. Table 4’s Hard Filter+Reweight variant changes both the ranking scores and the selection rule. Appendix E.6 reports identical subsets on 51 saved states for two nonzero penalty scales, making the zero-penalty control particularly informative.
There is also a failure mode with exact statistics. Consider four standardized candidates along the positive and negative coordinate axes in two dimensions, ordered as e1,e2,−e1,−e2 and scaled to have squared norm 2. Let their alignment scores be (0.3,0.2,−0.3,−0.2) and the budget be two. Equation (5) first selects candidate 1. It then scores candidate 3 at −0.3−(−2)=1.7, ahead of candidate 2 at 0.2. The exact solution to Eq. (6) with positive ridge assigns candidate 3 zero weight. The fixed interim coefficient thus spends half the selection budget on a candidate that the final objective discards, even in the absence of estimation noise.
2. The surrogate and the released solver each have a concrete failure mode. With candidates g=(0.1,1) and −g, target gradient h=(1,−1), and a scalar frozen preconditioner rescaled to the identity, the first-order alignments of the frozen update are (−0.9,0.9). Equation (4) gives (9,−9) using population standard deviations. With budget one, the surrogate selects an ascent direction even for a linear target loss. The fixed-subset replay in Appendix E.12 leaves this selection failure untested.
Separately, feeding the released NNLS routine the Gram and alignment vector formed from candidate rows (1,0,0),(0,1,0),(−1,0,1) and target (1,1,−2) yields normalized weights (0,1,0) at learning rate 0.0001 with the routine’s default ridge. The exact constrained solution, normalized in the same way, is (0.5,0.5,0). Before normalization, the first coefficient is zero even though increasing it strictly decreases the objective. The routine removes variables after the unconstrained solve and cannot restore an incorrectly removed variable. This matters because the manuscript uses zero coefficients to interpret redundancy and effective batch use. Could the authors identify the solver version used for the reported runs and check its optimality residuals on saved states? Released solver
3. The TyDiQA comparisons should be verified on the same evaluation items. The released evaluator sets
random.seed(42)but draws samples while iterating over a Python set of languages. With the public LESS data, changing onlyPYTHONHASHSEEDfrom 1 to 2 produced two sets of 1,713 evaluation questions with 1,034 in common. Whether this affected the reported results depends on the runtime configuration. Verifying the question and one-shot demonstration IDs across methods and checkpoints, or reevaluating on a fixed list, would resolve this concern. Evaluator · Public LESS dataGRAD-MATCH already refreshes weighted subsets during training, and LoGra Eq. (6) already projects both gradient factors before summing over tokens. The online fine-tuning integration has value, and these precedents make it especially important to isolate what this particular combination adds. On the current evidence, I recommend rejection. The result most likely to change my assessment is a comparison of residual filtering against matched-score top-k, using the same verified solver and a fixed evaluation set, with both downstream performance and actual one-step held-out loss changes reported. Each variant should be allowed to select its own subset in the replay. GRAD-MATCH · LoGra