acceptodds
Under review as a conference paper at ICLR 2027

ReGIFT: Refreshed Geometry for Instruction Fine-Tuning via Dynamic Data Selection

Abstract

Instruction-data selection aims to improve fine-tuning performance under a limited training budget. Static selectors fix a subset before training, whereas dynamic selectors revisit it as the model evolves. Repeated selection therefore requires a ranking criterion that adapts to the model and remains affordable to recompute. We introduce ReGIFT (Refreshed Geometry for Instruction Fine-Tuning), a dynamic selector that constructs and updates a geometric reference from the current model's internal representations of the training pool at every refresh. It estimates layer-wise principal directions from a small uniform pool probe and adjusts loss–entropy rankings using candidate representations obtained during the existing scoring forward passes. This design requires no separately curated reference set, external judge, or learned selection policy. Using Davis–Kahan analysis, we bound changes in the geometric reference between refreshes under bounded covariance change and positive spectral separation. In single-run comparisons, the method selects 10% of the pool per refresh and achieves the highest eight-task average among all compared methods in the main setting, including full-data fine-tuning. Across six backbone–pool settings, it ranks first on the task average in five and second in one among the methods evaluated in each. In the main setting, total runtime is 33.0% lower than Data Agent.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Instruction tuning, Dynamic data selection, Hidden-state representations, Large language model, Supervised fine-tuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MjUUOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ReGIFT, a dynamic data selection method for instruction fine-tuning. At each selection refresh, ReGIFT samples a small uniform probe from the training pool and computes layer-wise PCA directions from last-minus-first hidden-state differences. Candidate examples are then scored by projecting their mean hidden representations onto these directions. The resulting normalized alignment score multiplicatively adjusts a loss-entropy-based carrier score, and the top 10% of the training pool is selected for subsequent fine-tuning. The method does not require a separately curated reference set, external judge, or learned selection policy.

Critical strengths

1.ReGIFT reuses candidate forward passes needed for loss and entropy computation and adds probe extraction, PCA, pooling, and projection rather than training a separate selection policy. This makes the method practically simpler than approaches such as Data Agent.

2.In the main experiment, ReGIFT improves the eight-task average from 38.64 for Carrier-Only to 40.70 and improves every displayed task relative to Carrier-Only. Therefore, the complete scoring rule does appear to change the selected data in a potentially useful way.

Critical weaknesses
  1. The geometric signal is almost fully predictable from sequence length (R2=0.991R^2 = 0.991–0.9920.992), making the unique value of the proposed geometry unclear.

  2. The main results use only one random seed, and ReGIFT's λ\lambda is tuned while the baselines receive no analogous tuning, weakening the reliability and fairness of the comparison.

Questions for authors

1.Does ReGIFT still outperform a variant with a fixed geometric reference?

2.Can the authors provide multi-seed results for the main comparison?

  1. Was λ\lambda selected on a separate validation set?
3 comments
AuthorsOpenReview ↗

Reference refreshing, direct length controls, and repeated-run evidence

We implemented Frozen, Carrier+Length, and Longest to address your concerns about reference refreshing and length dependence. Thank you for helping us sharpen the experimental design.

Q1 & W1: Testing reference refreshing and direct length replacement.

Table 11B compares these controls with the main ReGIFT implementation under shared training and evaluation settings.

ControlChange relative to ReGIFT
FrozenFix the initial reference directions; keep updating candidate representations and the loss-entropy carrier.
Carrier+LengthKeep the carrier; replace alignment with response-length percentile, without learning geometric targets.
LongestSelect by response length alone.
Carrier-Only (CO)Keep the carrier; remove the alignment multiplier.

Across three runs per method, ReGIFT has the highest mean performance. AVG4 averages MMLU, SVAMP, MBPP pass@1, and TyDiQA pooled F1.

MethodAVG4 (mean ± sample SD)
ReGIFT46.27 ± 0.57
Carrier+Length45.73 ± 0.42
Longest45.45 ± 0.34
Frozen45.29 ± 0.18
Carrier-Only (CO)44.81 ± 0.90

These results support refreshing the reference as the model learns and retaining alignment rather than replacing it with either tested direct length rule. In particular, Carrier+Length preserves the adaptive carrier, so its comparison tests length replacement without also removing the carrier.

The separate supplementary ReGIFT-M controls (Table 11A) show that a length predictor trained on refreshed alignment achieves a slightly higher mean and lower SD than the original alignment. This limits a length-independent interpretation of that signal; Table 11B instead tests direct length rules without those geometric targets.

Q2 & W2: Repeated runs and reported variability.

We report three runs per method in each of the following performance comparisons, with the training seeds and sample standard deviations made explicit:

  • Table 11A: ReGIFT-M, length-matched Random, length-predicted alignment, and within-length shuffled alignment; seeds 1/7/42, mean ± sample SD.
  • Table 11B: ReGIFT, Frozen, Carrier+Length, Longest, and CO; seeds 1/7/42, mean ± sample SD.
  • Table 12: Individual task scores for all 15 runs underlying Table 11B, so readers can inspect variation beyond the aggregate.

Table 1 additionally tracks reference movement under two learning-rate schedules with three seeds (42/43/44), reporting medians. Figures 1(b), 4, and 5 show the corresponding selection and reference diagnostics with individual seeds or across-seed ranges. Thus each repeated comparison uses at least three runs, with sample SDs for performance and the stated summaries for diagnostics. The main eight-task comparison against Data Agent and Full FT remains single-run; the repeated downstream evidence addresses the length and reference controls above.

Validation question & W2: Weight sensitivity and a configurable scoring rule.

Every tested positive weight improves all four tasks over CO (Figure 3): the benefit is not confined to the selected value. We also retain λ=1\lambda=1 without further search in all five transfer settings, where ReGIFT exceeds Data Agent and Full FT in reported AVG4 (Appendix D).

We selected λ=1\lambda=1 by the highest reported AVG4 in the sweep, without a separate validation set; baselines received no analogous sweep. At this value, the multiplier has the simple range [1,2][1,2], limiting the geometric bonus to the carrier score. This gives a clear interpretation of the chosen setting, but does not make it universally optimal.

The algorithm already exposes λ\lambda as a parameter:

si(t)=Ri(t)(1+λ×aligni(t)),λ≥0.s_i^{(t)}=\mathcal R_i^{(t)}\bigl(1+\lambda \times \mathrm{align}_i^{(t)}\bigr),\qquad \lambda\ge0.

Our bounded-adjustment analysis applies to any nonnegative weight: the multiplier lies in [1,1+λ][1,1+\lambda], preserving carrier preferences whenever Ri>(1+λ)Rk\mathcal R_i>(1+\lambda)\mathcal R_k (Appendix B.1). The preferred weight may change with the model, dataset, or objective. Future work can therefore select it on validation data under a common tuning budget while retaining the reference construction and this theoretical property.

Revision locations: Section 3.4; Appendices C.1 and E.3–E.4 (control protocols); Tables 1, 11–12 and Appendix F (repeated evidence); Appendix E.1/Figure 3 and Appendix D (weight sweep and transfers); Appendix B.1 (general weight analysis).

You solved most of my problems, so I raised my score.

AuthorsOpenReview ↗

Thank you for your thoughtful feedback

Dear Reviewer,

Thank you so much for your thoughtful review and for taking the time to reconsider our work. Your questions and insights helped us strengthen the paper, and we have learned a great deal from this exchange. We will carry these lessons into our future research and strive to contribute to the community with the same constructive spirit. Thank you again for your time and support!

Reviewer_kxBmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReGIFT, a dynamic data selection method for instruction fine-tuning. ReGIFT combines loss–entropy scores with a geometry-based score obtained by applying PCA to the model's hidden representations. The geometric reference is updated periodically during training to adapt data selection to the evolving model. Experiments across several models and benchmarks show that ReGIFT achieves competitive performance while reducing the computational cost compared to Data Agent.

Critical strengths

The paper is well-written and easy to follow. The authors address an important question, and the proposed method performs competitively. Moreover, the finding that ReGIFT's geometry alignment score can be well predicted by length is interesting.

Critical weaknesses

The contribution is incremental, as ReGIFT can be viewed as a combination of NAIT [1] and Data Agent [2].

ReGIFT's geometry alignment score can be well predicted by length. However, selecting by length [3] is also a well-known method.

[1] Neuron-Aware Data Selection in Instruction Tuning for Large Language Models, ICLR, 2026.

[2] Data Agent: Learning to Select Data via End-to-End Dynamic Optimization, ICML, 2026

[3] Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning, ICML, 2024

Questions for authors

My main concern is about selection by length. It is an interesting discovery, but I think the author should explain why.

  1. The author has noted that ReGIFT's geometry alignment score can be predicted by length. Can you justify why? Since the R2R^2 is above 99%, it is reasonable to find out why.

  2. Moreover, if you can prove that ReGIFT equals selection by length, the proposed method seems to be a heuristic rule. I think the meaningful part of ReGIFT is the mean representation, since it relates to length (mean representation = sum representation/length).

  3. Can you conduct more ablation studies to find the difference between using fixed geometry and refreshed geometry?

  4. Can you directly compare with selecting by length?

3 comments
AuthorsOpenReview ↗

Contribution, direct length controls, and reference refreshing

Thank you for guiding the experimental design. We added direct length and fixed-reference controls and clarified the contribution.

W1: Contribution relative to NAIT and Data Agent.

We build on activation-PCA projections and loss-entropy signals used in NAIT [1] and Data Agent [2]. ReGIFT combines a reference refitted from a uniform probe of the training pool with a bounded adjustment to carrier rankings, requiring neither supplied capability-reference data nor a learned selection policy.

At λ=1\lambda=1, the multiplier lies in [1,2][1,2], preserving carrier preferences separated by a factor greater than two. Our conditional Davis-Kahan analysis relates reference movement to covariance change and spectral separation; the Frozen control below tests the downstream value of refreshing.

W2 / direct length and refresh comparisons.

Following your suggestions and Long Is More [3], we added:

  • Carrier+Length: keep the adaptive loss-entropy carrier; replace alignment with response-length percentile.
  • Longest: select by response length alone.
  • Frozen: fix the initial reference directions while updating candidate representations and carrier scores.

Table 11B uses the main ReGIFT implementation, with training and evaluation fixed across methods. Three runs per method use seeds 1/7/42; AVG4 averages MMLU, SVAMP, MBPP pass@1, and TyDiQA pooled F1.

MethodAVG4 (mean ± sample SD)
ReGIFT46.27 ± 0.57
Carrier+Length45.73 ± 0.42
Longest45.45 ± 0.34
Frozen45.29 ± 0.18
Carrier-Only (CO)44.81 ± 0.90

Neither tested direct length rule recovers ReGIFT's observed mean performance. Carrier+Length tests the choice of adjustment with the carrier retained. Frozen supports refreshing reference directions beyond updating candidate representations and carrier scores. CO removes the adjustment altogether.

Length mechanism and the role of ReGIFT-M.

Your mean-pooling observation identifies a route for length dependence. At fixed reference directions, let qkq_k be token kk's projection summed across layers. For K>1K>1 tokens, raw alignment is

a=1K∑k=1Kqk=q1K+K−1Kqˉ2:K.a=\frac{1}{K}\sum_{k=1}^{K}q_k=\frac{q_1}{K}+\frac{K-1}{K}\bar q_{2:K}.

A distinctive first-token projection can introduce length dependence. In a separate checkpoint diagnostic, excluding that token reduced length-prediction R2R^2 from 0.79 to 0.22 (Appendix E.4). This supports a pooling contribution, but does not fully explain Panel A's high predictability: token states also depend on content and the model.

Panel A reuses existing supplementary ReGIFT-M runs, with training and evaluation matched across controls. M denotes mean calibration of the reference sign, not candidate pooling. Its sign convention and training recipe differ from main ReGIFT (Appendix C.1).

Length closely reproduces its alignment and selection; training with the fitted proxy yields a slightly higher mean and lower SD (Table 11A). This does not establish an advantage beyond the length-predictable component in that configuration. However, the proxy learns refreshed alignment at every round; Panel B tests direct replacement without those geometric targets. Both panels include downstream training, and their scores are compared within each panel.

Revisions: Section 1 (contribution); Section 3.4 and Appendices C.1/E.3-E.4 (control purposes and protocols); Tables 11-12 (three-run summaries and individual results); Appendix B (ranking and reference movement).

[1] NAIT, ICLR 2026. [2] Data Agent, ICML 2026. [3] Long Is More for Alignment, ICML 2024.

Thanks for your answer. I will maintain my score.

AuthorsOpenReview ↗

Thank you!

Thank you for your careful reading and for engaging with our response. We understand and respect your decision. The length-controlled evidence is now in the manuscript, and we hope it will be useful in the AC discussion. Once again, we appreciate your time and effort.

Reviewer_rAX1OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReGIFT, a dynamic data selection method for instruction fine-tuning. At each selection refresh, the method uniformly samples a small probe from the training pool and applies PCA to the current model’s layer-wise hidden representations to reconstruct geometric reference directions. Candidate alignment with these directions adjusts a base score combining loss and predictive entropy. The top-scoring 10% of examples are then selected for the next training interval. The method requires no manually curated reference set, external judge, or separately trained selection policy. The authors report performance improvements across several models and datasets and provide a conditional matrix perturbation analysis characterizing changes in the reference directions during training.

Critical strengths
  1. The method is simple and has practical integration value. Candidate representations reuse the forward passes required to compute loss and entropy, and no separate selection policy needs to be trained. In the main experiment, total runtime decreases from 4.45 hours for Data Agent to 2.98 hours for ReGIFT, while average task performance improves.

  2. The authors clearly disclose several important limitations. The paper distinguishes selecting 10% at each refresh from accessing only 10% of unique examples throughout training. It also includes length-matched, length-predicted, and shuffled-alignment controls, which provide useful evidence about the selection mechanism.

Critical weaknesses
  1. In ReGIFT-M, alignment is highly predictable from sequence length (R² = 0.991–0.992), and length-predicted alignment achieves comparable performance. Although the predictor still requires geometric supervision, these results leave the independent benefit of geometric information beyond length unclear.

  2. The Carrier-Only ablation shows that geometric adjustment helps, but does not isolate the benefit of refreshing reference directions. A comparison with fixed initial directions under identical dynamic carrier scoring is needed. The stability analysis does not establish downstream benefits from refreshing.

  3. There is only a slight performance difference between the main comparison and transfer comparison, and the robust advantage remains uncertain. ReGIFT’s alignment weight is also selected through the reported AVG4 sweep, whereas baselines receive no comparable tuning, limiting confidence in the reported advantages.

  4. The mechanism studies use ReGIFT-M, which differs from the main method in direction-sign calibration, probe size, sequence cutoff, and training duration. Although disclosed, these differences limit how directly the length controls and geometric interventions explain the main method’s behavior.

Questions for authors

Please see the weaknesses.

5 comments
AuthorsOpenReview ↗

Direct length controls, reference refreshing, and comparison robustness

Thank you for the constructive feedback. We added direct training comparisons to test length-based replacement and reference refreshing.

W1-W2: Direct length replacement and reference refreshing.

  • Carrier+Length retains the adaptive loss-entropy carrier and replaces alignment with response-length percentile.
  • Longest selects by response length alone.
  • Frozen fixes the initial reference directions while candidate representations and carrier scores continue to update.

Table 11B. Three runs per method; mean ± sample SD.

MethodAVG4
ReGIFT46.27 ± 0.57
Carrier+Length45.73 ± 0.42
Longest45.45 ± 0.34
Frozen45.29 ± 0.18
Carrier-Only (CO)44.81 ± 0.90

These comparisons support retaining alignment over the tested direct length replacements and refreshing its reference during training. Our conditional Davis-Kahan analysis bounds reference movement by covariance change relative to spectral separation; Frozen supplies the downstream evidence for refreshing. Together with the bounded multiplier, this motivates adapting the reference to the evolving model while preserving sufficiently separated carrier preferences (Appendix B).

W1 & W4: Why the two length-control settings?

We agree that length strongly predicts ReGIFT-M alignment, and the fitted proxy achieves a similar downstream mean (Table 11A). High score correlation alone does not guarantee that direct length selection will preserve training performance: the proxy still learns from refreshed alignment.

Panel A reuses ReGIFT-M, the mean-calibrated implementation from our existing supplementary runs, keeping training and evaluation settings fixed across controls. These are full training comparisons: matching selected lengths does not recover the complete selector's mean performance, while predicted and shuffled alignment retain the carrier to test the alignment signal.

Panel B uses the ReGIFT implementation from the main experiments to test length-based replacements without geometric targets and the benefit of refreshing. Training and evaluation are again fixed across controls. Appendices C.1 and E.3-E.4 now explain the purposes and protocols together.

W3: Robustness and the role of the alignment weight.

All tested positive weights improve every evaluated task over CO (Figure 3). We selected λ=1\lambda=1 from this sweep and kept it unchanged in five transfer settings, where ReGIFT exceeds Data Agent and Full FT in reported task-average performance (Appendix D). The initial choice used reported AVG4 without a separate validation set or analogous baseline sweep (Appendix E.1).

The sweep and transfers support λ=1\lambda=1 as a practical starting point. The preferred strength can vary by model, dataset, or objective, so λ\lambda explicitly controls how strongly geometry adjusts the carrier ranking. For new settings, we recommend choosing it on held-out validation data under a common tuning budget.

Table 11B adds repeated-run evidence for the direct length and refresh controls; the main eight-task baseline comparison remains single-run.

Revisions: Section 3.4 and Appendices C/E clarify the two experimental questions, matched protocols, and three-run results; Appendix B relates reference movement to the selection rule; Appendices E.1/D report the weight sweep and transfers.

The author addressed the most important issue I care about.

However, given that ReGIFT improves upon Carrier+Length by only approximately 0.54 percentage points, how do the authors justify the independent contribution of geometric information beyond sequence length, and what are the implications of this modest gain for the paper's main claims?

AuthorsOpenReview ↗

Scope of the contribution beyond length and an additional replacement test

Thank you for this follow-up. We agree that the gain is modest and that length explains much of the alignment signal. Your question helped us clarify what our comparisons support and how narrowly we should state the contribution beyond length.

Our evidence supports a modest benefit over the tested length-based replacements, rather than a general contribution independent of length. We therefore frame the contribution around the complete refreshed selection rule and its observed performance.

Existing paired evidence. ReGIFT exceeds Carrier+Length in all three seed-matched runs, by +0.72, +0.50, and +0.43 AVG4 points. This supports retaining alignment over the tested response-length-percentile replacement, but not superiority over length-based selection generally.

Why an additional length control? Panel A tested length-predicted alignment in ReGIFT-M, where the proxy achieved a similar downstream mean. Panel B compared main ReGIFT with direct length rules: response-length percentile and longest-response selection. We therefore tested whether a learned length approximation could also replace alignment in the main configuration.

At each refresh, a five-fold out-of-fold ridge predictor estimates alignment from ten token-length features. Scaling and fitting use only the training folds, so each example receives a held-out prediction. These predictions replace alignment during selection, with the carrier rule and training recipe unchanged. Evaluating the resulting model tests whether the approximation preserves downstream performance.

Under a shared supplementary evaluation protocol, ReGIFT achieves 46.253 AVG4, versus 45.708 for the control (+0.545 points). The proxy preserves most of the performance, with a small additional gain from original alignment. This is one paired seed, reported separately from the earlier three pairs. The proxy still requires refreshed geometric targets.

Implications for our claims. We have added this comparison and its scope to Appendix E.4. Length captures much of the measured behavior. The additional result supports a modest benefit over this approximation, but does not establish reliability across training seeds or a generally length-independent effect.

Our contribution is a dynamic selection rule that refits pool-derived references using current model representations and applies a bounded adjustment to adaptive carrier scores. It reuses candidate-scoring forward passes without a curated reference set or a learned selection policy. The length controls assess the value of retaining alignment; the Frozen comparison separately addresses reference refreshing.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision summary: length controls and reference refreshing

We thank the reviewers for their constructive feedback. In the additional three-run comparisons, ReGIFT achieves higher mean performance than both direct length baselines, Carrier+Length and Longest, and than Frozen and the carrier-only control (Table 11B).

Role of ReGIFT-M. ReGIFT-M is the mean-calibrated implementation used in our existing supplementary runs. Panel A uses these runs as its reference, with training and evaluation held fixed within each comparison. Its sign convention and training setup differ from main ReGIFT, so the findings describe length dependence in that configuration. Panel B directly tests length-based replacements for ReGIFT.

Scope. Length predicts ReGIFT-M alignment well, and the fitted proxy has a slightly higher observed mean. However, it still learns from geometric alignment at every refresh. The two direct length rules, which need no geometric targets, do not recover ReGIFT's observed mean performance. Together with the Frozen and carrier-only comparisons, this supports the practical contribution of using and refreshing the geometric adjustment under the tested protocol.

Revisions. We clarified the two experimental questions and their protocols (Section 3.4; Appendices C/E), and report three runs with variability and per-run results for the direct comparisons. Appendix E.1 explains the alignment-weight sweep: λ=1\lambda=1 was selected by the best observed score without a separate validation set or analogous baseline sweep, then reused without retuning in five transfer settings (Appendix D). Substantive manuscript changes are marked in red.

Detailed point-by-point responses are provided separately for each reviewer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.