acceptodds
Under review as a conference paper at ICLR 2027

Efficient Test-Time Finetuning of LLMs via Convex Reconstruction and Gradient Caching

Abstract

Test-time finetuning (TTFT) is a rapidly evolving paradigm that adapts a language model to each prompt by retrieving related sequences, updating the model on them, and then evaluating the prompt. However, TTFT is only practical if it is fast: selection and finetuning both happen per query, making each a direct bottleneck. Existing methods trade speed for quality: fast retrieval is often redundant, while stronger diversity-aware selection adds prohibitive per-query cost. We introduce HullFT, a geometric approach to TTFT that addresses both bottlenecks. Given a query, HullFT first represents the query embedding as a sparse convex combination of few training sequences, using efficient projection-free Frank–Wolfe optimization. This yields a support set that is inherently relevant and diverse. We then convert the fractional convex weights into an exact integer multiset for finetuning through a geometric integerization procedure. The resulting multiplicities naturally create repeated examples, which we exploit with Gradient Reuse to amortize forward– backward computation across repeated finetuning steps. Our experiments show that HullFT improves the quality–efficiency tradeoff over current state-of-the-art TTFT methods, achieving lower bits-per-byte at substantially lower total runtime

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLMs, Fine-Tuning, Test-Time

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_ESAxOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  • This paper introduces HullFT, a geometric approach that improves the quality-efficiency tradeoff of TTFT. Motivated by approximate Carathéodory and the geometric intuition that direction encodes semantics, HullFT constructs a Frank-Wolfe sparse convex reconstruction of the query plus a geometric integerization into an N-example multiset, which achieves the largest gains at tight budgets. Extensive experiments demonstrate the generally competitive performance of HullFT.
Critical strengths
  • HullFT is supported by a relatively detailed theoretical analysis.
  • The method is evaluated on GPT-2 and Pythia-410M across 12 Pile subsets. On GPT-2, it achieves lower averaged BPB% than kNN and SIFT across the evaluated wall-clock budgets up to approximately 4 seconds. CPU-only experiments also provide evidence of its computational efficiency.
Critical weaknesses
  • In Section 3.1, the paper lacks a theoretical analysis of the connection between convex reconstruction quality and TTFT performance. Equation (1) and the approximate Carathéodory theorem only support sparse reconstruction in the embedding space, but do not explain why lower reconstruction error leads to better finetuning performance.
  • In Section 4 and Appendix B, the experiments do not establish whether HullFT's quality–efficiency advantage generalizes to modern LLMs and practical downstream tasks.
    • First, HullFT only evaluates BPB improvements on The Pile dataset, without testing common downstream tasks such as math, code generation and tool use task.
    • Second, HullFT is only evaluated on GPT-2 and Pythia-410M. The behavior of GPT-2-style models differ substantially from mainstream LLMs. Therefore, the claimed quality–efficiency advantage needs to be validated on more recent backbone models, such as the latest Qwen and Llama models.
    • Third, the empirical comparison is limited to kNN and SIFT, missing recent test-time adaptation methods such as TLM [1], FG-TTL [2], and QueST [3].
  • It remains unclear whether parameter-updating test-time adaptation is a practical choice for mainstream large LLMs, given the computational and memory costs of updating models with billions of parameters. The paper should therefore also compare HullFT with non-parametric test-time compute methods, such as in-context learning and Best-of-N sampling.

[1] Hu et al., Test-Time Learning for Large Language Models, ICML 2025.

[2] Bian et al., Future-Gain Guided Test-Time Learning for Large Language Models, ICML 2026.

[3] Song et al., Query-Conditioned Test-Time Self-Training for Large Language Models, Arxiv 2605.

Questions for authors
  • In Section 3.2, Gradient Reuse lacks a theoretical analysis of the optimization error. Equation (3) reuses gradients computed at earlier parameter states, and the cached gradients differ from the true gradients at the updated parameters. An analysis of how this difference affects subsequent parameter updates may help.
  • In Section 4 and Appendices A–B, the authors do not explain the selection criteria for hyperparameters such as the model learning rates, or whether validation data were used to select the gradient reuse interval.
Reviewer_ow12OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HullFT, a data-selection and finetuning pipeline for test-time finetuning (TTFT) of language models. Given a query, it retrieves a kNN candidate pool (K=200) and runs Frank–Wolfe on the simplex to approximate the query embedding as a sparse convex combination of candidates. It then converts the fractional weights into an exact N-element multiset through greedy filling and local swaps. Because the resulting multiset contains consecutive repeats, the authors apply Gradient Reuse: the gradient is recomputed only every r steps on a repeated example and reused for the steps in between. On 12 Pile subsets with GPT-2 (and Pythia-410M in the appendix), HullFT achieves lower bits-per-byte than kNN and SIFT at a fixed total-runtime budget, especially at tight budgets. The authors acknowledge that at matched N, SIFT selects slightly better data.

Critical strengths

The method is simple, cheap, and easy to adopt: Frank–Wolfe selection on a 200-point pool runs in tens of milliseconds, roughly an order of magnitude faster than SIFT, and reaches comparable diversity of the selected support (Table 6). The paper is also unusually transparent about its own limitations. It reports that SIFT is better at matched N (Table 8), presents diversity as parity rather than a win, and includes careful ablations of integerization, FW tolerance, pool size, and training order. These make the actual source of the gains easy to trace.

Critical weaknesses
  • The headline quality–runtime gains appear to come mostly from applying Gradient Reuse to HullFT but not to the baselines, rather than from the selection method. At matched N, SIFT selects better data (Table 8), and the selection-time saving (about 0.46s at N=50) is small relative to finetuning time (about 5.6s). Yet Fig. 2 and Table 1 compare HullFT with Gradient Reuse against kNN and SIFT without it. Table 2 shows that SIFT also produces many repeats (15.9 distinct points at N=50) and benefits from Gradient Reuse just as well: SIFT with global dedup and r=2 reaches 67.45 BPB% in 4.16s, compared with 67.70 BPB% in 3.82s for HullFT with r=2. Under this fairer comparison, the remaining advantage is roughly the selection-time difference. The 3.4–3.8% gaps reported at T=1.75–2s therefore likely overstate the benefit of HullFT itself.

  • Key simple baselines are missing, so it is unclear what the proposed components add. First, Gradient Reuse with Adam on an identical example is essentially a stale-gradient update, close to taking fewer steps with a larger learning rate. However, every method uses a fixed learning rate (5e-5) with one step per sequence, so the time-budget comparison conflates selection quality with the number of optimizer steps. A step-reduced, learning-rate-scaled variant of each method is the natural control. Second, cheap query-conditioned diversity heuristics run at about the same cost as Frank–Wolfe on a 200-point pool and directly test the claim that convex reconstruction is a particularly good fast diversity mechanism. Examples are MMR (which includes query relevance), kNN with near-duplicate filtering, and k-center selection. The scope is also narrow: only GPT-2 and Pythia-410M, Pile BPB only, and runtime that excludes retrieval. This makes it hard to judge whether the selection-time advantage matters at scales where finetuning cost dominates even more.

Questions for authors
  1. Can the authors reproduce Fig. 2 and Table 1 with SIFT combined with Gradient Reuse (global dedup and consecutive variants) as baselines across all budgets? Does HullFT remain Pareto-dominant, and by how much?

  2. How does Gradient Reuse compare with simply taking ceil(c_j / r) steps on each repeated example with a scaled or tuned learning rate? Was the learning rate tuned separately for each method under each time budget? If a learning-rate-scaled baseline matches Gradient Reuse, the finetuning-side contribution reduces to step-count control that applies equally to all selectors.

  3. How does HullFT compare with fast diversity-aware selectors on the same pool, such as MMR, kNN with near-duplicate filtering, or k-center greedy, which have similar selection cost? This would isolate whether convex reconstruction specifically matters.

  4. Table 7 shows that shuffling HullFT's multiset improves BPB% by 0.63, while Gradient Reuse requires consecutive blocks. Is the combined cost of this ordering constraint plus reuse (about 1.3 BPB% relative to shuffled training without reuse) worth the speedup? Also, how do the gains change with larger base models and with retrieval time included in the end-to-end latency?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.