acceptodds
Under review as a conference paper at ICLR 2027

Is Human-Readable Text Necessary for Effective LLM Fine-Tuning?

Abstract

Is human readability necessary for effective fine-tuning of large language models? We investigate whether model-conditioned training representations can preserve or improve adaptation utility without requiring a human-readable textual form. We propose Desired-Update-Aligned Synthetic Data (DASA), which uses activation-gradient feedback from a frozen reference model to guide the optimization of continuous synthetic input embeddings. Inspired by the role of activation gradients in local risk reduction, DASA targets useful adaptation updates rather than source-text reconstruction or linguistic fluency. The resulting embeddings are used directly for downstream fine-tuning; discrete token projections are employed only for qualitative inspection. Experiments on six models from the Llama and Qwen families, ranging from 1B to 32B parameters, cover six benchmarks spanning knowledge, mathematical reasoning, code generation, and commonsense reasoning. Under matched LoRA adaptation settings, DASA achieves performance comparable to the source natural-language data and surpasses it in multiple configurations, while outperforming GRADMM in most comparisons. Further experiments cover general-domain and task-specialized source data. Under the evaluated synthesis settings, DASA provides a – speedup over GRADMM with comparable peak GPU memory.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Synthetic Data Activation-Gradient Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Eza2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose to explicitly learn continuous SFT data by aligning the gradients that it induces with those from human-readable SFT data. They apply this approach to a variety of pretrained models and SFT datasets, and find that finetuning on their continuous SFT data matches or performs very slightly better than finetuning on the original human-readable SFT data.

Critical strengths
  • The setting of learning continuous representations and training on them is generally interesting.
  • The empirical evaluation is fairly comprehensive in terms of the number of models and SFT datasets it considers, although the gains are small.
Critical weaknesses
  • The novelty is limited. The claim that continuous inputs can effectively finetune an LLM is well-established in prior dataset distillation works, such as the cited GRADMM paper. So from my perspective, this work provides novelty if it gives an approach which we expect could actually perform better than directly training on the human-readable SFT dataset. The authors do report very mild gains over training on human-readable SFT data. However, this result simply doesn't make sense given the objective: the update direction that minimizes their objective is simply the gradient on the original text. So if the synthetic examples successfully optimize their objective, they should learn to imitate the direction that the original dataset already provides directly. Therefore I have a hard time understanding or believing that this approach works any better than direct SFT on human-readable text.
  • Clarity is poor due to heavy use of AI writing. The entire paper reads as heavily AI-written, in a manner that hurts readability. As an example, there are several sentences that are clear hallmarks of AI writing, but human authors would likely never produce, e.g., L107: "the residual expansion analysis motivates this objective but does not guarantee the outcome of finite-step fine-tuning." Which expansion analysis (it has not been mentioned up to this point)? And when would we ever expect theory to guarantee the outcome of multiple steps of finetuning? Similar at L112, where preliminaries/notation that are needed to understand the main equations in the main body are deferred to the Appendix: "The notation, basic setup, and standing assumptions used in this section are given in Appendix A". Another example at L125: "This is an existence result for the expanded model class; it does not assert that a particular dataset or optimizer will find the improving direction." And so on. The whole paper needs significant rewriting to clean things like this up. I see these phrases as going far beyond "polishing" the writing of the human authors, as they disclosed---rather it seems like GPT-6 generated the majority of the words in the paper.
  • There is no motivation provided for why we would intrinsically care about training on representations that are not human-readable. In fact, this seems to be throwing away a highly desirable property for essentially no gain.
Questions for authors

The paper is heavily AI-written and hard to read in its current form. Could the authors take a few passes and remove sentences like the above, and smooth out the writing significantly? This may be too significant of an undertaking for a rebuttal period.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Weakness 1

We study continuous synthetic inputs for adaptation on knowledge, mathematical reasoning, code generation, and commonsense tasks across two model families and six models up to 32B parameters. In the evaluated settings, DASA improves downstream performance over the GRADMM implementation used in our paper and achieves a 3.6–4.9× synthesis speedup under matched settings, with similar peak memory usage.

Another contribution of our paper is to show that when the activation changes induced by synthetic inputs align with the loss descent direction indicated by the reference activation gradients, even continuous inputs that are difficult for humans to interpret can achieve downstream performance comparable to that of real-text fine-tuning on the evaluated tasks.

DASA optimizes inputs using the first-order effect of their induced LoRA updates on the reference loss. We compute a candidate update from the current synthetic inputs, evaluate it using reference activation gradients, and backpropagate this evaluation to the inputs. The resulting continuous inputs are then used for fine-tuning. Our contributions are this input-optimization method, its analysis under LoRA, and the systematic evaluation of its adaptation utility.

Appendix C provides a mean–covariance decomposition of the first-order risk change. Theorem 1 bounds the covariance term. Theorem 2 establishes when continuous input optimization yields a lower reference loss than training on the initialization texts after one update, with the same input count and length, LoRA initialization, and parameter step size.

Conditions and mathematical explanation. Let J(Z)J(Z) be the input objective and Z0Z_0 the initialization embeddings. Theorem 2 assumes:

  • A1: Continuous perturbations are allowed near Z0Z_0, and JJ is differentiable at Z0Z_0.
  • A2: ∇ZJ(Z0)≠0\nabla_ZJ(Z_0)\neq0.
  • A3: The reference-loss gradient with respect to LoRA parameters is β\beta-Lipschitz in an open convex neighborhood U\mathcal U of the common initialization.

For a sufficiently small input step ρ>0\rho>0, setting Zs=Z0−ρ∇ZJ(Z0)Z_s=Z_0-\rho\nabla_ZJ(Z_0) gives:

ΔJ:=J(Z0)−J(Zs)=ρ∥∇ZJ(Z0)∥22+o(ρ)>0.\Delta_J:=J(Z_0)-J(Z_s)=\rho\Vert\nabla_ZJ(Z_0)\Vert_2^2+o(\rho)>0.

Thus, the synthetic inputs induce a smaller directional derivative of the reference loss. Let vs=v(Zs)v_s=v(Z_s) and vr=v(Z0)v_r=v(Z_0). Under A3, when both update paths remain in U\mathcal U:

Lref(fηvs)−Lref(fηvr)≤−ηΔJ+βη22(∥vs∥22+∥vr∥22).L_{\mathrm{ref}}(f_{\eta v_s})-L_{\mathrm{ref}}(f_{\eta v_r})\le-\eta\Delta_J+\frac{\beta\eta^2}{2}\left(\Vert v_s\Vert_2^2+\Vert v_r\Vert_2^2\right).

Consequently, the synthetic inputs yield a strictly lower reference loss whenever:

0<η<ηmax⁡:=2ΔJβ(∥vs∥22+∥vr∥22).0 < \eta < \eta_{\max}:=\frac{2\Delta_J}{\beta\left(\Vert v_s\Vert_2^2+\Vert v_r\Vert_2^2\right)}.

The reason is explicit: input optimization creates a first-order advantage ηΔJ\eta\Delta_J that exceeds the second-order remainder bound. The initialization texts need not induce the best update for the reference objective; optimizing their continuous representations can improve that update and lower the actual loss after one adaptation step.

Experimental verification. We conducted five independent runs on Llama-3.2-1B using FP64. From the same LoRA initialization, we separately applied the updates induced by the initialization texts and synthetic inputs, then recomputed the reference losses through forward passes. Let G=Lr−LsG=L_r-L_s be the measured loss advantage. With a guaranteed local smoothness upper bound β\beta, Theorem 2 gives:

GηΔJ≥1−ηηmax⁡.\frac{G}{\eta\Delta_J}\ge1-\frac{\eta}{\eta_{\max}}.

Table R1. Theoretical lower bounds and measured normalized loss advantages (mean ± SD).

η/ηmax⁡\eta/\eta_{\max}Lower boundG/(ηΔJ)G/(\eta\Delta_J)
0.50.50.999978±0.0000210.999978\pm0.000021
0.90.10.999960±0.0000380.999960\pm0.000038
1.000.999955±0.0000430.999955\pm0.000043
1.1−0.10.999951±0.0000470.999951\pm0.000047

Table R2. Objective improvement, smoothness bound, and actual loss advantage.

StatisticMean ± SD
ΔJ\Delta_J(0.825±0.382)×10−3(0.825\pm0.382)\times10^{-3}
β\beta2.665±1.3752.665\pm1.375
ηmax⁡\eta_{\max}(1.422±0.632)×10−3(1.422\pm0.632)\times10^{-3}
GG at η=2×10−4\eta=2\times10^{-4}(1.649±0.765)×10−7(1.649\pm0.765)\times10^{-7}

At η/ηmax⁡=0.5\eta/\eta_{\max}=0.5 and 0.90.9, which satisfy the strict improvement condition, the measured normalized advantages exceed the theoretical lower bounds and are close to the first-order prediction of 11. At η=2×10−4\eta=2\times10^{-4}, the predicted mean advantage is 1.650×10−71.650\times10^{-7}, close to the measured 1.649×10−71.649\times10^{-7}. These results connect the input-objective improvement to the observed reference-loss advantage after actual updates.

AuthorsOpenReview ↗

Rebuttal by Authors

Weakness 2

English is not our native language, and we used GPT-6 to assist with the English translation and language polishing of parts of the manuscript. We take responsibility for all content and wording in the paper and take the reading difficulties you identified seriously. In response, we have revised the wording throughout the manuscript to help readers understand the method and its theoretical basis.

Weakness 3

We recognize the value of natural text for human inspection and data management. Our paper investigates whether human readability is also necessary for effective adaptation of pretrained models. Therefore, even when synthetic inputs achieve performance comparable to real text, examining whether training utility can be retained after changing the input form has research value.

First, understanding what pretrained large language models use during subsequent learning. Natural text contains both human-readable forms of expression and training signals that models can use. We investigate whether these learning signals can still support effective fine-tuning when the inputs no longer retain fluent natural-language forms. We analyze this question through the parameter updates induced by the inputs and their effects on the reference loss. This helps us understand the relationship between training utility and the surface form of language and explore what forms of information pretrained models can use to continue learning.

Second, exploring fine-tuning approaches for settings involving confidential data. When original texts cannot be provided directly to the training party, one possible approach is for the data holder to optimize synthetic training representations using reference data and then provide these representations for subsequent fine-tuning, so that the training process does not require directly sharing the original reference texts. Our paper provides a method for transforming reference learning signals into continuous training inputs, which can serve as a basis for this direction. Practical applications would also require dedicated privacy protection mechanisms and leakage evaluations.

Third, exploring the utility of expanding small datasets. When real samples are limited, the learning signals provided by a small reference dataset can be used to optimize continuous inputs from different initializations and construct additional training instances. Existing data can therefore be used both for direct fine-tuning and to guide the generation of other training representations. Our method provides a concrete optimization criterion for this purpose: adjusting inputs according to the effects of the updates they induce, and using downstream tasks to assess whether the expanded data can improve adaptation. For experimental evidence on data expansion, please see our response to Reviewer pDgy’s Q1. We report five-run GSM8K experiments that expand 5, 10, 20, or 50 real examples into 100 continuous training inputs, without additional reference data. Under the same number of downstream LoRA steps, DASA achieves mean accuracy gains of 0.11–0.22 percentage points over repeatedly training on the original examples.

Thanks to the authors for their response. I appreciate the effort put into the new theory and experiments, but I remain unconvinced about the paper's overall contribution.

First, while the new theory establishes that optimizing continuous inputs can improve the one-step reference loss relative to the initialization texts, this is a pretty limited guarantee. I don't see this as a meaningful explanation for why the method could result in notably better performance than directly finetuning on natural language SFT data.

Second, I am still unconvinced by the motivations for converting natural language SFT data into continuous representations. The applications mentioned, like privacy-preserving SFT or data augmentation are interesting, but they are speculative, and so I would expect the authors to demonstrate these applications more completely given the notable con of losing interpretability when moving to continuous representations.

Lastly, though the empirical improvements over standard SFT are fairly consistent, they're quite small in magnitude. Given the existing literature demonstrating that continuous synthetic training data works, I don't think these gains are a notable empirical contribution.

Altogether, I appreciate the authors' efforts to address my concerns, but I don't feel that the rebuttal has changed my opinion of the paper, and therefore will maintain my score.

Reviewer_Nij8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Desired-Update-Aligned Synthetic Data (DASA), which optimizes continuous input embeddings so that their induced update directions locally reduce reference loss in a frozen model. The resulting embeddings, paired with the original next token targets, are used for subsequent LoRA fine-tuning. Experiments across Llama and Qwen models show performance comparable to or better than real text adaptation in several settings, while reducing synthesis time relative to the evaluated GRADMM implementation.

Critical strengths
  • DASA constructs synthetic inputs according to their induced updates’ first-order effect on reference loss, providing a concrete criterion for designing training representations beyond linguistic fluency.
  • Experiments span two model families, six models from 1B to 32B, multiple tasks, and both general-domain and task-specialized source data. Matched downstream adaptation settings and additional experiments support the usefulness of the proposed construction.
Critical weaknesses
  • The concrete update-direction analysis relies on a LoRA-parameterized residual branch at initialization, and downstream experiments primarily use only 32 LoRA optimization steps. It remains unclear whether the synthesis objective remains useful as the model moves further from its initial parameters, or whether the benefits extend to longer training and full-parameter fine-tuning. The current evidence therefore establishes effectiveness within a restricted adaptation regime rather than a broadly applicable fine-tuning method.
  • DASA requires 60 synthetic-input optimization steps involving differentiation through model-induced update directions before downstream fine-tuning. Although it is faster than the evaluated GRADMM implementation, this does not establish an advantage over directly training on real text. Given the often modest performance gains, comparisons against longer or better-tuned real-text training under the same total compute budget are needed to justify the additional synthesis cost. In pipelines where SFT primarily serves as a cold-start stage before substantial reinforcement learning, the practical role of DASA remains unclear. The paper does not investigate whether its synthesis approach can support RL or whether its gains persist after subsequent RL training. Given the additional synthesis cost and relatively modest improvements, its overall cost-effectiveness remains unestablished, potentially limiting its applicability in practical settings.
Questions for authors

What practical post-training setting is DASA primarily intended to serve? For pipelines that use SFT as a cold start before substantial RL, do you have evidence that DASA’s benefits persist after RL or improve overall training efficiency enough to offset synthesis costs? If its intended scope is supervised adaptation without subsequent RL, please clarify that scope and the circumstances in which the added cost is justified.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Weakness 1

Our original experiments primarily used 32-step LoRA fine-tuning. On Llama-3.2-1B, we added LoRA runs of 128 and 256 steps and full-parameter fine-tuning runs of 320 and 1,280 steps. Appendix C analyzes the change in reference loss after one update near initialization. During downstream training, the synthetic inputs are fixed, and we recompute parameter gradients at each step using the current model.

SettingMethodMMLUGSM8KMATHMBPPARC-CCSQAMean
0 stepsBase45.7946.3232.7049.654.6159.1348.03
Full-parameter, 320 stepsReal-text47.9948.7534.9051.856.9161.3450.28
Full-parameter, 320 stepsGRADMM47.6948.6035.0052.256.7461.4350.28
Full-parameter, 320 stepsDASA47.8948.8235.5052.657.0061.9250.62
Full-parameter, 1,280 stepsReal-text48.1948.9035.1052.057.0861.5150.46
Full-parameter, 1,280 stepsGRADMM48.0948.9035.2052.257.0861.6750.52
Full-parameter, 1,280 stepsDASA48.1949.0535.4052.457.1761.8350.67
LoRA, 128 stepsReal-text47.7948.4534.7051.656.7461.0250.05
LoRA, 128 stepsGRADMM47.5948.2934.8051.856.4861.2650.04
LoRA, 128 stepsDASA47.6948.5235.2052.256.8361.5150.32
LoRA, 256 stepsReal-text47.5948.2234.4051.456.4060.8549.81
LoRA, 256 stepsGRADMM47.3948.1434.5051.656.4060.9349.83
LoRA, 256 stepsDASA47.4948.2934.8051.856.4861.1049.99

DASA has the highest six-task mean in all four settings, leading Real-text by 0.34, 0.21, 0.27, and 0.18 percentage points, respectively. It outperforms our GRADMM implementation in all 24 task–setting combinations. Against Real-text, DASA scores higher in 20 combinations, ties in one, and scores lower in three. The mean score declines from 128 to 256 LoRA steps for all three methods, and DASA retains its lead.

Weakness 2

Table 4 compares synthesis times for DASA and GRADMM. We also compare DASA with Real-text on Llama-3.2-1B under matched end-to-end compute budgets, including reference signal precomputation, synthesis, and fine-tuning.

Compute (PFLOP)MethodMMLUGSM8KMATHMBPPARC-CCSQAMean
1,800Real-text48.9749.5135.8252.857.8561.9251.14
1,800DASA48.0449.5836.5853.857.3462.5751.32
3,590Real-text48.8549.6635.8252.257.8562.0051.06
3,590DASA47.9749.5136.3854.057.4262.5751.31
5,390Real-text48.7849.5135.6252.857.7662.0051.08
5,390DASA47.9649.9636.6653.857.1762.2451.30

DASA leads by 0.18, 0.25, and 0.22 percentage points on the six-task mean, respectively. At all three budgets, DASA scores higher on MATH, MBPP, and CSQA, while Real-text scores higher on MMLU and ARC-C.

Question 1

We construct continuous inputs from reference data for SFT of pretrained models, including the SFT stage before RL. We evaluated both SFT → GRPO pipelines on GSM8K using Llama-3.2-1B.

Training pipelineShared baseAfter SFTAfter 400 GRPO steps
Real-text SFT → GRPO46.32%47.54%59.29%
DASA SFT → GRPO46.32%48.52%60.96%

GRPO improves the DASA checkpoint from 48.52% to 60.96%. After 400 GRPO steps, DASA leads Real-text by 1.67 percentage points, or 22 more correct answers out of 1,319.

We then matched end-to-end compute, including reference signal precomputation, synthesis, SFT, and GRPO rollouts, reward computation, and policy updates.

Training pipelineShared baseAfter SFT (matched compute)After GRPO (matched total compute)
Real-text SFT → GRPO46.32%48.90%61.33%
DASA SFT → GRPO46.32%49.36%61.56%

DASA's lead narrows from 0.46 percentage points after SFT to 0.23 after GRPO.

Thank you for the additional experiments. The longer training runs and full-parameter results partially address my concern about the restricted adaptation setting, and I appreciate the comparisons under matched total compute.

However, the practical benefit in an SFT → RL pipeline remains insufficiently established. The additional evaluation covers only Llama-3.2-1B on GSM8K, and the advantage after GRPO decreases to 0.23 percentage points under matched total compute. Without evidence across additional tasks and models, or uncertainty estimates, this result is not sufficiently convincing to establish a reliable practical advantage.

These additions are useful, but the modest gains and limited pipeline evaluation leave my main concerns unresolved. I therefore maintain my original score.

AuthorsOpenReview ↗

Response to the Reviewer

We sincerely thank the reviewer for the time and care devoted to evaluating our paper and for the constructive follow-up. This paper asks whether fine-tuning inputs must be human-readable text. Our claims concern supervised adaptation of a pretrained model at a fixed sample budget. We make no claims about pipelines in which SFT is followed by reinforcement learning, and RL does not appear in the original manuscript’s problem statement, method, analysis, or experiments. Whether gains at the SFT stage carry through subsequent RL is a separate research question involving additional factors, such as reward design, rollout budget, and the RL algorithm, and requires a dedicated study to answer fully.

At the same time, we thank the reviewer for raising this question and agree that examining how DASA connects to subsequent RL helps clarify its scope of applicability. We conducted additional SFT → GRPO experiments with two models across four benchmarks. For each model–benchmark setting, we compared Real-text SFT → GRPO and DASA SFT → GRPO under the same computational budget. Each setting was evaluated over three independent training runs, and the table reports the averages over these three runs.

ModelBenchmarkReal-text SFT → GRPODASA SFT → GRPODifference
Qwen3-8BGPQA Diamond63.64 ± 0.5164.65 ± 0.84+1.01
Qwen3-8BLiveCodeBench61.02 ± 0.5561.75 ± 0.77+0.73
Llama-3.2-3BMATH58.02 ± 0.4258.61 ± 0.18+0.59
Llama-3.2-3BMBPP72.98 ± 0.8974.24 ± 0.28+1.26

Across these four settings, DASA SFT → GRPO achieves slightly higher final scores than Real-text SFT → GRPO, with differences ranging from 0.59 to 1.26 points. These results provide preliminary evidence that DASA-adapted models can serve as effective starting points for subsequent GRPO and show that they retain performance comparable to that of models adapted on real text within the evaluated RL pipelines.

We thank the reviewer again for the constructive feedback, which motivated these additional experiments and helped us clarify the paper’s scope. We hope that the clarifications and new results address your concerns about DASA’s applicability to subsequent RL. If you find that your main concerns have been adequately addressed, we would be grateful if you could reconsider your assessment in light of this additional evidence and consider revising your rating.

Reviewer_pDgyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether LLM fine-tuning requires human-readable text and proposes DASA. The method initializes ZZ from the input embeddings of sampled reference token sequences and uses a frozen model to compute reference activations and activation gradients. Then constructs a parameter-space update direction induced by the current synthetic inputs and optimizes ZZ to minimize the first-order change in reference loss along this direction. After TsynT_{\mathrm{syn}} synthesis steps, the resulting continuous embeddings are used directly for downstream LoRA adaptation. The experiments compare DASA with real-text fine-tuning and GRADMM across six Llama and Qwen models, six benchmarks, and three data budgets. The results show that the performance of DASA is comparable to or better than real-text fine-tuning and better than GRADMM in most evaluated settings. Under the reported synthesis settings, DASA generates data 3.6–4.9 times faster than GRADMM with comparable peak GPU memory.

Critical strengths
  1. The experiments in this paper are well organized, covering models from different families, a variety of sizes, with both general-domain and task-specialized data. The paper also conducts experiments on settings across random seeds, reports synthesis time and peak GPU memory, and studies sensitivity to LoRA rank and adaptation steps, covering most needed experiments.
  2. The method is motivated by first-order objective. The paper explains how synthetic inputs induce a local update direction, states the relevant assumptions, and proves a covariance bound for the specified LoRA residual branch in Appendix C. It also distinguishes this local analysis from a guarantee for finite-step fine-tuning, which makes the scope of the theoretical argument clear.
Critical weaknesses
  1. The paper raises a broad question that whether human-readable text is necessary for LLM fine-tuning, so the comparison with Real-text is rather important. Although adding algorithmic complexity, DASA is competitive with Real-text in many settings, but the advantage is inconsistent and not strong enough. This raises questions about the necessity of this synthetic pipeline compared to simply using the original natural-language data.
  2. The paper adapts GRADMM to match gradients across all layers and fine-tune directly on continuous embeddings, but the published method primarily matches last-layer gradients and uses vocabulary projection during generation. This provides a comparison under matched settings, but the reported performance may not reflect a comparison with published GRADMM configuration.
Questions for authors
  1. How consistent are DASA’s gains over Real-text across tasks and random seeds? Could the authors report uncertainty or additional runs for the settings where the differences are small?
  2. The evaluated GRADMM differs from the original method, which matches only last-layer gradients and uses vocabulary projection. Could the authors also report results using the original GRADMM configuration?
2 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Weakness 1

Please see our responses to Reviewer Eza2’s Weaknesses 1 and 3 for the theoretical explanation of when synthetic inputs can achieve a lower one-step reference loss than their initialization texts, and for the scientific motivation and potential applications of this work.

Q1: Gains over Real-text and their stability

Thank you for emphasizing the comparison with Real-text. Table 2 of the original manuscript reports means and standard deviations over three independent runs on Qwen3-8B and Qwen3-14B under the 200×2048200\times2048 budget:

ModelTaskReal-textDASA
Qwen3-8BMMLU76.97±0.1376.97\pm0.1378.09±0.1678.09\pm0.16
Qwen3-8BGSM8K93.93±0.2193.93\pm0.2193.78±0.3393.78\pm0.33
Qwen3-8BMBPP61.8±0.661.8\pm0.663.0±0.663.0\pm0.6
Qwen3-14BMMLU80.89±0.1780.89\pm0.1781.15±0.2981.15\pm0.29
Qwen3-14BGSM8K95.91±0.2295.91\pm0.2297.12±0.2397.12\pm0.23
Qwen3-14BMBPP66.8±0.466.8\pm0.466.6±0.666.6\pm0.6

DASA achieves higher means in four configurations, while Real-text performs better in two. We therefore describe the gains as task-dependent: continuous synthetic inputs can achieve comparable adaptation performance and improve some model–task configurations. We report uncertainty for small differences without claiming statistical significance.

We also added a five-run data expansion experiment on GSM8K using Llama-3.2-1B-Instruct. We use 5, 10, 20, or 50 training questions and their solutions to construct 100 continuous inputs. Both the reference objective and initialization use only these samples, with no additional reference texts. Real-text repeatedly uses the original samples and receives the same number of downstream LoRA steps as DASA.

Method5 → 10010 → 10020 → 10050 → 100
Base46.3246.3246.3246.32
Real-text46.06±0.2946.06\pm0.2946.56±0.2846.56\pm0.2846.42±0.2046.42\pm0.2046.92±0.2646.92\pm0.26
GRADMM45.80±0.2945.80\pm0.2946.71±0.2946.71\pm0.2946.40±0.1646.40\pm0.1646.83±0.2246.83\pm0.22
DASA46.22±0.3046.22\pm0.3046.68±0.2846.68\pm0.2846.64±0.1646.64\pm0.1647.03±0.1747.03\pm0.17

DASA has slightly higher mean accuracy than Real-text at all four sample budgets. With five samples, all methods remain below Base; with ten, GRADMM has the highest mean. These results illustrate a concrete use: a small set of real examples can guide the construction of additional continuous inputs, yielding modest average gains under matched downstream training steps. This new five-run experiment concerns data expansion; the repeated-run evidence for the main settings remains the three runs in Table 2.

To address synthesis cost, we also added matched total-compute comparisons on Llama-3.2-1B, including reference precomputation, synthesis, and fine-tuning. DASA's six-task average exceeds Real-text by 0.18, 0.25, and 0.22 percentage points at the three budgets. Full results are provided in our response to Reviewer Nij8.

Q2: Comparison with the original GRADMM configuration

Following your suggestion, we added the original GRADMM configuration with last-layer gradient matching and vocabulary projection. Our manuscript's implementation uses all-layer matching and continuous inputs without vocabulary projection. The Llama-3.2-1B results are:

MethodMMLUGSM8KMATHMBPPARC-CCSQA
Base45.7946.3232.7049.654.6159.13
Real-text48.0148.6734.8851.856.9161.26
GRADMM (all-layer, continuous)47.4348.4535.2052.456.5761.59
GRADMM (original)47.1948.1434.8252.056.3161.26
DASA47.8848.9036.1253.457.1762.49

DASA outperforms the original GRADMM configuration on all six tasks by 0.69–1.40 percentage points and also outperforms our all-layer continuous implementation. In this setting, the all-layer implementation scores higher than the original configuration on all six tasks.

Thanks to the response. Most of my concerns have been addressed. I will retain my current score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.