acceptodds
Under review as a conference paper at ICLR 2027

Beyond Full-Model Rollback: AuroSFT for Adapter-State Multi-Task Fine-Tuning

Abstract

Multi-task supervised fine-tuning (SFT) often casts a heterogeneous data mix-ture as a single optimization problem, even though different tasks may reach their best generalization at different times. MSFT is a scheduler that exposes this mismatch through task-wise roll-out, exclusion, and rollback. However, it materializes its scheduler state as full-model checkpoints, which makes stage transitions costly to store, restore, and deploy. This paper introduces AuroSFT, a parameter-efficient framework that recasts the carried state of overfitting-aware multi-taskSFT as a compact, mergeable adapter state. AuroSFT freezes the pretrained backbone, trains only injected adapters, and rolls back adapter checkpoints at task-wise peaks. At the layer level, each adapter applies an AuroRA-inspired adap-tive nonlinear layer (ANL) to a low-rank weight factor rather than to the sample representation. The resulting update remains linear in the input, rank-bounded, and exactly mergeable into the frozen projection. On the same ten benchmarks and five backbones as the MSFT reference, AuroSFT achieves 61.36% average accuracy, compared with 59.85% for the full-model reference and 61.28%for a scheduler-matched linear-LoRA baseline. Its accuracy exceeds the full-model reference on all five backbones. A stage-log diagnosis further traces themathematics-group regression to interference that the eviction rule misreads as overfitting, and an interference-aware repair recovers this group above the full-model reference on Qwen2.5-3B. Our code is available at the anonymous repository: https://anonymous.4open.science/r/AuroSFT-80D1.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Supervised Fine-Tuning, Low-Rank Adaptation, Large Language Models, Knowledge Inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6XMwOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AuroSFT, a parameter-efficient multi-task SFT framework that reduces the overhead of full-model rollback. Building on MSFT, it replaces full-model checkpoints with lightweight adapter states and introduces nonlinear transformations to adapter weights. The proposed framework demonstrates improvement across a wide range of task settings.

Critical strengths
  1. Broad coverage of domains and tasks. The experiments cover multiple tasks across different domains, providing a relatively comprehensive empirical evaluation of the proposed method.
  2. Practical significance. Introducing PEFT into the multi-task fine-tuning rollback framework substantially reduces the number of parameters that need to be stored and restored, improving storage efficiency.
Critical weaknesses
  1. Limited novelty. The proposed method primarily replaces full-parameter fine-tuning in MSFT with AuroRA-based parameter-efficient fine-tuning, with the main modification being the application of the nonlinear ANL to adapter weights rather than projected sample representations.
  2. Marginal improvements. AuroSFT achieves only 0.08% improvement over the straightforward combination of LoRA and MSFT. Moreover, its improvements over MSFT become marginal for models larger than 1.5B parameters.
  3. Additional ablation studies are needed to evaluate the benefits of modification on the nonlinear ANL in AuroRA.
Questions for authors
  1. Clarify the contributions and novelty.
12 comments
AuthorsOpenReview ↗

Response to Reviewer 6XMw — Overview and index

Response to Reviewer 6XMw

We thank the reviewer for critically assessing the novelty and empirical scope of our work. The comments prompt us to clarify which components are inherited, which research questions arise only when those components interact, and where our method does or does not provide measurable benefits. They also encourage us to strengthen the ablation evidence and to report the limits of scalability with the same prominence as the positive results.

Throughout this response, accuracies are percentages and differences are percentage points. Overall accuracy is the unweighted mean of the ten benchmark accuracies; mathematics is the mean of GSM8K and AQUA. Submitted results, fresh experiments, structure-matched adapter controls, and trajectory analyses are identified separately. New results and reporting corrections are incorporated into the revised manuscript; section and table references below refer to that revision.

The revised interpretation separates three objects: adapter-only rollback determines which state is restored; F2 determines which task-group factors remain mutable; ANL determines how the update is parameterized and optimized. F1 is the eviction-rule intervention, F1ctl its fixed non-victim protection control, F2linear the linear version of the protected structure, and F3 the stagnation-based termination rule. None of these names implies a general guarantee of improved accuracy.

The evidence freeze excludes the budget-matched full-model mSFT control and additional fresh F2/F2linear training-seed lines that have not yet passed completion and provenance review. No intermediate value from these lines is used below. A result will be added only after its actual training seed, stage chain, final checkpoint, and same-checkpoint evaluation records agree; if a line fails, stops, or remains unverifiable, we will report that status rather than leave an unfulfilled experimental claim.

Reading guide

This response is divided into threaded comments to comply with the 5,000-character limit. No substantive response content is omitted. Each W/Q thread begins with the direct answer; continuation comments retain the manuscript locations, complete tables, derivations, and revised-manuscript excerpts in their original order.

There are 4 W/Q threads and 9 W/Q comments in total:

  • W1 — 2 comments: W1. Limited novelty beyond mSFT and AuroRA-based parameter-efficient fine-tuning
  • W2 — 2 comments: W2. Marginal improvement over LoRA and limited gains for models larger than 1.5B
  • W3 — 3 comments: W3. Additional ablations are needed to evaluate the nonlinear ANL modification
  • Q1 — 2 comments: Q1. Clarification of the contributions and novelty

Continuation titles use the form [part/total]. Please read each numbered thread in ascending part order.

AuthorsOpenReview ↗

W1. Limited novelty beyond mSFT and AuroRA-based parameter-efficient fine-tuning [1/2]

W1. Limited novelty beyond mSFT and AuroRA-based parameter-efficient fine-tuning [1/2]

Concern. The method primarily replaces full-parameter fine-tuning in mSFT with AuroRA-based parameter-efficient fine-tuning, and its main modification appears to be applying ANL to adapter weights rather than projected sample representations.

We appreciate this criticism because it helps us identify an overbroad presentation of novelty in the original manuscript. Our direct response is that the contribution is the formulation, diagnosis, and repair of scheduler-carried adapter state. We study the adapter as a time-dependent object under eviction and rollback, measure post-exclusion erosion, introduce group-protected state transitions, and evaluate structure-matched and negative controls.

The inherited components are not counted as new contributions. mSFT supplies the task scheduler and AuroRA-S already provides a closely related static weight-side adapter. We do not claim the scheduler, the static adapter equation, input linearity, rank boundedness, or exact mergeability as independent new contributions. The research question begins with the state transitions that those inherited properties do not address.

A direct comparison with prior work clarifies the resulting research focus:

WorkDynamic task-wise schedulingCompact carried stateTask-peak rollbackPost-eviction state-erosion analysisGroup-protected rollback
mSFTYesNo; full-model stateYesNot its main research objectNo
Standard LoRANoYesNot its main research objectNot its main research objectNo
AuroRA-SNoYesNot its main research objectNot its main research objectNo
Our revised focusInherited from mSFTAdapter stateAdapter-only, stage-dependentYesYes, through F2

The research challenge begins after the components are combined. A static adapter under a fixed schedule only needs to represent the current model update. An adapter inside mSFT must instead represent a sequence of stage-dependent states. The scheduler evicts tasks according to validation peaks, selects a checkpoint associated with one task, and continues training on the surviving mixture. This creates difficulties that cannot be solved by storage compression alone:

  1. Peak interpretation: an epoch-0 peak followed by degradation can indicate cross-task interference rather than genuine overfitting.
  2. Zero-gradient erosion: an excluded task receives no further gradient, but surviving tasks continue modifying the shared low-rank factors on which its predictions depend.
  3. Rollback mismatch: a checkpoint selected for one task may not be the best state for another task.
  4. Unbounded stagnation: later stages can consume substantial stage budget without improving the global-best checkpoint.

At the theoretical level, the submitted propositions establish state sufficiency and an accounting framework. The state-sufficiency argument shows that a frozen backbone makes the adapter state and metadata sufficient to reconstruct a stage projection. The peak-to-final decomposition then separates task-peak gain from subsequent stage-transfer change, providing a way to measure whether the peak gain survives. Neither proposition by itself solves the interaction failures above.

Direct mechanistic analysis motivates the new state-aware repairs. F2 introduces block-diagonal task-group adapters, freezes a group after all its tasks have been evicted, and rolls back only active groups. F3 addresses stagnation by stopping after two consecutive stages without global-best improvement or after two whole-stage rollbacks to epoch 0. F1 changes the interpretation of epoch-0 peaks, although our attribution control shows that its measured benefit should not be attributed specifically to correct victim identification.

Direct empirical evidence supports a nuanced contribution rather than a universal new adapter claim. F2 improves the fresh Qwen2.5-3B base by 0.60 overall points and 1.25 mathematics points. F2+F3 improves the OLMo2-1B base by 4.10 overall points and 4.25 mathematics points. Conversely, repairs underperform the base in several smaller-backbone configurations. This combination of positive and negative findings helps identify when protected state is useful and when rank splitting or early termination costs accuracy.

AuthorsOpenReview ↗

W1. Limited novelty beyond mSFT and AuroRA-based parameter-efficient fine-tuning [2/2]

W1. Limited novelty beyond mSFT and AuroRA-based parameter-efficient fine-tuning — continued [2/2]

Complementary auxiliary evidence comes from the submitted mixture-scale and MedMCQA experiments. AuroSFT improves over fixed-mixture SFT by 11.9, 0.7, and 0.6 points at mixture sizes 5, 10, and 15, respectively. The MedMCQA analysis reports a 1.25-point overall improvement over SFT; in its 11-group display, AuroSFT is no worse than reproduced mSFT on 10 groups and strictly better on 8. The reported overall improvement follows the original aggregation rather than an unweighted mean of these 11 displayed bars. These results support the broader value of adapter-state continuation, but we do not use them to claim that every component is uniformly beneficial.

We separate inherited properties from the proposed investigation. mSFT supplies the task-wise schedule; LoRA supplies compact low-rank adaptation; AuroRA-S already supplies the closely related static weight-side nonlinear form. Input linearity, rank boundedness, and mergeability are not counted as new mathematical contributions. The revised method explanation uses these inherited properties to make stage-state reconstruction precise, then studies subsequent transitions that those properties do not protect.

The proposed structure is not an unconditional no-forgetting guarantee. F2 separately parameterizes task groups, freezes a group after its last task is evicted, and later rollback restores only the active groups. This protects the saved factors themselves; changing active blocks elsewhere in the same forward network can still alter an excluded task's predictions. Similarly, an epoch-0 peak is a diagnostic signature compatible with interference rather than proof of its unique cause. The revised contribution is an empirical and structural investigation of this interaction, with these qualifications stated explicitly.

Group separation does not mean task-conditional computation. The implemented forward pass sums all group updates for every input, so an active task can update every unfrozen group. The block-diagonal description concerns the latent parameter-state organization, not separate input/output coordinates or per-task routing. Explicit freezing and selective rollback protect a fully evicted group's factors; no structural claim of zero cross-task influence is made for the complete network.

Changes in the revised manuscript. Section 1, “Introduction”; Section 2, “Related Work”; Section 4.5, “Diagnosis: Where the Shared-Adapter Schedule Loses Mathematical Tasks”; Section 5, “Discussion and Limitations”; Appendix “Rebuttal Evidence and Reporting Corrections,” subsection “State Protection and Audited Replication.”

Revised manuscript text.

The static weight-side adapter used here has essentially the same functional form as AuroRA-S. Input-linearity, rank boundedness, and exact mergeability are inherited properties of that static family, rather than independent novelty claims. Our dynamic setting asks when its state should be saved, which blocks should be restored, and how retained-task accuracy changes after exclusion.

Compact state does not guarantee retained knowledge. A task with an epoch-0 peak can be excluded even when its immediate decline reflects interference rather than ordinary overfitting. Once excluded, it receives no gradient, but surviving tasks keep changing the shared adapter. A rollback chosen for one task can also discard useful progress for another. Our contribution is to measure these interactions and evaluate state-aware repairs, with controls that separate state restoration, protected structure, and nonlinear optimization.

Group factors are separately parameterized; every input uses their sum. All unfrozen blocks can receive gradients from active tasks. A group is frozen once all its assigned tasks are excluded, and rollback restores only still-active groups.

Reviewer_jFGTOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a parameter-efficient framework (named AuroSFT) for multi-task supervised fine-tuning, with the motivation of memory-efficient overfitting-awareness. Building upon the existing mSFT technique, a method that excludes tasks by validation peaks and rolls back to earlier checkpoints, AuroSFT replaces full-model rollback with LoRA-style adapters to save memory. It further introduces a weight-side adaptive nonlinear layer (ANL), borrowing the idea from AuroRA, to preserve input linearity and weight mergeability. Experimental results support average multi-task performance gains on various benchmarks across several backbones.

Critical strengths
  1. From the perspective of memory, the method effectively saves checkpoint storage during the rollback and restoring process, while keeping the final average multi-task performace.

  2. The authors provide some theoretical analysis to well claim some insights, e.g., the proposed ANL can preserve the low-rank structure and the exact mergability of LoRA.

Critical weaknesses
  1. This paper combines two ideas from existing techniques, i.e., mSFT and AuroRA. While it has some practical impact on memory savings, I still suspect that it is only an engineering improvement rather than fundamental insights on multi-task learning or PEFT. Also, the technical innovations are really limited.

  2. The two components, i.e., adapter-only rollback and ANL seems to have little connections. In other words, the relationship between them is insufficiently motivated. The readers may struggle to comprehend why ANL is significantly valuable to the main idea of memory-efficient multi-task learning via adapter-only rollback.

Questions for authors
  1. Do the authors have any insight on whether ANL can mitigate the forgetting of the early-exiting/excluded tasks? The exclusion of tasks can alleviate the risk of over-training or over-fitting, but it also brings or improves the risk of forgetting task-specific knowledge of the early-excluded tasks, especially when some tasks may conflict with each other.

  2. Why can the method with standard LoRA (i.e, mSFT + standard LoRA) have similar performance with AuroSFT? The improvement is only 0.08%0.08\%, which is quite incremental.

Reviewer_iXUeOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
3(+1)
Presentation
2
Soundness
3
Read review
Paper summary

This work studies how to reduce checkpoint storage in multi-task supervised fine-tuning. AuroSFT uses the MSFT schedule with a frozen backbone and small adapters. It applies a nonlinear layer to a low-rank weight factor, so the trained update can be merged into the backbone. The paper evaluates five backbones on ten benchmarks. It also shows how the task-removal rule can degrade the mathematics performance.

Critical strengths
  1. This work addresses a practical problem. The proposed approach is also sound.
  2. The experimental coverage is broad, including five backbones and ten benchmarks, with evaluations on task mixtures and scheduling selections.
  3. The authors report the experimental setup clearly, which ensures reproducibility.
Critical weaknesses
  1. I truly appreciate the authors for concluding their work honestly and for not "overclaiming" it. However, it makes the contribution sloppy and unhighlighted. In particular, the scheduler comes from MSFT. The small and mergeable updates are already a main benefit of LoRA. More importantly, AuroRA includes a static version as AuroRA-S. This is close to the weight-side adapter used in this work. The paper discusses the input-level version of AuroRA but does not compare the difference from this version. The authors should make clear what changes are beyond.
  2. The reconstruction score comparison is unclear. Table 1 states that some group scores are reconstructed from the overall average, including the MSFT. The paper then uses this number to claim that the repaired method exceeds full-model MSFT. However, an overall average does not determine the mathematics score.
  3. The presentation is sometimes hard to follow and needs improvement. For example, the motivation of the whole design is not highlighted. The main table mixes results from multiple sources, including the source paper, reproduced results, and reconstructed scores. In addition, there is a mismatch of 3B MSFT score (73.10 in Table 1, but 73.00 in Tables 12 and 16).
Questions for authors
  1. How does the weight-side adapter differ from AuroRA-S?
  2. Can the authors provide and compare the measured Qwen2.5-3B group scores, especially the full-model MSFT mathematics score?
1 comment

I appreciate the reviewers' effort on the rebuttal. It seems that the updated manuscript has a different font class from the standard conference template. Please check carefully before submitting the latest version.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Manuscript Revisions and Responses to All Reviewers

We sincerely thank Reviewers iXUe, jFGT, and 6XMw for their careful assessments and constructive questions. Their comments prompted substantive revisions to our contribution positioning, experimental audit, controls, and limitations. We have updated the manuscript, abstract, and Supplementary Material. This comment summarizes these changes and guides readers to the detailed threads.

1. Contribution positioning and conceptual clarification. We now explicitly state that the scheduler is inherited from mSFT and that our static weight-side nonlinear adapter has essentially the same functional form as AuroRA-S. Input linearity, rank boundedness, and exact mergeability are not independent novelty claims. Our research object is the adapter as dynamic scheduler-carried state: what is restored, which factors remain mutable after exclusion, and how excluded-task accuracy changes during continued optimization. We distinguish adapter-only rollback (state restored), F2 group protection (mutable factors), and ANL (update parameterization). These changes appear in the Introduction, Related Work, Method, and Discussion and Limitations.

2. Corrections to empirical reporting and checkpoint provenance. We removed reconstructed task-group entries from the main comparison rather than inferring group scores from overall averages. Historical references, original component studies, full-model reproductions, and fresh H20 runs are labeled as distinct evidence families, not interchangeable replicates. The fresh-base result uses the completed pipeline's final selection, 73.30 overall and 61.00 mathematics, rather than the superseded Stage-10-entry tracker. Group values come from the same selected checkpoint; unresolved records are marked Audit pending and excluded from conclusions.

3. New direct measurements, replications, and controls. On fresh Qwen2.5-3B seed-20 runs, full-model mSFT obtains 73.60 overall and 61.50 mathematics; the AuroSFT base obtains 73.30/61.00, F2 obtains 73.90/62.25, and F2linear obtains 73.00/58.00. The full-model reference uses ten epochs per stage with a constant schedule, whereas adapters use two epochs with cosine scheduling. This is therefore a direct reference comparison, not an equal-budget causal comparison. Two audited base/F2 seed pairs improve overall/mathematics by +0.60/+1.25 and +1.05/+2.25 points; this is directional evidence, not statistical significance. F2/F2linear controls, post-exclusion trajectories, non-victim controls, and negative cross-backbone results are included. Retention evidence favors protected structure rather than ANL; frozen factors do not imply zero whole-network forgetting.

4. Explicit empirical boundaries. The five-backbone average difference from scheduler-matched LoRA remains only 0.08 points, with LoRA slightly better on three backbones, and the 7B margin over the full-model reference is only 0.05 points. We make no uniform-accuracy or increasing-scalability claim. Repairs that trail the base, controls that challenge our initial interpretations, and termination sensitivity are reported alongside positive findings. Unfinished or unverified runs are excluded. The Supplementary Material includes the revised evidence appendix and key logs, selected-checkpoint records, configurations, and integrity manifests.

Guide to the reviewer-specific responses.

  • Reviewer iXUe (10 W/Q comments): contribution beyond mSFT, LoRA, and AuroRA-S; withdrawal of reconstructed-score evidence; directly measured Qwen2.5-3B group results; and corrections to table provenance, motivation, and numerical presentation.
  • Reviewer jFGT (9 W/Q comments): the distinct roles of rollback, protected structure, and ANL; post-exclusion retention and its limits; and why the 0.08-point LoRA difference is compatible with configuration-dependent optimization effects.
  • Reviewer 6XMw (9 W/Q comments): contribution and novelty; limited margins and scalability; additional ANL ablations, structure-matched controls, audited replications, and negative results.

The revised claim is consequently narrower and better supported: compact adapter rollback is the supported resource benefit, whereas accuracy and retention effects are configuration dependent and must be interpreted with their provenance, budgets, and uncertainty. We thank the reviewers and the Area Chair for the time and care devoted to assessing this revision.

Area_Chair_NLhZOpenReview ↗

Response to the rebuttal

Dear reviewers,

The authors have submitted rebuttal responses. Please read through their replies and update your review as you see fit. Thank you for your work.

AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.