- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The authors propose TALE, a framework for memory-efficient model adaptation on edge devices that constructs a compressed emulator to preserve the original model’s training gradients. It connects gradient mismatch to divergence between LoRA adapters trained on the emulator and the full model, then derives a compression procedure using input and output-gradient statistics, weighted SVD, and adaptive layer-wise regularization. Adapters trained on the emulator can be transferred directly to the full model without post-hoc correction. Experiments on visual question answering, language-model personalization, and image generation demonstrate improvements over EMLoC and robustness to differences between calibration and downstream data.
Critical strengths
- Empirical breadth: the experiments cover visual question answering, language-model personalization, and image generation.
- Robustness in task customization: EMLoC suffers a substantial performance drop under this distribution mismatch, with its average accuracy decreasing from 53.3 to 48.4. In contrast, TALE maintains highly comparable performance across both paradigms, achieving 55.9 and 55.3 respectively, and notably outperforms EMLoC by 6.9 points in the server-side setting.
Critical weaknesses
-
For the baseline presented in the paper - the results differ very incrementally, and without std values I’m not sure if the differences are significant. “Consistently outperforms” is too strong.
-
The paper briefly cites OBD-LLM which already derives the same construction: use input and output-gradient statistics to weight both sides of a weight matrix, truncate its SVD, then undo the transformations. Its objective and resulting factors match TALE’s construction, up to notation and the choice of matrix square-root factors. Moreover, preserving adapter gradients when constructing an emulator is already the explicit motivation of GradOT. The combination of these two prior directions makes the broad conceptual and algorithmic novelty claims difficult to accept without a direct comparison.
-
The theoretical analysis provides a local, single-layer justification under simplifying assumptions, but does not directly establish gradient preservation when many layers are compressed simultaneously.
Questions for authors
- If I understand correctly, Figure 1 reports correlations between output alignment and gradient alignment. However, how does TALE reduce absolute gradient error, or does it preserve gradients throughout adaptation?
5 comments
Response to Rebuttal
Thanks for your response.
-
Comparison to the OBD-LLM and GradOT:
- the rebuttal result looks like authors selected the datasets on which the results are most promissing, otherwise - how is this selection is motivated? I think that the authors should extend the main Table with full results from these two relevant methods.
- The authors attempt to distinguish TALE by saying that OBD-LLM focuses on inference compression while TALE focuses on gradient preservation during adaptation. But Theorem 1 (Eq. 4) and Corollary 1 (Eqs. 5 - 6) appear to recover essentially the same weighted low-rank approximation objective and closed form solution already derived in OBD-LLM (Section 4.2, Eqs. 9–13). Both methods use input and output-gradient covariance statistics to construct a doubly transformed weight matrix, apply truncated SVD, and invert the transformations to obtain the compressed weights.
-
Accuracy gains over EMLoC at 50% compression: again I dont see all datasets. In addition the most important configuration is 25% + QLora which combines aggressive memory reduction with quantization, a natural setting for resource-constrained adaptation. TALE does provide improvements with fixed regularization, but their magnitude is limited, and the proposed adaptive regularization fails to improve the overall result in this configuration. This provides a concrete counterexample to the manuscript's broad claims of consistent improvement.
-
Regarding your last point:
- I think there is a type in the response since I don't see Appendix G in the paper.
- My question was not answered, I asked about absolute gradient error, not correlation.
- according to the results in the new plot (Figure 5), although these values are consistently higher than EMLoC's, the gradients become nearly orthogonal to those of the full model after the early adaptation stage. This is a substantial limitation for a method claiming to faithfully preserve training signals.
-
Additional points I would like to discuss after an additional review I've made:
-
On-device feasibility is not convincing (Appendix C, Table 6). The method has approximately the emulator construction compute of EMLoC and provides no additional peak-memory reduction during adaptation.
-
The paper claims robust generalization to unseen tasks and users. However, the task-generalization experiments are conducted across four question categories within the same VQA-Med dataset. This is a relatively restricted setting compared with genuinely unseen datasets or substantially different domains. Moreover, in Table 2, the Abnormality task has extremely low accuracy for every method, including full-model fine-tuning (6.2%). Therefore, that task provides little meaningful evidence of successful adaptation.
-
Overall, I believe the paper requires substantial revision, including more comprehensive empirical comparisons with the most closely related methods, a more carefully narrowed theoretical contribution, and a detailed discussion of the methodological similarities and differences between TALE, OBD-LLM, and GradOT.
Response to Reviewer 9cwY (1/3)
Dear Reviewer 9cwY,
Thank you for the careful second-round review. Your summary names three revisions the paper needs: (i) comprehensive comparisons with OBD-LLM and GradOT, (ii) a more carefully narrowed theoretical contribution, and (iii) a detailed discussion of how TALE, OBD-LLM, and GradOT relate. The revised manuscript makes all three. This response also adds the evidence that our first response lacked:
- Full baseline comparison: On all seven VQA benchmarks, TALE outperforms GradOT on all seven benchmarks and OBD-LLM (Table E).
- 25% + QLoRA: With dataset-specific λ, adaptive regularization becomes the best TALE variant and outperforms EMLoC on average (Table F).
- Absolute gradient error: TALE has lower error than EMLoC in 23 of 24 LLM blocks (Table G, Appendix F).
- On-device cost: Emulator construction adds 3.5% total compute over EMLoC and stays within inference-level memory.
- Distribution shift: When calibration and fine-tuning data differ, TALE retains its accuracy while EMLoC degrades (e.g., WebSRC 84.5 vs. 76.9; Table H).
Below, we answer each remaining point directly, including those on which we agree with you.
Comparison with Gradient-Based Baselines and Clarification of TALE’s Contributions
- Full Comparison with GradOT and OBD-LLM across VQA Benchmarks
We apologize for the partial comparison in our first response. Given the time and resource constraints during the rebuttal period, we initially reported results on datasets with completed experiments. We now provide additional results, which reinforce our original conclusions. Our method outperforms GradOT across all evaluated datasets, with gains of up to 12%p despite more aggressive compression, and surpasses OBD-LLM on six out of seven benchmarks, with particularly pronounced improvements on domain-specific datasets.
[Table E] Full comparison with gradient-based compression baselines across major VQA benchmarks.
| Method | ChartQA | DocVQA | InfoVQA | TextVQA | PMC-VQA | WebSRC | WC-VQA |
|---|---|---|---|---|---|---|---|
| GradOT (80%) | 82.3 | 83.9 | 45.6 | 79.5 | 45.1 | 72.1 | 48.1 |
| OBD-LLM (50%) | 85.1 | 84.8 | 45.2 | 79.6 | 46.0 | 81.8 | 50.6 |
| Ours (50%, Adaptive) | 85.1 | 84.9 | 46.1 | 79.7 | 49.5 | 84.3 | 51.2 |
- Clarification on TALE’s Theoretical and Algorithmic Contributions
We further clarify that distinguishing TALE from a training perspective is not merely a conceptual distinction but provides the foundation for our theoretical contributions. Specifically, Proposition 1 establishes that the terminal divergence between the adapters of the full model and the emulator is bounded by the accumulated per-step gradient gaps, theoretically justifying the minimization of LoRA gradient gaps. This motivates our formal definition of a training-aware emulator and the subsequent derivation of Theorem 1. Such training-oriented theoretical justification is absent in OBD-LLM, which approaches emulation from an inference perspective, despite the mathematical equivalence arising from the approximation and decomposition of the LoRA gradient gap. TALE further introduces an additional algorithmic contribution: the adaptive regularization.
- Empirical and Analytical Contributions
Finally, we introduce server-side and user-side on-device training scenarios under different assumptions regarding computational resources and data access. Through extensive experiments, we demonstrate the effectiveness of TALE in improving adaptation performance in user personalization (Figure2) and task customization (Table2). Furthermore, our in-depth analysis in Figure 4 examines the extent to which and how TALE preserves the full model’s training signals from three complementary perspectives. These empirical and analytical contributions are absent in OBD-LLM.
We are deeply grateful for your detailed review and for engaging with our method at the level of its derivation. Your comments clearly identified where our original claims exceeded the evidence. To address this, we have not only adjusted our statements for precision but also added in-depth analytical evidence, ensuring our contributions are now fully supported.
We have moderated the performance claims in the revised manuscript. To address the variability concern, we conducted an additional three-seed comparison on five benchmarks at 50% compression.
[Table A] Accuracy gains over EMLoC at 50% compression (mean ± std over three seeds; percentage points).
These results support the gains reported in the paper, with mean improvements exceeding the corresponding across-seed standard deviations on all five benchmarks.
The magnitude of improvement is also meaningful relative to full-model LoRA. The results show that TALE with adaptive regularization reduces the average gap from 2.94 to 1.18 percentage points, closing 59.9% of EMLoC’s gap to full-model performance. We will incorporate these findings into the manuscript and make effort to extend the multi-seed evaluation and mean ± std reporting to all main results.
Unlike OBD-LLM, which targets inference fidelity, and GradOT, which maximizes model privacy at the risk of suboptimal downstream adaptation, TALE directly targets training-signal fidelity without such constraints.
Contribution beyond OBD-LLM: Despite similar compression formulations, TALE explicitly addresses gradient discrepancies that accumulate across fine-tuning steps (Proposition 1) and propagate across layers through backpropagation, introducing layer-wise adaptive regularization to stabilize empirical gradient statistics (SG). As a result, TALE outperforms OBD-LLM after adaptation, across three domain-specific datasets (Table B) and under server-side distribution shifts (Table C), where ill-conditioned gradient statistics can increase sensitivity to calibration errors and amplify downstream gradient mismatches.
Conceptual Distinctions from GradOT: While both methods aim to preserve gradients, GradOT maximizes model privacy by intentionally degrading the compressed model’s standalone performance, introducing a trade-off between privacy and adaptation fidelity. As shown in Table D, GradOT achieves lower downstream performance than TALE at the same compression ratio, despite its gradient-preserving objective.
[Table B] Comparison of TALE with OBD-LLM and EMLoC on three domain-specific downstream adaptation benchmarks.
[Table C] Downstream task customization performance of OBD-LLM and TALE under user-side and server-side settings.
[Table D] Performance comparison of TALE and GradOT at different compression ratios on general-domain (InfoVQA) and domain-specific (WebSRC) benchmarks.
To examine gradient preservation beyond the local theoretical setting, we added block-wise gradient alignment measurements on the actual compressed model in the new Figure 1(b). Across LLM blocks, TALE achieves roughly twice the average gradient alignment of EMLoC, with the most evident gap emerging in the early blocks where EMLoC's alignment drops near zero. We further extend this analysis in the new Section 4.6 to examine accumulated LoRA updates, transfer losses, and training trajectories, supplemented by step-wise gradient alignment across fine-tuning in Appendix G.
We welcome your feedback on whether our revised claims now accurately reflect the evidence, and will expand our variance and comparative analyses in the final version. We hope the revised manuscript addresses your concerns and invites a more favorable assessment.