- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes loss smoothing, a method that linearly interpolates between source and target objectives during neural network adaptation. The core claim is that retaining source supervision improves target adaptation even when source-task performance is not a goal. A toy study isolates the gradient reweighting mechanism, and experiments span four settings: continual supervised learning, pretrained vision adaptation, LLM fine-tuning, and offline-to-online RL. The paper systematically rules out simpler explanations and shows retention advantages persist after source supervision ends
Critical strengths
The primary strength is the breadth of empirical coverage. Loss smoothing is a single, simple recipe tested across four genuinely distinct adaptation settings (vision, language, RL, and continual learning) with consistent directional improvements. This cross-domain validation is rare and supports the "broadly useful" framing more convincingly than a deep dive into one setting would. The systematic ablation controls (Section 5.3) ruling out KL regularization, EWC, LP-FT, and coefficient-only schedules as alternative explanations are well-designed and strengthen the claim that the source signal itself (not slower learning or initialization preservation) drives the benefit.
Critical weaknesses
"Conflicting features" lack a general definition. The shared vs. conflicting feature distinction is only formalized within the toy study, where it is separable by construction. No general mathematical definition is provided for arbitrary networks. The paper acknowledges the two "cannot be disentangled" in practice, yet the mechanistic narrative is built on this distinction.
Limited model scale undermines generalizability claims. The paper claims loss smoothing is "a broadly useful tool for model adaptation" but the largest trained model is OLMo-1B. Vision experiments use ViT-Base (~86M params), RL experiments use small MLPs. No experiment exceeds 1B parameters. The paper does not discuss scalability to larger models nor acknowledge model scale as a limitation. Domain breadth is a strength, but scale coverage is insufficient to fully support the generalizability claim.
Inconsistent and insufficient statistical evidence. No formal significance tests (paired t-tests, Wilcoxon, etc.) are reported anywhere in Section 4. Statistical reporting varies across experiments: RL results use 95% bootstrap intervals over 5 seeds (adequate), continual CIFAR uses SEM over 10 seeds (adequate), but OLMo LLM results rely on only 3 seeds with SEM values not numerically stated in the main text making it impossible to verify whether the 1.9–2.3 pp improvement margins exceed variance. The paper relies entirely on visual inspection of error bars with no formal hypothesis testing, which is insufficient to support the breadth of claims made.
Compute cost acknowledged but not quantified. The limitations section mentions smoothing "approximately doubles compute" but provides no concrete measurements in the core paper. Appendix H contains some useful data on token exposure and image overhead that should be in the core paper, not buried in supplementary material.
No code released. The paper provides no code repository, anonymized link, or supplementary code. A text-only Reproducibility Statement points to Appendix A for hyperparameters and training details, which are reasonably thorough. However, given four distinct experimental settings (vision, LLM, RL, continual learning), each with its own codebase, dependencies, and implementation subtleties, text-based descriptions alone significantly raise the reproduction barrier.
Minor: Undefined shorthand at lines 74–75. The contribution bullet references benefits coming "not from simpler alternatives" without specifying what these are. The reader only learns the paper means KL regularization, EWC, LP-FT, and coefficient-only control much later. These should be briefly characterized inline.
Minor: Core evidence placed in appendix. Tables 6 and 7 are in the appendix but directly cited in Section 4 as primary evidence.
Minor: Missing related work coverage. The paper does not discuss or compare against adversarial adaptation methods or assessor-guided/meta-learning sample-weighting approaches: both relevant to the problem of managing task transitions under distribution shift.
Questions for authors
Can you provide formal significance tests (e.g., paired bootstrap or Wilcoxon) where improvement margins appear comparable to variance?
How do you expect loss smoothing to behave at model scales beyond 1B parameters? Is there any theoretical or empirical reason to believe the gradient reweighting mechanism scales, or could the shared/conflicting feature dynamics change qualitatively at larger scale?
Can you provide concrete cost guidance?
2 comments
I thank the authors for the rebuttal and the additional results, but I am keeping my rating, as the new information does not substantially change my overall assessment of the paper.
Rebuttal
We thank the reviewer for their feedback and are glad they found the breadth of our evaluation valuable and our controls effective at isolating the role of the source signal. We address their concerns and questions below.
Definition of shared feature
Defining shared and conflicting features in general is difficult. Unlike in the toy example, we cannot say that everything the model computes from one part of the input is shared and everything computed from the other part is conflicting. However, our analysis of the toy example is mostly based on a gradient decomposition, which we think can be recreated in the general case. For example, given the gradients gt and gs of the target and source losses, the shared part can be defined as the part along the ∣gt∣gt+∣gs∣gs direction, which improves both losses, and a conflicting part as the one along the ∣gt∣gt−∣gs∣gs direction which trades one loss for another. Loss smoothing reweights these two parts: its update αgt+(1−α)gs gives the shared part the weight 2α∣gt∣+(1−α)∣gs∣ and the conflicting part the weight 2α∣gt∣−(1−α)∣gs∣. Applied to the toy example, this decomposition places the conflicting part mostly in the MNIST branch and the shared part mostly in the CIFAR branch. We think this analysis can be generalized to the more practical settings and will incorporate it into the final version of the paper.
Loss Smoothing Scale
While we appreciate the need to demonstrate scale, as you mentioned our experiments have already covered several orders of magnitude of scale, starting with small MLPs for RL/continual learning (300k params), to standard Vision Transformer architectures (~86M params), to LLMs (1B params). We limited our experiments to 1B for LLMs both because of practical compute limitations. We will run some additional experiments testing loss smoothing with 7B parameter LLMs to analyze the transferability to larger scales.
Significance Tests
We follow the reporting practice recommended by Agarwal et al. (2021) for small numbers of runs: interval estimates of the effect rather than binary significance tests. Results are reported as 95% bootstrap intervals for RL and DomainNet, and as SEM for CIFAR and OLMo. The key comparisons are not close calls:
For OLMo at the final checkpoint (smoothing vs hard switch):
We have also added the specific SEM numbers as a table in Appendix C.3.
Cost Guidance
The “approximately doubles” statement applies only within the smoothing window. Over a full run, the overhead scales with the window length. On DomainNet, measured training time rises from 1.86 to 2.14 hours (+15%) at our default 25% window, comparable to the KL baseline (2.25 hours), and only 1.91 hours (+3%) for a 5% window, which retains approximately half of the target gain. For OLMo, the 128-update window adds 26.5% tokens.
We also add a new ablation (Appendix H.3) where we examine the effect of using fewer resources when smoothing. Mixing by sampling, where source examples fill part of a fixed-size batch, removes the extra forward/backward pass entirely and matches loss mixing on DomainNet target accuracy (76.81 vs 76.77 IQM at 25%, see Appendix H). Only 1,024 source images are needed to keep almost all of the target gain (76.69). We have added this discussion and the new sample mixing results to Appendix H.
Code Release
We do plan on releasing the full codebase upon final release of the paper.
Other minor comments
Thank you for pointing out the other small issues with the manuscript. We have incorporated the feedback into the manuscript.
We hope this addresses your concerns. If it does, we would appreciate your reconsidering the score. We are happy to answer any remaining questions.
[1] Agarwal, Rishabh, et al. "Deep reinforcement learning at the edge of the statistical precipice." Advances in neural information processing systems 34 (2021): 29304-29320.