- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a bilevel optimization objective for aligning gradients between pretraining and downstream tasks. The actual implementation first takes one gradient ascend step on the normalized projection on to the downstream task gradient with respect to the loss coefficients/weights, updating the coefficients/weights to increase alignment, and then it does one gradient descent on the backbone on the combined loss.
Critical strengths
- The objective itself is sensible.
- experiments seem to be comprehensive and covering many different domains and settings
Critical weaknesses
- I am not sure if normalizing a backward signal is a new idea.
- There is no convergence guarantee on constantly shifting loss objective, especially when W can be unbounded.
- The experiment results are mixed and improvements are within margin. I am not sure if there is other merits to this method like more hardware efficient, but it does not seem to have comparisons with baseline methods so it remains unclear.
minor complaints on the terminologies: usually weights are referred to as the model parameters, and update referred to the optimizer step, the paper uses rather unconventional wordings and often times would confuse readers.
Questions for authors
Table 4 shows significant slowdowns, but the paper claims "efficiency"?
We thank the reviewer for the constructive feedback and for recognizing the motivation of our objective and the breadth of our empirical evaluation.
W1. Novelty. We do not claim that gradient normalization itself is novel. Our contribution is the analysis demonstrating that conventional weight normalization distorts the optimal composite-gradient direction (Fig. 2), and the proposed solution based on composite-gradient normalization (Sec. 3.2).
Unlike Du et al. (2018), who directly use cosine similarity as individual auxiliary loss weights, GraP optimizes the cosine similarity between the composite pretraining gradient and the downstream gradient, accounting for interactions between individual objectives. Compared with OL-AUX, our normalization mitigates uncontrolled weight growth and improves initial learning speed in the RL experiment (App. G). The normalization ablation (Tab. 9) further demonstrates an improvement from 77.48 to 78.05 under identical supervision.
W2. Convergence and unbounded weights. Although the raw loss weights can theoretically grow without bound, the normalized alignment objective is invariant to their uniform positive rescaling. Moreover, the norm of the composite representation-space gradient propagated through the backbone is bounded by construction. Thus, increasing weight magnitudes does not directly lead to exploding backbone gradients.
Nevertheless, we acknowledge that this does not establish convergence of the joint optimization procedure. Obtaining such guarantees for simultaneously evolving model parameters and loss weights is an interesting direction for future work. Empirically, the learned weights generally exhibit smooth and stable trajectories (App. H).
W3. Empirical results and practical benefits. Our objective is not to universally outperform existing MTL methods, but to efficiently select pretraining loss weights targeting downstream performance. GraP achieves the best average performance across event-sequence benchmarks for all three backbones and significantly outperforms most adaptive-weighting baselines after Holm correction. More importantly, it achieves performance comparable to carefully tuned Optuna weights without multiple independent training runs.
We acknowledge that improvements in computer vision are marginal. Nevertheless, GraP identifies a redundant objective (Attn-NNCLR), demonstrating its additional diagnostic value.
Q1. Computational efficiency. The overhead reported in Tab. 4 is relative to a single standard training run. However, our primary computational reference is hyperparameter optimization, which requires multiple runs. GraP introduces an average epoch-time overhead of 34% (17–71% across datasets) while optimizing loss weights online.
Furthermore, compared with a direct implementation of gradient-based loss-weight optimization, our shared-representation approximation and gradient reuse reduce the number of full-backbone backward passes from K to one per iteration. This is particularly beneficial for expensive backbones and composite objectives with many components.
Terminology. We agree that distinguishing loss weights, model parameters, composite gradients, and actual optimizer updates would improve clarity and will revise the terminology accordingly.
We hope these clarifications better establish the methodological and practical contributions of GraP. We would be grateful if the reviewer would reconsider their assessment and consider updating the score.