- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether interactions between different loss functions need to be considered when training a shared model, and whether more complex models of these interactions lead to practical benefits. The authors study this question using dynamics models in continuous-control tasks, with five training losses corresponding to different prediction depths. They compare two approaches for predicting the benefit of combining losses: an additive model that sums the contributions of individual losses, and a quadratic model that also considers pairwise interactions. The authors estimate the effects of different loss combinations through short training updates and use these estimates to select losses for subsequent training.
On the theoretical side, the paper analyzes the effects of different loss combinations under small-step SGD, showing that individual loss contributions dominate at first order, while interactions appear at higher orders. Experiments on two continuous-control tasks with feedforward and recurrent models show that, although modeling interactions can sometimes improve prediction accuracy, the additive and quadratic models make the same selection in 178 out of 180 cases. The more complex model also fails to provide consistent improvements in training outcomes. Further experiments with different optimizers, loss combinations, and computational budgets show that using all losses or spending the additional computation on continued training can sometimes be more effective than complex loss selection.
Overall, the paper suggests that the presence of interactions between losses does not necessarily justify modeling them explicitly. More complex approaches are useful only when they lead to better decisions and the benefits outweigh the additional computational cost.
Critical strengths
S1. The paper addresses a meaningful research question. It examines whether modeling interactions between different loss functions is necessary, considering not only prediction accuracy but also its impact on training decisions and computational costs. This helps clarify when more complex loss selection methods may be useful.
S2. An interesting theoretical analysis is provided. The authors derive different orders for individual loss effects, pairwise interactions, and higher-order interactions under small-step SGD, offering some explanation for why a simple additive model may be sufficient.
S3. The experimental evaluation covers several important aspects. Beyond comparing the prediction accuracy of additive and quadratic models, the authors examine their effects on loss selection and subsequent training performance. Additional experiments with different optimizers, regularization settings, loss weights, and computational budgets help assess the benefits and limitations of modeling interactions.
Critical weaknesses
W1. Limited experimental settings and generalizability. The experiments cover only two continuous-control tasks (Walker-walk and Cheetah-run), using feedforward and recurrent models with five losses at different prediction depths. Since these losses come from the same dynamics model and may be highly correlated, it remains unclear whether the findings generalize to other tasks, model architectures, or more diverse loss combinations.
W2. Limited scope of the theoretical analysis. The main theoretical results rely on fixed-step SGD with small learning rates, while adaptive optimizers commonly used in practice may behave differently. Although the authors include experiments with Adam, these use a fresh optimizer state rather than preserving the optimizer state during continuous training. This limits how well the theoretical findings apply to more general training settings.
W3. Unclear practical value of interaction modeling. The additive and quadratic models make the same selection in 178 out of 180 cases, and the more complex quadratic model does not consistently improve downstream performance. While these negative results align with the paper's goal, the experiments do not clearly show when interaction modeling would be beneficial. As a result, the three proposed conditions mainly summarize the observations, and their practical guidance for training strategy design remains limited.
Questions for authors
Q1. Could the authors extend the evaluation to more diverse tasks, model architectures, or loss combinations to better assess the generalizability of the findings?
Q2. How would the results change when using Adam with a continuously maintained optimizer state rather than a fresh state?
Q3. Can the authors provide concrete examples where modeling loss interactions leads to better training decisions and downstream performance?