- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates the discrepancy between prediction accuracy and downstream control performance in learned dynamics models. The authors argue that reducing mean squared error does not necessarily improve control outcomes, as changes in prediction error directions can lead to more harmful consequences. To evaluate this issue, they introduce a consequence-based audit that compares finite horizon returns from true and predicted successor states under a fixed controller. They further propose a budgeted search method that uses Ridge Regression to approximate normalized consequence loss and prioritize subsequent simulation queries, supported by a local Taylor approximation bound. Experiments across several continuous control tasks demonstrate that model updates can reduce MSE while introducing new violations. The proposed method improves failure discovery in the initial task suite, although its advantages are less consistent on new tasks and do not translate into clear improvements in model repair or repeated use selection.
Critical strengths
The paper seemingly addresses an important limitation of evaluating learned dynamics models solely through prediction error magnitude. Even when MSE decreases, changes in the direction of prediction errors can lead to worse downstream control outcomes. I find this observation meaningful, as prediction accuracy alone does not fully capture the consequences of model errors in decision-making systems. The experiments provide concrete examples that support this concern and motivate evaluating learned dynamics beyond MSE.
Critical weaknesses
- Connection between the proposed consequence loss and actual control performance.
The proposed consequence loss compares returns from true and predicted successor states under the same fixed controller. However, these returns can differ even when the controller takes identical actions in both continuations. Therefore, the metric does not directly measure the performance loss caused by using an incorrect model prediction in actual control. Furthermore, the one sided violation criterion allows large negative consequence losses to pass, even though such values may indicate substantial discrepancies between predicted and true states. Although the authors explicitly define their objective as conditional deterioration, its relationship to practical model reliability remains unclear.
- Applicability of the theoretical approximation.
The theoretical analysis relies on a twice differentiable consequence response with bounded curvature. However, some of the most consequential model errors may occur near action switching regions, termination boundaries, or other discontinuities. These cases are not covered by the proposed Taylor approximation bound. Moreover, the required gradient and curvature bounds are not estimated in practice, and the analysis does not establish a guarantee on violation discovery. Consequently, the theoretical results provide limited justification for the algorithm in the settings where consequence auditing may be most important.
- Methodological novelty.
The proposed acquisition method combines residual magnitude, geometric exploration, and local Ridge Regression to prioritize simulation queries. These are relatively standard techniques, and the approximation bound follows from Taylor expansion and basic Ridge Regression properties. While their application to consequence auditing is reasonable, the additional methodological contribution is not entirely clear, especially given that simpler response based alternatives outperform the proposed method on the new tasks.
- Evidence of downstream benefits
The experiments provide convincing examples of model updates that reduce MSE while introducing new consequence violations. However, the proposed acquisition method does not demonstrate consistent advantages on new tasks. Furthermore, consequence based repair fails to establish a significant incremental benefit over simpler repair strategies, while consequence based model selection produces lower returns than the baselines in the repeated use evaluation. These results substantially limit the practical implications of the proposed method beyond regression diagnosis.
Questions for authors
- What happens if the consequence loss is evaluated using its absolute value?
The current violation criterion considers only , allowing arbitrarily large negative consequence losses to be treated as acceptable. However, a large negative loss also indicates a substantial discrepancy between the reference and predicted continuations. Although Appendix I.5 provides a post-hoc sensitivity analysis using a two-sided criterion, how would replacing with affect the proposed acquisition strategy and its failure discovery performance?
- How does the proposed approximation behave when the consequence function is not twice differentiable?
The theoretical approximation assumes that is twice differentiable with bounded curvature. However, consequential prediction errors may arise near action switching thresholds, termination boundaries, or other discontinuities, where these assumptions may not hold. How does the proposed acquisition method behave in such cases? In particular, what theoretical or empirical evidence supports the reliability of the approximation when the smoothness assumptions are violated?