- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies a safety decoding problem about how evidence from different lookahead horizons can be combined for candidate action selection. The authors propose MVSD, which combines absolute and pairwise loss predictions across horizons into calibrated constraints for selecting the final action.
Critical strengths
1. The paper studies an safety decoding question about how evidence from different lookahead horizons can jointly affect action selection. The formulation is quite insightful. Casting cross-horizon evidence as constraints for action selection is easy to follow and reasonable.
2. MVSD has a simple and practical design. Compared with heuristic fusion, its design philosophy is more principled. The component and decision-level analyses make this method more concrete.
Critical weaknesses
1. I see that the authors use the predictive complementarity across horizons as the main motivation for MultiView. Here the views are based on nested observations, where a longer horizon can contain the shorter horizon information. I am concerned that the observed complementarity may mainly reflect different prediction errors, without showing that the views provide genuinely independent evidence.
2. The identification of the composition effect is not very clean. The main comparisons change both the available evidence and how that evidence is combined. The current results show that different views can provide complementary signals, but do not isolate how much of the gain comes from composition itself. e.g, the constructed examples show that multiple views can jointly support an action that no single view supports, but do not show it can drive the observed gains.
Questions for authors
1. Given that the horizon views are nested, what evidence shows that horizon composition contributes beyond horizon-specific prediction errors?
2. On the actual evaluation set, how often does the selected action depend on evidence from multiple views, and how often do those composition-driven action changes improve over the reference action?
3. How do the current analyses isolate the effect of the composition rule while keeping the fitted predictions and calibration intervals fixed?
4. What uncertainty estimates accompany the common-bank improvements, especially the relatively small benign-loss gain?