- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies preference reversals that can occur when information from multiple sources is integrated, even when the individual sources initially agree. The authors analyze this phenomenon through representation geometry, deriving conditions for preference preservation using Gram matrices and developing a normalized Gram-mismatch criterion for assessing integration stability. They also propose geometry regularization methods, including Gram whitening and direct alignment, and evaluate them on controlled experiments and several real-world decision tasks.
Critical strengths
-
The paper studies an interesting problem that is not captured by aggregate predictive-performance metrics alone. The theoretical analysis provides explicit OLS-based conditions linking representation geometry to preference preservation.
-
The paper distinguishes preference reversal, decision changes, and utility degradation rather than treating them as equivalent.
-
The paper is clearly written and easy to understand.
Critical weaknesses
-
The main results assume a fixed representation map and an OLS readout. The experiments also use a shared representation map for source and pooled readouts, which is narrower than the motivating setting of integrating independently trained models. The extent to which the conclusions apply to jointly optimized representations or nonlinear decision heads remains unclear.
-
Although normalized Gram mismatch provides a useful geometric diagnostic, its practical predictive value is not fully established. The paper's own cross-domain results show that similar mismatch values can have different consequences. More evidence is needed to establish the criterion's practical value beyond source margins, conditioning, and the accompanying theoretical bound.
-
The interventions improve some geometric metrics and preference-preservation rates, but improvements in utility outcomes are less consistent. Several paired uncertainty intervals for utility-loss differences include zero, so the evidence for practical decision-quality improvements is limited.
Questions for authors
-
How do the results extend to independently learned representations and nonlinear decision heads?
-
The proposed regularization methods improve geometric alignment, but this does not consistently translate into lower utility loss. Could the authors explain this discrepancy and clarify when improved geometric alignment is expected to improve decision quality?
-
How sensitive are the results to the task-defined utility functions and source-partition choices?