acceptodds
Withdrawn from ICLR 2027

More Data, Worse Decisions? Explaining Preference Reversals in Neural Networks through Representation Geometry

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural networks increasingly incorporate additional information from multiple sources to improve generalization, but combining such information can also alter previously consistent decisions. While more data provides richer information, information integration may disrupt previously consistent decisions and induce decision instability. This raises the question of why additional information can sometimes lead to worse decisions by reversing previously consistent preferences. We study this question from the perspective of representation geometry. We first reveal that such preference reversals arise from changes in representation geometry during information integration. Building on this mechanism, we develop a geometric criterion for assessing whether candidate information sources can preserve decision stability before information integration. Finally, we show that representation geometry can be controlled during training to improve compositional stability under information integration. Experiments on controlled settings and real-world decision tasks support the identified mechanism, examine the relationship between geometric compatibility and decision stability, and evaluate how geometry-based interventions influence decision stability. These findings provide a geometric perspective for understanding and improving decision stability in neural network systems that integrate information from multiple sources.

Primary area:
learning theory
Keywords:
Representation Learning, Neural Networks, Decision Stability, Preference Reversal, Data Composition, Representation Geometry, Case-Based Decision Theory, Ordinary Least Squares

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_UWNLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies preference reversals that can occur when information from multiple sources is integrated, even when the individual sources initially agree. The authors analyze this phenomenon through representation geometry, deriving conditions for preference preservation using Gram matrices and developing a normalized Gram-mismatch criterion for assessing integration stability. They also propose geometry regularization methods, including Gram whitening and direct alignment, and evaluate them on controlled experiments and several real-world decision tasks.

Critical strengths
  • The paper studies an interesting problem that is not captured by aggregate predictive-performance metrics alone. The theoretical analysis provides explicit OLS-based conditions linking representation geometry to preference preservation.

  • The paper distinguishes preference reversal, decision changes, and utility degradation rather than treating them as equivalent.

  • The paper is clearly written and easy to understand.

Critical weaknesses
  • The main results assume a fixed representation map and an OLS readout. The experiments also use a shared representation map for source and pooled readouts, which is narrower than the motivating setting of integrating independently trained models. The extent to which the conclusions apply to jointly optimized representations or nonlinear decision heads remains unclear.

  • Although normalized Gram mismatch provides a useful geometric diagnostic, its practical predictive value is not fully established. The paper's own cross-domain results show that similar mismatch values can have different consequences. More evidence is needed to establish the criterion's practical value beyond source margins, conditioning, and the accompanying theoretical bound.

  • The interventions improve some geometric metrics and preference-preservation rates, but improvements in utility outcomes are less consistent. Several paired uncertainty intervals for utility-loss differences include zero, so the evidence for practical decision-quality improvements is limited.

Questions for authors
  1. How do the results extend to independently learned representations and nonlinear decision heads?

  2. The proposed regularization methods improve geometric alignment, but this does not consistently translate into lower utility loss. Could the authors explain this discrepancy and clarify when improved geometric alignment is expected to improve decision quality?

  3. How sensitive are the results to the task-defined utility functions and source-partition choices?

Reviewer_KrNrOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes a theory of preference reversals resulting from combining data from multiple sources, as a consequence of differences in representational geometry between those sources. A set of proofs is presented to show that these differences can drive preference reversals in linear models applied to fixed features. A measure of geometric mismatch is also proposed as a way to screen for reversals before combining sources. Experiments are then presented involving linear models applied both to raw inputs and to MLP features trained on various real-world datasets. A regularization method is proposed to reduce the geometric differences that are claimed to lead to preference reversal.

Critical strengths
  • The paper investigates an interesting question with real-world consequences.
  • The paper is clearly written and the experiments are well documented. The proofs appear to be correct.
  • A combination of synthetic and real-world datasets is considered in the experiments.
Critical weaknesses

The theory only applies to linear readouts from fixed features. The theory is also only supported by the controlled experiment (Figure 2), which involves a synthetic setup with a linear readout applied directly to the inputs. The setup in this experiment is designed to produce preference reversals as representational geometry diverges. The experiments with neural networks largely do not accord with the predictions of the theory. The trend in Figure 3 is driven by differences between datasets (the partition with the highest mismatch shows almost no reversals), and in Table 8, most configurations that reduce mismatch show lower rather than higher preference preservation. The proposed regularization method also does not show substantial benefits.

Questions for authors

Is there any explanation for why the neural network experiments don't show the predicted relationship? What factors are missing from the theory that might explain these discrepancies, and how might these be leveraged to develop an improved regularization method?

Reviewer_piByOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper looks at why training on two data sets that agree can produce a new model that disagrees. It does so by investigating the representation geometry in the model training process and argues that the pooled result is not a simple average, for example in the particular least squares setting it discusses. The paper uses experiments coming from tasks and a synthetic setup generated across a few different data sets.

Critical strengths

The paper formalizes the difference between the agreement in source data and the reversal of the action clearly, and the gram resolve provides a simple condition to determining whether or not pooling would preserve agreement, at least in this limited case.

Critical weaknesses

It is unclear why the paper compares its Lee square setting to modern neural network methods and it does not sufficiently argue that the results it finds can be practically used in this alternative setting. Furthermore, the strength of the results, especially the evidence for regularization, is relatively weak. And the focus on measuring only gains and losses as opposed to the magnitude of those changes makes it difficult to tell if we actually get benefits from preserving agreement.

Questions for authors

Do the regularizers reliably improve? What is the principal contribution and how does it go beyond the least squares identity? Can you compare how pooling affects the overall utility and accuracy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.