More Data, Worse Decisions? Explaining Preference Reversals in Neural Networks through Representation Geometry

Neural networks increasingly incorporate additional information from multiple sources to improve generalization, but combining such information can also alter previously consistent decisions. While more data provides richer information, information integration may disrupt previously consistent decisions and induce decision instability. This raises the question of why additional information can sometimes lead to worse decisions by reversing previously consistent preferences. We study this question from the perspective of representation geometry. We first reveal that such preference reversals arise from changes in representation geometry during information integration. Building on this mechanism, we develop a geometric criterion for assessing whether candidate information sources can preserve decision stability before information integration. Finally, we show that representation geometry can be controlled during training to improve compositional stability under information integration. Experiments on controlled settings and real-world decision tasks support the identified mechanism, examine the relationship between geometric compatibility and decision stability, and evaluate how geometry-based interventions influence decision stability. These findings provide a geometric perspective for understanding and improving decision stability in neural network systems that integrate information from multiple sources.