- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates egocentric bias in visuospatial perspective taking in multimodal large language models. The authors use controlled synthetic scenes to probe both perspectives, decompose hidden representations, and compare their contributions to model outputs. They then propose magnitude suppression and norm-preserving rotation to reduce egocentric influence. Experiments on two models and three datasets show improvements on perspective-conflict cases, alongside substantial degradation on some non-conflict cases and mixed cross-domain transfer results.
Critical strengths
-
Well-controlled diagnostic design. Systematically varying object position and target orientation while balancing spatial relations under both perspectives reduces confounding factors. This provides a useful experimental foundation for isolating and studying egocentric bias.
-
Coherent progression from analysis to intervention. The study proceeds logically from probing perspective information to subspace decomposition, contribution analysis, and targeted interventions. This structure makes the proposed explanation testable, while improvements on conflict cases and transfer in several settings provide preliminary evidence linking the representation analysis to model behavior.
Critical weaknesses
-
It is unclear whether the proposed decomposition cleanly separates the two perspectives. Class-mean subspaces capture variation associated with perspective labels, but may still contain mixed information. The 55.1% altercentric readout accuracy in Figure 5 supports a perspective-related preference, but does not establish clean separation. Cross-decoding experiments would help assess whether each component contains information about the other perspective and how much information remains in the residual. The stability of the decomposition across data splits should also be examined.
-
The intervention experiments only modify the egocentric component, while the reliability of the altercentric component remains unclear. Including corresponding interventions, such as amplifying the altercentric component, would help assess the relative roles of excessive egocentric influence and an unreliable altercentric component.
-
The paper reports the best results from layer–strength sweeps, but does not clarify whether configurations were selected using an independent validation set. Without a separate selection procedure, the reported improvements may be optimistic and may not hold on unseen data.
-
The overall gains come with substantial degradation on non-conflict examples and depend on the evaluation distribution. For Qwen on the controlled dataset, magnitude suppression reduces non-conflict accuracy from 94.23% to 46.15%. Based on Table 1, weighting conflict and non-conflict subsets equally reduces accuracy from 50.96% to 44.55%. This trade-off makes the aggregate benefit sensitive to the proportion of conflict examples and limits the evidence for a robust improvement in perspective-taking ability.
I find the research question and proposed approach interesting. I would be open to raising my score if the authors provide convincing additional experiments or clarifications that address the concerns above.
Questions for authors
See Weaknesses.
5 comments
C. [Weakness 2] Altercentric Amplification and Reliability
Including corresponding interventions, such as amplifying the altercentric component, would help assess the relative roles of excessive egocentric influence and an unreliable altercentric component.
-
Altercentric amplification improves conflict accuracy across all three models on Controlled, with substantially larger gains when the altercentric component already supports the correct answer.
-
Altercentric Amplification
- Following your suggestion, we introduced a complementary intervention that amplifies the altercentric component:
- We fixed and reused the validation-selected layer from egocentric suppression.
Model Method Overall Conflict Non-conflict Qwen3-VL-4B Ego Sup. 43.75 (+14.42) 42.95 (+35.26) 46.15 (−48.08) Alter Amp. 37.02 (+7.69) 17.95 (+10.26) 94.23 (+0.00) Molmo2-4B Ego Sup. 43.75 (+13.94) 37.82 (+28.85) 61.54 (−30.77) Alter Amp. 46.15 (+16.35) 32.69 (+23.72) 86.54 (−5.77) SpatialAxiom-9B Ego Sup. 66.83 (+14.42) 62.18 (+19.23) 80.77 (+0.00) Alter Amp. 69.23 (+16.83) 62.82 (+19.87) 88.46 (+7.69) - Altercentric amplification improves conflict accuracy with smaller non-conflict losses than egocentric suppression on Controlled. Notably, SpatialAxiom-9B improves both conflict (+19.87 pp) and non-conflict (+7.69 pp) accuracy, achieving 69.23% overall.
- These improvements, achieved without removing the egocentric component, further support the role of relative perspective contributions in VSPT predictions.
-
Reliability of the Altercentric Component
-
To assess how these gains depend on altercentric reliability, we grouped Controlled samples by whether the pre-intervention altercentric-component readout matches the correct answer.
-
[Conflict samples]
Model Alter readout n Baseline (%) Alter Amp. (%) Δ (%p) Qwen3-VL-4B Correct 63 14.29 41.27 +26.98 Incorrect 93 3.23 2.15 −1.08 Molmo2-4B Correct 71 14.08 69.01 +54.93 Incorrect 85 4.71 2.35 −2.35 SpatialAxiom-9B Correct 100 45.00 73.00 +28.00 Incorrect 56 39.29 44.64 +5.36 -
[Non-conflict samples]
Model Alter readout n Baseline (%) Alter Amp. (%) Δ (%p) Qwen3-VL-4B Correct 25 100.00 100.00 +0.00 Incorrect 27 88.89 88.89 +0.00 Molmo2-4B Correct 28 92.86 100.00 +7.14 Incorrect 24 91.67 70.83 −20.83 SpatialAxiom-9B Correct 35 77.14 91.43 +14.29 Incorrect 17 88.24 82.35 −5.88 - On conflict samples, amplification yields substantially larger gains when the altercentric readout is correct. When it is incorrect, improvements are limited or accuracy decreases.
- On non-conflict samples, Molmo2-4B and SpatialAxiom-9B also show gains in the correct-readout subsets but degradation in the incorrect-readout subsets.
-
-
Together, these results suggest that adjusting the relative contributions of the two perspectives can improve VSPT, while the reliability of altercentric information influences the effectiveness of such interventions.
-
We have added the amplification and correctness-conditioned analyses to Appendix D.5 (Tables 10–11).
D. [Weakness 3] Independent Hyperparameter Selection
Without a separate selection procedure, the reported improvements may be optimistic and may not hold on unseen data.
- We agree that separating hyperparameter selection from evaluation is important for ensuring a more rigorous and transparent assessment of intervention performance. Accordingly, we revised our evaluation protocol to select intervention configurations independently of held-out test performance.
- For in-domain evaluation (Table 1), the intervention layer and strength are selected on inner validation folds using nested cross-validation, with performance reported on held-out outer folds.
- For domain transfer (Table 2), configurations are selected exclusively on Controlled and applied to the target benchmarks without target-domain tuning.
- The revised results retain the overall improvements observed in our original experiments.
- We have updated Tables 1 and 2 and added the selection details to Appendix B.6, with layer–strength sensitivity results in Appendix D.6 (Figures 11–12).
E. [Weakness 4] Non-conflict Degradation and Distribution Dependence
The overall gains come with substantial degradation on non-conflict examples and depend on the evaluation distribution
-
We agree that overall accuracy gains depend on the proportion of conflict and non-conflict examples and should not be interpreted as uniform improvements across both subsets. Nevertheless, we consider this trade-off an informative finding of our analysis, as also recognized by other reviewers (R#Jjys, R#D1CH), who highlighted our transparent reporting of both improvements and degradation.
-
Our additional experiments with SpatialAxiom-9B show that non-conflict degradation is not inevitable. On Controlled, SpatialAxiom-9B achieves substantial conflict improvements without reducing non-conflict accuracy.
Model Method Conflict Non-conflict Overall SpatialAxiom-9B Baseline 42.95 80.77 52.40 Magnitude 62.18 80.77 66.83 Alignment 62.18 84.62 67.79 -
Magnitude suppression improves conflict accuracy by 19.23 pp without non-conflict degradation, while alignment rotation improves both conflict (+19.23 pp) and non-conflict (+3.85 pp) accuracy. Under equal weighting, accuracy also improves from 61.86% to 71.48% and 73.40%, respectively.
-
These results show that the trade-off can be mitigated in models with stronger spatial reasoning capabilities, although intervention effects remain model- and benchmark-dependent. We have also addressed this limitation in Section 8 (Limitations and Future Work).
Thank you for your insightful comments and constructive suggestions.
A. [Weakness 1] Component Separation and Stability
Cross-decoding shows that each perspective is most accurately decoded from its corresponding component, while information retained in the residual has limited influence on output decisions under the tested interventions.
Cross-decoding of Decomposed Components
Residual Suppression
Values are accuracies (%) with changes from baseline (pp). * indicates Holm-adjusted (p<0.05) (two-sided exact McNemar test; 18 comparisons).
We acknowledge that our decomposition does not achieve complete perspective separation. However, our goal is not perfect disentanglement, but to investigate how decodable perspective information contributes asymmetrically to model predictions and whether intervening on this imbalance can mitigate egocentric bias. Our cross-decoding and residual-suppression results support this objective despite the incomplete separation.
B. [Weakness 1] Subspace Stability