acceptodds
Under review as a conference paper at ICLR 2027

Whose View Wins? Inside Egocentric Bias in Visuospatial Perspective Taking

Abstract

Visuospatial perspective taking (VSPT) requires reasoning from a viewpoint different from the viewer’s own, yet Multimodal Large Language Models (MLLMs) often exhibit egocentric bias. We investigate the representational mechanism underlying this failure using controlled scenes that independently manipulate egocentric and altercentric spatial relations. Linear probing shows that both relations are decodable from hidden representations despite poor VSPT performance under perspective conflict, suggesting that altercentric information is not simply missing. We then decompose hidden representations into egocentric and altercentric components and quantify their contributions along the ego–alter decision direction. Under conflict, the egocentric component contributes systematically more strongly toward its corresponding answer than the altercentric component. This imbalance is accompanied by a large and consistent egocentric advantage in component magnitude, while alignment differences are weaker and model-dependent. Guided by this analysis, we intervene on magnitude and alignment separately; both improve conflict-case performance, with magnitude suppression generally yielding larger gains. The decomposition also transfers to natural-image benchmarks in several settings. Together, our results suggest that egocentric bias stems from an egocentric-favoring imbalance in how concurrently represented perspectives contribute to model predictions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
vision-language models, visuospatial perspective taking, egocentric bias, spatial reasoning, representation analysis, representation decomposition

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7pswOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates egocentric bias in visuospatial perspective taking in multimodal large language models. The authors use controlled synthetic scenes to probe both perspectives, decompose hidden representations, and compare their contributions to model outputs. They then propose magnitude suppression and norm-preserving rotation to reduce egocentric influence. Experiments on two models and three datasets show improvements on perspective-conflict cases, alongside substantial degradation on some non-conflict cases and mixed cross-domain transfer results.

Critical strengths
  1. Well-controlled diagnostic design. Systematically varying object position and target orientation while balancing spatial relations under both perspectives reduces confounding factors. This provides a useful experimental foundation for isolating and studying egocentric bias.

  2. Coherent progression from analysis to intervention. The study proceeds logically from probing perspective information to subspace decomposition, contribution analysis, and targeted interventions. This structure makes the proposed explanation testable, while improvements on conflict cases and transfer in several settings provide preliminary evidence linking the representation analysis to model behavior.

Critical weaknesses
  1. It is unclear whether the proposed decomposition cleanly separates the two perspectives. Class-mean subspaces capture variation associated with perspective labels, but may still contain mixed information. The 55.1% altercentric readout accuracy in Figure 5 supports a perspective-related preference, but does not establish clean separation. Cross-decoding experiments would help assess whether each component contains information about the other perspective and how much information remains in the residual. The stability of the decomposition across data splits should also be examined.

  2. The intervention experiments only modify the egocentric component, while the reliability of the altercentric component remains unclear. Including corresponding interventions, such as amplifying the altercentric component, would help assess the relative roles of excessive egocentric influence and an unreliable altercentric component.

  3. The paper reports the best results from layer–strength sweeps, but does not clarify whether configurations were selected using an independent validation set. Without a separate selection procedure, the reported improvements may be optimistic and may not hold on unseen data.

  4. The overall gains come with substantial degradation on non-conflict examples and depend on the evaluation distribution. For Qwen on the controlled dataset, magnitude suppression reduces non-conflict accuracy from 94.23% to 46.15%. Based on Table 1, weighting conflict and non-conflict subsets equally reduces accuracy from 50.96% to 44.55%. This trade-off makes the aggregate benefit sensitive to the proportion of conflict examples and limits the evidence for a robust improvement in perspective-taking ability.

I find the research question and proposed approach interesting. I would be open to raising my score if the authors provide convincing additional experiments or clarifications that address the concerns above.

Questions for authors

See Weaknesses.

5 comments

Thank you for your insightful comments and constructive suggestions.

A. [Weakness 1] Component Separation and Stability

Cross-decoding experiments would help assess whether each component contains information about the other perspective and how much information remains in the residual.

  • Cross-decoding shows that each perspective is most accurately decoded from its corresponding component, while information retained in the residual has limited influence on output decisions under the tested interventions.

  • Cross-decoding of Decomposed Components

    • We trained linear probes to predict egocentric and altercentric labels from each decomposed component. The table reports final-layer probing accuracy.
    ModelTarget Labelhegoh_{ego}halterh_{alter}hrh_r
    Qwen3-VL-4BEgo10033.6571.63
    Alter25.4851.2046.15
    Molmo2-4BEgo99.7631.7365.39
    Alter23.0852.6443.99
    SpatialAxiom-9BEgo95.9135.5877.40
    Alter26.4472.3661.54
    • These results support the perspective selectivity of our decomposition, although perspective-related information remains in the residual.
    • Notably, SpatialAxiom-9B, a spatial reasoning model newly added to our experiments, exhibits higher altercentric decoding accuracy from its corresponding component than the two 4B models, further supporting improved perspective selectivity.
    • We have added the layer-wise cross-decoding results to Appendix C.2 (Figure 9).
  • Residual Suppression

    • To assess the behavioral influence of information retained in the residual, we compared residual suppression with egocentric suppression on Controlled.
    • Unlike egocentric suppression, residual suppression produces no statistically significant accuracy changes from baseline.
    ModelMethodConflictNon-conflictOverall
    Qwen3-VL-4BEgo Sup.42.95 (+35.26)*46.15 (−48.08)*43.75 (+14.42)*
    Residual Sup.5.77 (−1.92)94.23 (+0.00)27.88 (−1.44)
    Molmo2-4BEgo Sup.37.82 (+28.85)*61.54 (−30.77)*43.75 (+13.94)*
    Residual Sup.10.26 (+1.28)88.46 (−3.85)29.81 (+0.00)
    SpatialAxiom-9BEgo Sup.62.18 (+19.23)*80.77 (+0.00)66.83 (+14.42)*
    Residual Sup.39.74 (−3.21)78.85 (−1.92)49.52 (−2.88)

    Values are accuracies (%) with changes from baseline (pp). * indicates Holm-adjusted (p<0.05) (two-sided exact McNemar test; 18 comparisons).

    • These findings suggest that decodable information in the residual does not have the same behavioral influence as the egocentric component under the tested interventions.
    • We have added this control to Appendix D.2 (Table 7).
  • We acknowledge that our decomposition does not achieve complete perspective separation. However, our goal is not perfect disentanglement, but to investigate how decodable perspective information contributes asymmetrically to model predictions and whether intervening on this imbalance can mitigate egocentric bias. Our cross-decoding and residual-suppression results support this objective despite the incomplete separation.

B. [Weakness 1] Subspace Stability

The stability of the decomposition across data splits should also be examined.

  • Both egocentric and altercentric subspaces exhibit substantially higher stability than their permutation baselines.
  • We evaluated subspace stability across the three agent-disjoint Controlled folds using normalized subspace overlap and maximum principal angles, with 200 label-permutation repetitions.
ModelSubspaceOverlapPermutation overlapMaximum Principal AnglePermutation Principal Angle
Qwen3-VL-4BEgo.996 ± .001.531 ± .0864.9° ± 0.9°71.5° ± 8.7°
Alter.828 ± .079.532 ± .08639.6° ± 12.1°71.6° ± 8.7°
Molmo2-4BEgo.986 ± .005.471 ± .07510.3° ± 2.4°74.3° ± 7.0°
Alter.930 ± .018.478 ± .08723.2° ± 4.3°74.2° ± 8.1°
SpatialAxiom-9BEgo.967 ± .017.400 ± .08114.5° ± 5.3°74.8° ± 6.8°
Alter.953 ± .015.401 ± .08018.6° ± 3.6°74.8° ± 7.2°
  • Consistent with the cross-decoding results, SpatialAxiom-9B exhibits the highest altercentric subspace stability among the evaluated models, although altercentric subspaces remain less stable than egocentric ones in the two 4B models.
  • We have added this analysis to Appendix C.1 (Table 5).

C. [Weakness 2] Altercentric Amplification and Reliability

Including corresponding interventions, such as amplifying the altercentric component, would help assess the relative roles of excessive egocentric influence and an unreliable altercentric component.

  • Altercentric amplification improves conflict accuracy across all three models on Controlled, with substantially larger gains when the altercentric component already supports the correct answer.

  • Altercentric Amplification

    • Following your suggestion, we introduced a complementary intervention that amplifies the altercentric component:

    h′=h+β∥hego∥2∥halter∥2halterh' = h+\beta \frac{\Vert h_{ego} \Vert_2}{\Vert h_{alter}\Vert_2}h_{alter}

    • We fixed β=1\beta=1 and reused the validation-selected layer from egocentric suppression.
    ModelMethodOverallConflictNon-conflict
    Qwen3-VL-4BEgo Sup.43.75 (+14.42)42.95 (+35.26)46.15 (−48.08)
    Alter Amp.37.02 (+7.69)17.95 (+10.26)94.23 (+0.00)
    Molmo2-4BEgo Sup.43.75 (+13.94)37.82 (+28.85)61.54 (−30.77)
    Alter Amp.46.15 (+16.35)32.69 (+23.72)86.54 (−5.77)
    SpatialAxiom-9BEgo Sup.66.83 (+14.42)62.18 (+19.23)80.77 (+0.00)
    Alter Amp.69.23 (+16.83)62.82 (+19.87)88.46 (+7.69)
    • Altercentric amplification improves conflict accuracy with smaller non-conflict losses than egocentric suppression on Controlled. Notably, SpatialAxiom-9B improves both conflict (+19.87 pp) and non-conflict (+7.69 pp) accuracy, achieving 69.23% overall.
    • These improvements, achieved without removing the egocentric component, further support the role of relative perspective contributions in VSPT predictions.
  • Reliability of the Altercentric Component

    • To assess how these gains depend on altercentric reliability, we grouped Controlled samples by whether the pre-intervention altercentric-component readout matches the correct answer.

    • [Conflict samples]

      ModelAlter readoutnBaseline (%)Alter Amp. (%)Δ (%p)
      Qwen3-VL-4BCorrect6314.2941.27+26.98
      Incorrect933.232.15−1.08
      Molmo2-4BCorrect7114.0869.01+54.93
      Incorrect854.712.35−2.35
      SpatialAxiom-9BCorrect10045.0073.00+28.00
      Incorrect5639.2944.64+5.36
    • [Non-conflict samples]

      ModelAlter readoutnBaseline (%)Alter Amp. (%)Δ (%p)
      Qwen3-VL-4BCorrect25100.00100.00+0.00
      Incorrect2788.8988.89+0.00
      Molmo2-4BCorrect2892.86100.00+7.14
      Incorrect2491.6770.83−20.83
      SpatialAxiom-9BCorrect3577.1491.43+14.29
      Incorrect1788.2482.35−5.88
      • On conflict samples, amplification yields substantially larger gains when the altercentric readout is correct. When it is incorrect, improvements are limited or accuracy decreases.
      • On non-conflict samples, Molmo2-4B and SpatialAxiom-9B also show gains in the correct-readout subsets but degradation in the incorrect-readout subsets.
  • Together, these results suggest that adjusting the relative contributions of the two perspectives can improve VSPT, while the reliability of altercentric information influences the effectiveness of such interventions.

  • We have added the amplification and correctness-conditioned analyses to Appendix D.5 (Tables 10–11).


D. [Weakness 3] Independent Hyperparameter Selection

Without a separate selection procedure, the reported improvements may be optimistic and may not hold on unseen data.

  • We agree that separating hyperparameter selection from evaluation is important for ensuring a more rigorous and transparent assessment of intervention performance. Accordingly, we revised our evaluation protocol to select intervention configurations independently of held-out test performance.
  • For in-domain evaluation (Table 1), the intervention layer and strength are selected on inner validation folds using nested cross-validation, with performance reported on held-out outer folds.
  • For domain transfer (Table 2), configurations are selected exclusively on Controlled and applied to the target benchmarks without target-domain tuning.
  • The revised results retain the overall improvements observed in our original experiments.
  • We have updated Tables 1 and 2 and added the selection details to Appendix B.6, with layer–strength sensitivity results in Appendix D.6 (Figures 11–12).

E. [Weakness 4] Non-conflict Degradation and Distribution Dependence

The overall gains come with substantial degradation on non-conflict examples and depend on the evaluation distribution

  • We agree that overall accuracy gains depend on the proportion of conflict and non-conflict examples and should not be interpreted as uniform improvements across both subsets. Nevertheless, we consider this trade-off an informative finding of our analysis, as also recognized by other reviewers (R#Jjys, R#D1CH), who highlighted our transparent reporting of both improvements and degradation.

  • Our additional experiments with SpatialAxiom-9B show that non-conflict degradation is not inevitable. On Controlled, SpatialAxiom-9B achieves substantial conflict improvements without reducing non-conflict accuracy.

    ModelMethodConflictNon-conflictOverall
    SpatialAxiom-9BBaseline42.9580.7752.40
    Magnitude62.1880.7766.83
    Alignment62.1884.6267.79
  • Magnitude suppression improves conflict accuracy by 19.23 pp without non-conflict degradation, while alignment rotation improves both conflict (+19.23 pp) and non-conflict (+3.85 pp) accuracy. Under equal weighting, accuracy also improves from 61.86% to 71.48% and 73.40%, respectively.

  • These results show that the trade-off can be mitigated in models with stronger spatial reasoning capabilities, although intervention effects remain model- and benchmark-dependent. We have also addressed this limitation in Section 8 (Limitations and Future Work).

Reviewer_D1CHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper looks at egocentric bias when a model has to take someone else's visual perspective. Using 208 controlled scenes and agent-disjoint probes, the authors show both egocentric and altercentric labels can be decoded from activations. They then decompose activations with class-mean subspaces and a joint pseudoinverse, and LM-head readouts suggest the egocentric component contributes more to the output. Two interventions are tried, suppressing the magnitude and a norm-preserving rotation, on the controlled scenes and some natural images, including a transfer experiment (pp. 3-9; Appendices A-D).

Critical strengths
  1. The design separates "is the information there" from "does the model use it," which is the right question. Agent-disjoint folds and fitting on training folds take care of the obvious identity leakage. The altercentric probe accuracies (70.2% and 67.7%) are enough to support the modest claim that the information is not simply missing (pp. 3-4, 12).
  2. The authors report negative results and controls that many papers would leave out: the non-conflict degradation, the failed Molmo transfer, the full configuration sweeps, and the matched-gating suppression (pp. 8-9, 14-15, Table 5). It makes the tradeoffs visible.
Critical weaknesses
  1. Selection and evaluation are not cleanly separated. Section 6.2 reports the best overall accuracy over a sweep of layers and strengths (24 nonzero configs), and I could not find an independent selection split or any nested evaluation. Fitting the subspace on training data only doesn't remove the optimism from picking the best config on the test numbers. Same issue for transfer, if the config is picked on the target benchmark (pp. 8-9, 13-14). This matters because the natural-image gains are small and the subgroup tradeoff is large, e.g. controlled Qwen non-conflict accuracy drops from 94.23% to 46.15% (Table 1).
  2. The causal story about the specific factor needs more support. The joint subspaces are not necessarily independent. If you rotate the ego vector while keeping its norm, there is no guarantee it stays inside the ego subspace, or that the alter coordinates after re-decomposition are unchanged. Matched gating is there, but perturbation magnitude and cross-component leakage are not controlled (pp. 4, 7, 15). I also was not sure how the pre-final-norm activations plus the "effective head" actually map to the real output margins. So the claim that magnitude rather than alignment is the main causal factor feels stronger than what is shown.
Questions for authors
  1. Were the layer and strength chosen without looking at the held-out predictions? For transfer, can the config be selected on source data only?
  2. What normalization and effective-head definition are used, and how well do mean + component + residual actually reconstruct the real margins?
  3. Do the conclusions hold with matched perturbation norms? How much do the alter coordinates move after rotation, and how stable are the subspaces across folds? Other model families can be left for later.
5 comments

Thank you for your insightful comments and constructive suggestions.

A. [Weakness 1 & Question 1] Independent Hyperparameter Selection

Were the layer and strength chosen without looking at the held-out predictions? For transfer, can the config be selected on source data only?

  • We agree that separating hyperparameter selection from evaluation is important for ensuring a more rigorous and transparent assessment of intervention performance. Accordingly, we revised our evaluation protocol to select intervention configurations independently of held-out test performance.
  • For in-domain evaluation (Table 1), the intervention layer and strength are selected on inner validation folds using nested cross-validation, with performance reported on held-out outer folds.
  • For domain transfer (Table 2), configurations are selected exclusively on Controlled and applied to the target benchmarks without target-domain tuning.
  • The revised results retain the overall improvements observed in our original experiments.
  • We have updated Tables 1 and 2 and added the selection details to Appendix B.6, with layer–strength sensitivity results in Appendix D.6 (Figures 11–12).

B. [Weakness 2 & Question 2] Normalization and Margin Reconstruction

What normalization and effective-head definition are used, and how well do mean + component + residual actually reconstruct the real margins?

  • We clarified the normalization used in each analysis and verified that the decomposed components reconstruct the output margins with negligible numerical errors.

  • Semantic validation and interventions use pre-final-norm activations, whereas the final-layer contribution analysis in Section 4.2 uses post-final-norm activations. Component contributions in the latter are computed using the frozen LM head.

  • To verify margin reconstruction in the pre-final-norm space, we incorporate the pretrained RMSNorm into a sample-dependent effective head:

    \frac{W\operatorname{diag}(\gamma)} {\sqrt{d^{-1}\sum_{j=1}^{d}h_j^2+\epsilon}},$$ where $W$ is the frozen LM-head matrix, $\gamma$ is the RMSNorm gain, and the denominator is computed from the full hidden state and shared across components.
  • Reconstruction accuracy: The reconstructed margins closely match the model's original output margins:

    ModelPost-norm MAEPre-norm MAEPre-norm Margin Sign Agreement
    Qwen3-VL-4B0.075990.07542100% (156/156)
    Molmo2-4B1.574 × 10⁻⁶1.599 × 10⁻⁶100% (156/156)
  • The small reconstruction errors for Qwen3-VL-4B primarily reflect BF16 rounding in model outputs rather than decomposition errors. The additive decomposition itself reconstructs the hidden representations to numerical precision.

  • We have clarified these procedures and added the reconstruction results to Appendix B.3–B.4.

C. [Weakness 2 & Question 3] Causal Analysis of Magnitude and Alignment

Do the conclusions hold with matched perturbation norms? How much do the alter coordinates move after rotation?

  • We agree that our original wording may have overstated the causal distinction between magnitude and alignment. We have revised our interpretation accordingly and conducted additional analyses of perturbation norms, In-subspace rotation, and cross-component leakage.
  • Perturbation norms and In-subspace rotation
    • To address both concerns, we introduced an alignment rotation constrained to the estimated egocentric subspace (In-space A) and compared it with magnitude suppression matched to the same activation perturbation norm (Matched M). The intervention layer and rotation strength were selected on validation folds.

      MethodConflictNon-conflictOverall
      Qwen3-VL-4BBaseline7.6994.2329.33
      In-space A32.6965.3840.87
      Matched M30.7769.2340.38
      Molmo2-4BBaseline8.9792.3129.81
      In-space A31.4171.1541.35
      Matched M31.4171.1541.35
      SpatialAxiom-9BBaseline42.9580.7752.40
      In-space A63.4675.0066.35
      Matched M59.6276.9263.94
    • In-space rotation improves overall accuracy across all three models, achieving slightly higher accuracy than matched magnitude suppression for Qwen3-VL-4B and SpatialAxiom-9B, and identical accuracy for Molmo2-4B.

  • Cross-Component Leakage
    • To measure cross-component leakage, we re-decomposed the intervened activations and measured the relative changes in the altercentric and residual components.

      ModelMethodAlter Change (%)Residual Change (%)
      Qwen3-VL-4BOriginal Alignment32.0958.45
      In-subspace rotation0.621.59
      Molmo2-4BOriginal Alignment13.0820.23
      In-subspace rotation<0.0001<0.0001
      SpatialAxiom-9BOriginal Alignment2.176.56
      In-subspace rotation0.470.87
    • The constrained rotation substantially reduces cross-component leakage while retaining improvements in VSPT accuracy. These results show that alignment interventions can remain effective within the estimated egocentric subspace.

    • We have added the constrained intervention and matched-norm comparison to Appendix D.4.


D. [Question 3] Subspace Stability

How stable are the subspaces across folds?

  • Both egocentric and altercentric subspaces exhibit substantially higher stability than their permutation baselines.

  • We evaluated subspace stability across the three agent-disjoint Controlled folds using normalized subspace overlap and maximum principal angles, with 200 label-permutation repetitions.

    ModelSubspaceoverlapPermutation overlapMaximum Principal AnglePermutation Principal Angle
    Qwen3-VL-4BEgo.996 ± .001.531 ± .0864.9° ± 0.9°71.5° ± 8.7°
    Alter.828 ± .079.532 ± .08639.6° ± 12.1°71.6° ± 8.7°
    Molmo2-4BEgo.986 ± .005.471 ± .07510.3° ± 2.4°74.3° ± 7.0°
    Alter.930 ± .018.478 ± .08723.2° ± 4.3°74.2° ± 8.1°
    SpatialAxiom-9BEgo.967 ± .017.400 ± .08114.5° ± 5.3°74.8° ± 6.8°
    Alter.953 ± .015.401 ± .08018.6° ± 3.6°74.8° ± 7.2°
  • SpatialAxiom-9B, a spatial reasoning model newly added to our experiments, exhibits the highest altercentric subspace stability among the evaluated models, although altercentric subspaces remain less stable than egocentric ones in the two 4B models.

  • We have added this analysis to Appendix C.1 (Table 5).

Reviewer_JjysOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates why multimodal models often answer perspective-taking questions from the camera’s viewpoint rather than the specified target agent’s viewpoint. Using controlled scenes, the authors show that both egocentric and altercentric relations are linearly decodable. They decompose hidden states into perspective-related components and find stronger egocentric contributions, particularly in component magnitude. Magnitude suppression and alignment rotation improve conflict-case accuracy but can reduce non-conflict accuracy, with some transfer to natural-image benchmarks. The main contribution is a representational analysis supported by targeted interventions.

Critical strengths
  1. The controlled design distinguishes information availability from output behavior. Independently varying object position and target orientation, with agent-disjoint evaluation, provides a useful setting for studying perspective errors. The finding that altercentric relations are decodable despite egocentric predictions narrows the explanation beyond complete absence of the relevant information, without requiring that an external probe’s success imply the model already uses that information correctly.

  2. The interventions expose an informative behavioral trade-off. Separating magnitude suppression from alignment rotation and reporting both conflict and non-conflict outcomes makes the analysis more informative than an overall-accuracy comparison alone. Natural-image transfer adds evidence that the estimated components are not useful only in the rendered setting. For an analysis paper, the observed non-conflict losses can themselves be informative; I do not regard the lack of an unconditional performance improvement as a sufficient reason to reject the work.

Critical weaknesses
  1. The intervention evidence does not sufficiently distinguish perspective correction from suppression of a dominant answer. Reducing the egocentric component can improve cases where the dominant answer is wrong and harm cases where it is correct, even without restoring the use of altercentric information. For example, Qwen’s controlled conflict accuracy rises from 7.69% to 42.95%, while non-conflict accuracy falls from 94.23% to 46.15% (Table 1). This is consistent with the proposed account, but is also compatible with a more generic redistribution of answer preferences. A perspective-agnostic control matched for suppression of the original preferred answer, together with destination-specific answer transitions, is needed to establish the specificity of the intervention. The distinction is especially important for binary questions, where suppressing the wrong option automatically favors the correct one. This limits the mechanistic conclusion rather than showing that the reported changes are incorrect.

  2. The relationship between the proposed mechanism and perspective-specific supervision remains unresolved. The study characterizes frozen pretrained checkpoints, but does not test whether a small amount of targeted perspective supervision corrects the behavior or changes the identified contribution imbalance. Insufficient supervision and representational imbalance could be causally connected, rather than competing explanations. A small-data adaptation analysis would therefore be valuable not simply as a stronger performance baseline, but as a test of whether the proposed measure tracks behavioral correction. If errors improve while the measured imbalance remains, the interpretation would need qualification; if both change together, the account gains support, although that alone would not prove mediation. I am not claiming that the original training data lacked such examples, or that an analysis paper must outperform fine-tuning.

Questions for authors
  1. [Decision-relevant evidence] Does perspective-component suppression outperform a perspective-agnostic control that similarly reduces the model’s original preferred answer, with strengths selected on held-out data? For conflict examples, please distinguish changes to the correct altercentric answer from changes to other wrong answers. On controlled groups with fixed object/camera positions and varying target orientation, does the intervention improve geometrically correct tracking of the target’s viewpoint? These analyses would distinguish specific correction from generic answer suppression.

  2. [Decision-relevant evidence] Is any small-data perspective fine-tuning result available on held-out agents or scenes, with conflict and non-conflict outcomes and the contribution analysis repeated before and after adaptation? Please account for the label budget used by both adaptation and subspace estimation. The main question is whether the proposed imbalance tracks correction under targeted supervision; if such evidence is unavailable, please clarify what the present analysis can establish about the mechanism, separately from the origin of the bias.

2 comments

Thank you for your insightful comments and constructive suggestions.

A. [Weakness 1 & Question 1] Perspective-Specific Intervention Effects

  • Perspective-Agnostic Control

    Does perspective-component suppression outperform a perspective-agnostic control that similarly reduces the model’s original preferred answer, with strengths selected on held-out data?

    • Yes. Egocentric suppression outperforms a perspective-agnostic, logit-matched control across all three models on Controlled and even on 3DSR, which uses binary-choice questions.
    • To distinguish perspective-specific correction from generic answer suppression, we reduced the logit of the model's originally preferred answer by an amount matched to the reduction induced by egocentric suppression. The ego-suppression layer and strength were selected on inner validation folds using nested cross-validation.
    • The table reports accuracy differences (Ego suppression − logit control, pp), where positive values favor egocentric suppression.
      DatasetModelConflictNon-conflictOverall
      ControlledQwen3-VL-4B+35.90−34.62+18.27
      Molmo2-4B+18.59−7.69+12.02
      SpatialAxiom-9B+4.49−3.85+2.41
      3DSRQwen3-VL-4B+13.22−8.78+5.07
      Molmo2-4B+6.79−4.88+2.46
      SpatialAxiom-9B+2.30−1.17+1.01
      ViewSpatialQwen3-VL-4B+0.28−3.23−0.24
      Molmo2-4B−1.25−7.26−2.14
      SpatialAxiom-9B+4.46−8.87+2.50
    • The consistent conflict-case advantage on Controlled and 3DSR suggests that the improvements cannot be explained solely by suppressing the originally preferred answer logit. This distinction is particularly relevant to 3DSR, where the binary answer format makes generic answer suppression a plausible alternative explanation. Full results are provided in Appendix D.1.
  • Correct Altercentric Answers vs. Other Incorrect Answers

    For conflict examples, please distinguish changes to the correct altercentric answer from changes to other wrong answers.

    • Egocentric suppression produces more corrections to the correct altercentric answer than the logit control across all three models.
    • We analyzed Controlled conflict samples with initially egocentric predictions, distinguishing Ego→Alter (correct), Ego→Others (incorrect), and Ego→Ego (unchanged).
      ModelMethodEgo → AlterEgo → OthersEgo → EgoTotal
      Qwen3-VL-4BEgo supp.524829129
      Logit control24123129
      Molmo2-4BEgo supp.403445119
      Logit control194258119
      SpatialAxiom-9BEgo supp.2333157
      Logit control1623957
    • Perspective-specific correction: Across all three models, egocentric suppression produces more transitions to the correct altercentric answer than the logit control, providing further evidence that its effects extend beyond generic suppression of the originally preferred answer.
    • Decomposition fidelity: The two 4B models also exhibit substantial transitions to other incorrect answers. This is consistent with the weaker semantic selectivity of the altercentric component observed in our original analysis (Figure 5), suggesting that suppressing egocentric influence does not necessarily guarantee correct altercentric predictions when the remaining altercentric representation is imperfect.
    • SpatialAxiom-9B: In contrast, SpatialAxiom-9B shows 23 corrections to the altercentric answer and only three transitions to other incorrect answers. This suggests that stronger spatial reasoning capabilities, together with more selective altercentric representations, may help translate reduced egocentric influence into correct altercentric predictions.
    • We have added this analysis to Appendix E.1.
  • Geometrically Correct Viewpoint Tracking

    On controlled groups with fixed object/camera positions and varying target orientation, does the intervention improve geometrically correct tracking of the target’s viewpoint?

    • Yes. Egocentric suppression can improve orientation-dependent predictions under fixed scene geometry.
    • We evaluated 52 groups (13 agents × 4 object positions), each containing four target orientations while keeping agent and object positions fixed.
    • Suppression increases the number of correctly predicted orientations in 30 of 52 groups (57.7%) for each 4B model and 27 of 52 groups (51.9%) for SpatialAxiom-9B. These results demonstrate improvements in viewpoint-dependent predictions beyond changes in overall accuracy.
    • Figure 13 in Appendix E.2 illustrates representative cases where egocentric suppression improves predictions across target orientations under fixed scene geometry.

B. [Weakness 2 & Question 2] Spatial Supervision and Representational Imbalance

  • Representational Imbalance and Spatial Reasoning Performance

    If errors improve while the measured imbalance remains, the interpretation would need qualification; if both change together, the account gains support, although that alone would not prove mediation.

    • We observed that improved perspective-taking performance was accompanied by substantially reduced egocentric contribution imbalance when comparing SpatialAxiom-9B with Qwen3.5-9B.

    • SpatialAxiom-9B is a model obtained by full-parameter supervised fine-tuning of Qwen3.5-9B on spatial reasoning tasks, including perspective taking.

    • Perspective-taking performance

      • SpatialAxiom-9B achieves substantially higher conflict accuracy on both Controlled and 3DSR, although a conflict–non-conflict accuracy gap remains, indicating persistent egocentric bias.
      Model[Controlled] Conflict[Controlled] Non-conflict[3DSR] Conflict[3DSR] Non-conflict
      Qwen3.5-9B7.0592.3125.4061.13
      SpatialAxiom-9B42.9580.7744.1458.59
    • Representational imbalance

      • Using the measure defined in Section 4.2, we find that the proportion of ego-dominant samples on Controlled conflict cases decreases from 94.9% to 51.9%, while the median contribution gap decreases from +3.51 to +0.06.
      ModelEgo-dominant Samples (%)Median Contribution Gap
      Qwen3.5-9B94.9%+3.51
      SpatialAxiom-9B51.9%+0.06
    • These findings strengthen the association between egocentric contribution imbalance and perspective-taking performance, while suggesting that representational balance alone may not be sufficient for accurate perspective taking. Investigating the additional factors underlying the remaining failures is an important direction for future work.

    • We have added this comparison and the corresponding visualization (Figure 10) to Appendix C.3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers,

We sincerely thank all reviewers for their interest in our research questions and insightful suggestions on how to strengthen our claims.

We were encouraged by your recognition of several strengths of our work: our design for distinguishing information availability from its use (R#Jjys, R#D1CH, R#7psw), our finding that altercentric information remains linearly decodable from model representations despite egocentrically biased predictions (R#Jjys, R#D1CH), the intervention results including transfer to natural-image benchmarks (R#Jjys, R#7psw), and our transparent reporting of the trade-off between conflict and non-conflict performance (R#Jjys, R#D1CH).

Our central goal is to understand how linearly decodable egocentric and altercentric information contributes to model predictions, and whether intervening on their asymmetric contributions can mitigate egocentric bias. The perspective decomposition serves as a tool for this analysis, rather than an attempt to achieve perfect disentanglement of the two perspectives.

The revised manuscript includes two major additions addressing concerns raised across the reviews.

1. Additional Experiments with SpatialAxiom-9B [1]

To address the concerns raised by the reviewers, we conducted additional experiments with SpatialAxiom-9B, a model obtained by further supervised fine-tuning Qwen3.5-9B [2] to enhance its spatial reasoning capabilities.

  • Cleaner perspective decomposition (R#7psw): SpatialAxiom exhibits higher altercentric probing accuracy, more stable altercentric subspaces, and more selective altercentric component readouts than the two 4B models.
  • Reduced non-conflict trade-off (R#D1CH): On Controlled, egocentric suppression improves conflict accuracy without degrading non-conflict accuracy in SpatialAxiom, suggesting that the trade-off can be mitigated.
  • Reduced contribution imbalance (R#Jjys): Compared with its base model, Qwen3.5-9B [2], SpatialAxiom achieves higher conflict accuracy alongside substantially reduced egocentric contribution imbalance.

2. Independent Intervention Configuration Selection (R#D1CH, R#7psw)

Our revised evaluation protocol uses nested cross-validation for in-domain evaluation, selecting intervention configurations on inner validation folds and reporting held-out performance. For domain transfer, configurations are selected exclusively on Controlled, without target-domain tuning.

Additional experiments and analyses addressing reviewer-specific concerns are detailed in our individual responses.

We thank all reviewers again for their valuable feedback.

All Authors


[1] Yujing Lou et al. “SpatialAxiom: An Open Spatial Intelligence Model for General Spatial Reasoning.” July 2026. https://d2i-ai.github.io/SpatialAxiom

[2] Qwen Team. “Qwen3.5: Towards Native Multimodal Agents.” February 2026. https://qwen.ai/blog?id=qwen3.5

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.