- Rating
- 1
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates how Vision-Language Models (VLMs) internally represent "country identity" (geographic identity). The authors' central claim is that when a linear probe recovers macroeconomic signals such as GDP PPP from VLM hidden states with high accuracy, this does not reflect an encoding of the abstract concept of "wealth." Instead, it reflects a country-identity direction that has memorized which country is being seen. They detect this with Leave-One-Country-Out (LOCO) cross-validation, and claim this collapse signature replicates across four architectures. They further show that the signal is distributed across the residual stream and resists all six classes of post-hoc intervention tested (PGD, SAE feature ablation, UAP, ROME, LoRA, and closed-form linear erasure via INLP/LEACE). The signal is also behaviorally consequential: a single UAP direction reduces the mortgage approval gap, but this comes from collapsing predictions toward the unconditional mean (distributional collapse) rather than from removing the signal. From these results the authors argue that a reduction in the output gap is not evidence that the signal was removed, that post-hoc debiasing of VLMs is unreliable, and that the remaining intervention surface is training-time concept regularization.
Critical strengths
- The paper challenges a widely shared assumption in interpretability research, namely that high linear-probe accuracy implies that the concept is understood. It poses the competing hypothesis that the probe may be reading nothing more than a memorized country identity, and it operationalizes this as a testable LOCO gap. LOCO itself has been used in prior work, but this paper brings it into the probing of VLM hidden states and moves the discussion from "demonstrating that bias exists" to "causally asking whether it can be removed." As a cautionary note on the interpretation of probing results, this has clear value.
- Confirming both the LOCO collapse and the L1 birth across four families from different developers and with different architectures (Gemma-4/3, Qwen3-VL, MiniCPM-V) is a real merit. Observing the same tendency across different weights supports the claim that, at a minimum, these phenomena are not specific to a single model.
Critical weaknesses
- The paper claims that the country-identity signal resists all six classes of post-hoc intervention. However, as the authors themselves acknowledge, there is no positive control showing that the same pipeline can successfully intervene on an attribute that is trivially manipulable. As a result, it is not possible to separate "the intervention failed because the signal is robust" from "the intervention failed because the pipeline is simply non-functional on this model class." The paper's strongest claim, and the conclusion drawn from it that "training-time intervention is the remaining path," rests entirely on the fact that all six interventions failed. Because the cause of that failure is undetermined, the falsifiability of the conclusion itself is undermined.
- The paper makes strongly general and assertive claims such as "consistent across four models" and "training-time is the only remaining intervention surface," yet the design has several constraints: (a) five of the six interventions were run on Gemma-4 alone; (b) only one city per country was used, so a city-specific visual fingerprint cannot be separated from country identity; and (c) validation is concentrated at roughly the 4B scale. Point (b) bears directly on the central interpretation, since it cannot rule out the possibility that the LOCO collapse reflects the failure to generalize a single city's visual fingerprint rather than the failure to generalize a country's economic properties. There is a non-negligible gap between the strength of the conclusions and the reach of the evidence, which leaves the claims and the evidence insufficiently aligned.
Questions for authors
The following questions all bear on the assessment of the paper.
-
For the five interventions run on Gemma-4 alone, is there evidence that the core geometry-behavior dissociation replicates on at least one additional architecture? Can the authors clarify precisely which claim the phrase "across four architectures" in the Abstract applies to, namely the LOCO diagnostic or the intervention resistance?
-
Is the phrase "training-time concept regularization is the remaining intervention surface" intended to exclude other untested post-hoc methods (such as nonlinear erasure or architectural modification)? Or is it a qualified claim limited to "within the six classes tested in this paper"? Can the authors clarify the scope of this statement?