acceptodds
Under review as a conference paper at ICLR 2027

Who Acts on Whom? Probing the Binding of Objects to Roles in Vision Transformers

Abstract

Humans understand scenes not only by recognizing which objects are present but also by determining who acts on whom, that is, by assigning objects to event roles, such as agent and patient. Replicating this human capability could provide artificial intelligence with a foundation for representing and reasoning about relations between objects. Recent work shows that Vision Transformers (ViTs) learn to encode, in low-dimensional subspaces of their representations, whether two patches belong to the same object. However, it remains unclear how these models represent the binding of different objects to agent and patient roles. CLIP-style models with ViT image encoders are known to struggle to distinguish captions in which these roles are swapped, raising the question of whether role information is absent from their representations or present but insufficiently used. In this paper, we train a bilinear probe on ordered pairs of object patches and find that the binding of objects to roles is encoded in a *relation-binding subspace* of patch representations in CLIP-style models. Given the region of one object in a relation, the probe assigns high scores to the region of its partner (the agent given the patient, and vice versa). Ablating this subspace reduces image-text matching accuracy on captions with role swaps more than on any other type of caption modification, whereas ablating a random subspace of the same dimension has no such effect.

Primary area:
interpretability and explainable AI
Keywords:
Vision Transformers, Vision–Language Models, Relational Reasoning, Interpretability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4HU3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether vision–language image encoders represent which objects occupy agent and patient roles, and whether those representations influence image–text matching. It trains low-dimensional asymmetric bilinear probes on patch representations from four frozen CLIP-family and SigLIP encoders using directed interactions derived from the Panoptic Scene Graph dataset. The authors examine layer-wise decoding, visualize instance-specific interactions, and intervene on the learned subspaces through image-wise mean ablation and exchanges between object regions. Experiments report that role-sensitive benchmark categories generally experience the largest mean degradation under learned-subspace ablation, while object-region exchanges can shift preferences toward captions containing the donor object category. The appendices investigate negative types, random seeds, alternative ablations, subspace dimensions, and cross-seed stability. Together, these results support the presence of compact, behaviorally relevant interaction information, while the interpretation of that information as specifically encoding object-to-role binding remains subject to the qualifications below.

Critical strengths
  1. The paper addresses a useful distinction between information being available in an encoder and being used in its final predictions. The progression from probing in Section 3.1 to ablation in Section 3.2 and exchange interventions in Section 3.3 gives the investigation a coherent structure. A failure on a role-sensitive caption comparison does not, by itself, establish that the visual representation lacks the relevant information. Conversely, successful decoding does not establish that the original model uses it. Studying both questions across four encoders is a substantive contribution to understanding compositional failures, independently of whether the work improves benchmark accuracy.

  2. The visualizations provide concrete evidence of instance-specific interaction information beyond object-category compatibility alone. Figures 3(b) and 4 examine scenes containing multiple instances of the same object categories, including riders and motorcycles or people and horses. The responses distinguish particular interacting partners within these scenes, rather than merely highlighting every instance of a compatible category. The additional examples in Appendices B.2 and B.3 support this qualitative observation across models or layers. These examples matter because generic category co-occurrence would not be sufficient to explain correct partner selection among repeated categories. They are not a substitute for aggregate controlled evaluation, but they provide meaningful positive evidence that should not be overlooked.

  3. The mean-ablation intervention preserves the image-wide patch mean while selectively removing variation in the learned subspace. In Section 3.2, Equation (2) replaces each patch’s projected coordinates with their image-wise mean. Because the rows of WW are orthonormal, this leaves the orthogonal complement unchanged and preserves the sum of the patch representations at the intervention layer, including after the fixed affine denormalization. The intervention therefore has a useful structural property: it removes spatial variation in selected directions without simply eliminating their image-wide average. Tables 1 and 2 report the largest mean losses on SugarCrepe swap-object and VALSE actant-swap, respectively, for all four encoders. Appendix B.4 also reports five-run variability and an alternative zero-ablation intervention. This provides stronger evidence of behavioral relevance than probe accuracy alone, although it does not establish that every affected direction is specific to binding.

  4. The exchange experiment includes an appropriate control for the object-region averaging operation. Section 3.3 compares exchanged object-region subspace means with a no-exchange condition in which each region receives its own mean. This controls for the fact that replacing patch-level coordinates with an object-level mean is itself an intervention. Table 3 additionally reports the original and control accuracies, the number of eligible initially correct examples, and the resulting conditional flip rates, while Figure 5 shows the layer-wise behavior rather than only the selected maxima. For example, CLIP ViT-L has an agent-exchange flip rate of 49.33 ± 3.61%, compared with 1.03 ± 0.29% for the random-subspace control at the reported layer. These design and reporting choices make the demonstrated susceptibility to targeted feature exchange interpretable, even though the precise mechanism of the flips requires further qualification.

  5. The dimensionality and stability analyses add information about the compactness and reproducibility of the identified representations. Appendix B.5 examines dimensions from 2 to 32 instead of relying exclusively on the main 16-dimensional configuration. Table 7 reports cross-seed subspace overlap, while Tables 8 and 9 connect dimensionality to downstream intervention effects. A particularly informative example is SigLIP at dimension 2: the reported overlap is 0.962 ± 0.004, and SugarCrepe swap-object accuracy decreases by 3.02 ± 0.22 points under learned-subspace mean ablation, compared with 0.73 ± 0.34 points for random ablation. The corresponding VALSE effect is much weaker, so this does not establish a universally sufficient two-dimensional binding representation. It nevertheless provides useful evidence that some role-sensitive behavior depends on a very compact and reproducible component.

Critical weaknesses
  1. The evidence does not yet fully distinguish object-to-role binding from correlated category, identity, and scene-context information. Section 3.1 trains the probe on directed annotated object pairs pooled across interaction predicates, with swapped and replacement negatives. Performance can therefore benefit from information about typical category combinations, directionality, or contextual compatibility, in addition to instance-specific binding. The same-category visualizations are valuable evidence against a purely category-level explanation, but the supplied material does not provide an aggregate evaluation that quantifies their prevalence or controls these alternatives across examples. In Section 3.3, the compared captions also differ in an object category, so an exchange-induced preference change could partly reflect altered category evidence. This limits the specificity of the central interpretation rather than establishing that the representations contain no binding information. A targeted same-category partner-selection analysis and evidence about preservation of object identity under intervention would substantially strengthen the claim.

  2. Object-region exchange can alter the image-wide amount of subspace evidence when the two regions contain different numbers of patches. Under the construction in Section 3.3, let the exchanged regions contain nan_a and ncn_c patches, with projected means μa\mu_a and μc\mu_c. Relative to the own-mean control, exchanging their means changes the summed projected representation by

    (na−nc)(μc−μa).(n_a - n_c)(\mu_c - \mu_a).

    Thus, unlike the mean-ablation intervention in Section 3.2, this exchange generally changes the global feature mean when region sizes differ. A caption preference could consequently shift partly because the donor category gains aggregate feature evidence, even without a selective reassignment of an otherwise preserved binding representation. This is a confirmed property of the intervention, not a demonstrated explanation of the reported results; the paper does not establish its empirical contribution. A region-size analysis, approximately equal-size exchanges, or a control matching the global mean change would help determine how specifically the exchange results support the proposed mechanism.

  3. Equal-dimensional random subspaces do not necessarily provide a matched intervention-strength control. Sections 3.2 and 3.3 compare learned and random subspaces of the same dimension. However, the per-feature normalization described in Appendix A.2 does not whiten the representation. For mean ablation, if P=W⊤WP = W^\top W and CimgC_{\mathrm{img}} is the within-image covariance of the normalized patch features, the average squared displacement is tr⁡(PCimg)\operatorname{tr}(P C_{\mathrm{img}}). This quantity can differ between learned and random subspaces despite identical rank. The reported contrast therefore establishes that the learned directions matter more than typical random directions, but does not fully separate role specificity from differences in removed variation or intervention magnitude. This is a potentially fixable control limitation, not evidence that magnitude explains the findings. Reporting displacement norms and a modest magnitude-matched comparison would clarify how much specificity remains.

  4. Configuration selection and the reported uncertainty leave the strength of some comparative claims insufficiently calibrated. Appendix B.5 states that the main dimensionality was chosen with reference to probe accuracy, stability, and downstream ablation behavior; consequently, the complete configuration was not selected solely using an independent probing criterion. Table 3 also reports layers selected for high exchange flip rates, although Figure 5 transparently provides the full curves. These are reasonable exploratory choices, but their selected values should be interpreted accordingly. Moreover, the reported five-run standard deviations characterize training-related variation rather than uncertainty from sampling evaluation images. Some claimed category rankings have relatively small mean separations, such as CLIP ViT-L on VALSE actant-swap versus existence in Table 6. The appendices already provide substantial robustness evidence; the remaining concern is the distinction between an observed largest mean and a reliably larger effect. Clearer selection accounting and paired uncertainty estimates would improve confidence in the stronger comparative statements.

Questions for authors
  1. How well does the probe recover interaction partners when object categories alone cannot distinguish the alternatives? Figures 3(b) and 4 provide encouraging same-category examples. Could the authors quantify performance on an available subset containing multiple instances of the relevant categories, ideally including a simple category-and-geometry comparison or a matched distractor analysis? This question concerns the specificity of the binding interpretation in Critical Weakness 1. Consistent partner discrimination beyond those alternatives would substantially strengthen that interpretation. If the effect is limited to a small subset of examples, the conclusion should instead distinguish reliable directed-interaction decoding from more tentative instance-binding evidence.

  2. How much of the exchange effect remains after accounting for unequal object-region sizes and the induced global mean change? For the 816 examples used for each exchange type in Section 3.3, please report the distribution of exchanged-region patch-count ratios and examine how the preference change depends on that ratio. A result on approximately equal-size regions, or a control producing the same global mean change without exchanging the two region means, would directly address Critical Weakness 2. Persistence of donor-directed shifts under such a control would strengthen the role-reassignment interpretation. Strong dependence on region imbalance would instead indicate that aggregate feature redistribution contributes materially to the result. This could be examined using the existing examples without requiring a new benchmark.

  3. Does the learned-subspace intervention preserve object-identity evidence while changing relational preference? Because the captions in Section 3.3 differ in an object category, it would be useful to distinguish a change in which object fills a role from a change in the availability or strength of the relevant object-category representation. Could the authors provide a targeted object-identity diagnostic before and after intervention, or compare donor-directed relational changes with changes in object-only recognition or matching? Preserved identity evidence together with altered relational preference would support a more specific binding interpretation. Broad identity changes would not invalidate the intervention effect, but would require a narrower explanation of what it demonstrates.

  4. Are the learned and random interventions comparable in their actual perturbation magnitudes? Please report the feature displacement or removed within-image variation for the learned and random subspaces, preferably both in normalized coordinates and after denormalization. If these differ substantially, could a matched-magnitude comparison or a small intervention-strength sweep be provided? This question concerns Critical Weakness 3. A stronger role-sensitive effect at comparable displacement would make the specificity argument more convincing. If the difference largely disappears after matching intervention strength, the current random comparison would support importance of the selected directions more clearly than specificity to object-to-role binding.

  5. What do the complete paired preference transitions and continuous score changes show in the exchange experiment? The conditional flip rate in Table 3 is valid for asking whether an initially correct example can be redirected; it is not itself a population-level net change. Could the authors additionally report both directions of the preference transition and the paired change in the intended-caption versus donor-caption similarity margin? A comparison on a common initially correct subset would also help because learned and random controls can have different eligible subsets. This would clarify how broadly the exchange produces a donor-directed shift and help interpret the agent–patient differences. A uniformly positive net effect on every evaluated subset is not required, but the full transitions would determine how far the present conditional result can be generalized.

  6. Which choices were fixed before inspecting downstream effects, and how stable are the comparative conclusions under the reported alternatives? Please distinguish the selection of probe layers, subspace dimension, and exchange-reporting layers, including the role played by downstream ablation behavior in choosing k=16k = 16. The dimensionality and layer analyses already reported are useful; could the authors summarize which central conclusions hold across them and provide paired image-level uncertainty for the principal role-sensitive comparisons? Where multiple examples share an image, the uncertainty analysis should preserve that dependence. This would clarify whether the strongest reported values are exploratory maxima or representative effects, and whether close category rankings should be described as reliable differences or simply differences in reported means.

  7. Can a swapped negative also be an annotated positive under another retained directed interaction? Section 3.1 pools multiple predicates without providing the predicate as an input to the probe. Removing symmetric predicate types does not by itself guarantee that both directions of an object pair cannot be positive under different retained interactions. Please clarify whether swapped negatives are checked against the union of annotated positive directed pairs, and report the frequency of any such collisions after filtering. A negligible frequency, or an explicit filtering rule, would resolve the concern. If collisions are substantial, a filtered evaluation would help establish whether the probe’s accuracy and learned subspace are affected. The supplied material does not establish that such conflicts actually occur.

6 comments

Dear Reviewer 4HU3,

We thank the reviewer for the careful reading of our paper and for recognizing its contribution to understanding compositionality as well as our experimental results. We also appreciate the many helpful comments. We ran the suggested experiments and are delighted that they improved the paper considerably. Below, we address each comment.

[W1, Q1] How well does the probe recover interaction partners when object categories alone cannot distinguish the alternatives?

To show that our probe binds objects to roles without relying on category shortcuts, we computed probe accuracy on PSG val in which each negative patch has the same object category as the corresponding positive patch. That is, Swap is restricted to objects of the same category, and each replacing object in Agent-Rep, Patient-Rep, and Both-Rep must match the category of the corresponding original object. Because the positive and negative pairs have the same categories, these datasets cannot be solved from the typical combination of categories alone. Here are our results:

Table A: Probe accuracy (%) on PSG val with negatives of the same categories

ModelAvgSwapPatient-RepAgent-RepBoth-Rep
CLIP ViT-B69.0363.5267.7273.9870.91
CLIP ViT-L70.4263.7270.8272.7274.44
OpenCLIP ViT-L68.5261.5268.1071.4773.00
SigLIP70.7162.1273.3577.5669.82

Our probe remains well above the chance level of 50% in all four models and for every negative type. The probe therefore reads out role information that the combination of object categories cannot provide. The results are reported in Table 11 of Appendix D.

We also performed a simple category-and-geometry comparison. We compared our probe with simple rules that predict the agent from annotations alone. The category rule predicts the agent from the categories of the two objects. For example, for a person and a non-human animal, it predicts the person as the agent. For the geometry comparison, the position rules predict the agent from the vertical position, the horizontal position, or their combination of the two objects, using their centroids. For example, the horizontal rule predicts the object on the right as the agent. Here are our results:

Table B: Accuracy (%) on PSG val

MethodCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Ours84.2684.9883.6685.85
Category rule72.9372.9372.9372.93
Vertical55.9955.9955.9955.99
Horizontal49.7749.7749.7749.77
Joint position56.9556.9556.9556.95

Our probe outperforms all rules in all four models. The probe therefore reads out role–filler binding that category and geometry shortcuts cannot provide. Details are given in Appendix D.

[W2, Q2] How much of the exchange effect remains after accounting for unequal object-region sizes and the induced global mean change?

To check that the effect of the subspace exchange does not come from unequal object-region sizes, we evaluated the exchange on approximately equal-size regions (items where one region is at most twice as large as the other). We also added a control producing the same global mean change without exchanging the two region means, and compared it with the learned subspace exchange. In addition, we compared with the random subspace and with the random subspace whose perturbation size is matched to that of the learned subspace (rescaled). Here are our results:

Table C: Preference-flip rates (%) on approximately equal-size regions (Within 2×) and on all items

ExchangeModelItemslearnedrandomrescaledglobal mean
Agent-sideCLIP ViT-BWithin 2×24.192.077.760.19
All25.492.368.174.18
CLIP ViT-LWithin 2×49.960.885.421.24
All49.331.035.494.94
OpenCLIP ViT-LWithin 2×28.691.032.830.00
All25.581.723.941.44
SigLIPWithin 2×10.700.270.670.42
All11.230.601.341.11
Patient-sideCLIP ViT-BWithin 2×7.500.580.730.14
All11.580.861.911.37
CLIP ViT-LWithin 2×6.840.420.570.00
All14.580.351.601.09
OpenCLIP ViT-LWithin 2×5.510.270.830.41
All6.700.501.280.54
SigLIPWithin 2×1.520.610.150.00
All3.210.450.480.57

On the agent side, the flip rates of the learned subspace exchange on approximately equal-size regions stay close to those for all items. On the patient side, they are lower than those for all items, but they remain more than twice as high as those of every control in every model. The control producing the same global mean change gives far lower flip rates than the learned subspace exchange. Thus, the exchange effect does not depend on region imbalance, and the global mean change alone does not reproduce it, which supports a contribution of the learned subspace to binding objects to roles. See Appendix H for details.

[W3, Q4] Are the learned and random interventions comparable in their actual perturbation magnitudes?

We measured the feature displacement of the learned and random interventions for subspace ablation and exchange experiments, both in normalized coordinates, ∥Δxℓ,i∥2\|\Delta x_{\ell,i}\|^2, and after denormalization, ∥Δhℓ,i∥2\|\Delta h_{\ell,i}\|^2, averaged over the intervened patches. In normalized coordinates, the displacements of the two subspaces are similar but after denormalization, the displacement of the learned subspace becomes considerably larger than that of the random subspace for both ablation and exchange in every model.

We therefore provide a matched-magnitude comparison. We scaled the random subspace intervention to match the displacement of the learned subspace intervention after denormalization. We used it as another control in the subspace ablation and subspace exchange experiments. Even at comparable displacement, the random ablation lowers accuracy on the tasks that swap the roles of two objects much less than the learned ablation, and the random exchange flips the caption preference less often than the learned subspace exchange. These results show that the stronger role-sensitive effect of the learned subspace does not come from its larger feature displacement. Appendix F provides the full results.

[W4, Q6] Which choices were fixed before inspecting downstream effects, and how stable are the comparative conclusions under the reported alternatives?

Due to the response length limit, we have posted our full response to [W4, Q6] as a separate follow-up comment below.

[Q3] Does the learned-subspace intervention preserve object-identity evidence while changing relational preference?

To distinguish a change in which object fills a role from a change in the availability or strength of the relevant object-category representation, we evaluated the subspace exchange with two captions that both name all three objects aa, pp, and uu. For example, for agent-side exchange, the captions are "A photo of a {a} {r} a {p}. The photo also shows a {u}." and "A photo of a {u} {r} a {p}. The photo also shows a {a}." Because both captions mention all three objects, a change in object evidence can hardly affect the preference between them, and a change in role assignment is needed to flip it. We computed the preference-flip rate and compared the learned subspace with a random subspace. Here are our results:

Table D: Preference-flip rates (%) when both captions name all three objects

ExchangeSubspaceCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Agent-sidelearned25.1734.1516.905.93
random2.151.001.270.85
Patient-sidelearned12.0611.114.652.35
random1.951.710.970.86

The learned subspace gives higher preference-flip rates than the random subspace in every model and on both sides, most clearly for agent-side exchange. Because both captions name the same objects, these flips come from the exchange of role assignments, not from a change in the availability or strength of the object-category representation. See Appendix G for details.

To check whether object identity remains accessible after the exchange, we trained separate linear classifiers to predict the object categories from the full normalized patch representation xℓ,ix_{\ell,i}, the subspace coordinates Wℓxℓ,iW_\ell x_{\ell,i}, or the orthogonal component xℓ,i−Wℓ⊤Wℓxℓ,ix_{\ell,i}-W_\ell^\top W_\ell x_{\ell,i}, and measured their category classification accuracy. Here are our results:

Table E: Category classification accuracy (%) on PSG val

ModelLayerFull representationSubspace coordinatesOrthogonal component
CLIP ViT-B8 (agent side)65.750.664.1
9 (patient side)66.651.565.4
CLIP ViT-L1464.846.063.4
OpenCLIP ViT-L1362.543.861.0
SigLIP27 (agent side)58.338.856.9
12 (patient side)67.046.565.5

In every model, the classifier on the orthogonal component is almost as accurate as the classifier on the full representation. Because the exchange does not change the orthogonal component, this category information stays at the original patch locations. These results show that most of the category information remains at the original locations after the exchange, so object identity remains available after the intervention. Therefore, the learned subspace intervention preserves object-identity evidence while changing relational preference. Appendix G provides the full results.

Reviewer_sX1FOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether CLIP-style vision encoders represent which objects fill the roles of an interaction, despite their limited performance on captions with swapped roles. The authors train a low-rank bilinear probe on ordered patch pairs from the Panoptic Scene Graph dataset, identifying a 16-dimensional subspace that predicts annotated agent–patient pairs with approximately 83–86% accuracy across four encoders. They then test its functional relevance through two interventions. Mean-ablation produces the largest accuracy drops on role-swap tasks within SugarCrepe and VALSE, while exchanging subspace components between an interacting object and a third object can shift caption preference toward the corresponding replacement. These results suggest that role-related information is both decodable and used in image–text matching.

Critical strengths
  • The paper is well written and easy to follow. The research question is clear and well motivated: distinguishing information that is absent from representations from information that is present but insufficiently used is important for understanding compositional failures.

  • The combination of probing, ablation, and component exchange is a strong point. The paper does not rely only on supervised decoding accuracy, but also tests whether modifying the identified subspace changes the pretrained model’s behavior.

  • The ablation results show an interesting and consistent pattern. Across all four models, the largest mean accuracy drop occurs on SugarCrepe swap-object and VALSE actant-swap within their respective benchmarks. Selecting intervention layers using PSG validation accuracy rather than the target benchmark results also strengthens this evaluation.

  • The exchange experiments provide useful complementary evidence. For example, CLIP ViT-L reaches a 49.33% agent-side preference-flip rate, compared with 1.03% for the random-subspace control at the same layer. The within-object averaging control, five-run evaluation, and explicit reporting of item counts make these results easier to assess.

Critical weaknesses
  1. My main concern is how specifically the probe identifies relational role binding rather than category and interaction shortcuts. The probe is trained without conditioning on the predicate, and negatives are obtained through role reversal or random replacement with other objects. Object categories, typical agent–patient combinations, proximity, and pose can therefore provide useful signals. Excluding explicitly spatial predicates does not remove these correlations. The examples with multiple riders and motorcycles are useful evidence against a simple category-only explanation, but the main quantitative evaluation does not isolate this setting. Category-matched hard negatives, simple category/position baselines, or a controlled evaluation with multiple same-category interaction pairs would help establish how much performance requires identifying the correct relational partner.

  2. The exchange experiment does not fully isolate moving role assignments from moving object identity or other semantic information. Section 3.3 assumes that the unchanged components retain information about what each object is, but this separation is not directly established. Since the third object must have a different category, exchanging components could alter the category evidence at the interacting object’s location and thereby favor the replacement caption. The selective ablation results provide complementary support for role-related information, but the exchange interpretation would be stronger with checks that object identity remains accessible and localized correctly after intervention. Comparisons with an object-category or same-object-binding subspace would also help clarify what is specific to role binding.

  3. The random-subspace control matches dimension, but not necessarily the amount of information removed or the magnitude of the intervention. Per-dimension standardization does not remove correlations between dimensions, so a learned subspace can contain substantially more variance than a random subspace of the same rank. Its ablation or exchange may consequently produce larger changes in the residual stream. This does not explain away the selective effects on role-swap tasks, but it leaves uncertainty about how much of the learned-versus-random difference reflects the content of the subspace. Reporting perturbation norms and removed variance, together with an appropriately matched control, would strengthen the intervention evidence.

  4. The interpretation of the agent–patient asymmetry needs to account for differences in the initial caption preferences and the evaluated subsets. Flip rates are conditioned on the no-exchange control preferring the original caption, and this set can differ across layers and subspaces. Table 3 helpfully reports the counts, but also shows substantial differences between exchange types: for CLIP ViT-L, the original caption is preferred without intervention on 69.49% of agent-side items versus 83.33% of patient-side items. The corresponding score-margin distributions may also differ, making one direction easier to flip. This does not remove the evidence for an intervention effect, but a comparison on a common eligible subset and an analysis of continuous caption-margin changes would make the claimed role asymmetry more interpretable.

Questions for authors

I would appreciate it if the authors could elaborate further on the weaknesses above.

5 comments

Dear Reviewer sX1F,

Thank you for your review and for your positive comments about the motivation, importance, and experiments of our work. We ran the experiments you suggested, and they helped us address weak points in the paper. We answer your comments one by one below.

[W1] My main concern is how specifically the probe identifies relational role binding rather than category and interaction shortcuts.

To address this concern, we ran the three analyses you suggested: an evaluation on category-matched hard negatives, a comparison with simple category/position baselines, and an evaluation on scenes with multiple same-category interaction pairs.

Category-matched hard negatives. We computed probe accuracy on comparisons in which each negative patch has the same object category as the corresponding positive patch. That is, Swap is restricted to objects of the same category, and each replacement object in Agent-Rep, Patient-Rep, and Both-Rep must match the category of the corresponding original object. Table A shows the results.

Table A: Probe accuracy (%) on category-matched hard negatives

ModelAvgSwapPatient-RepAgent-RepBoth-Rep
CLIP ViT-B69.0363.5267.7273.9870.91
CLIP ViT-L70.4263.7270.8272.7274.44
OpenCLIP ViT-L68.5261.5268.1071.4773.00
SigLIP70.7162.1273.3577.5669.82

Our probe remains well above the chance level of 50% in all four models and for every negative type. The probe therefore reads out role information that category shortcuts cannot provide. Details are given in Appendix D.

Simple category/position baselines. We compared our probe with simple rules that predict the agent from annotations alone. The category rule predicts the agent from the categories of the two objects. For example, for a person and a non-human animal, it predicts the person as the agent. The position rules predict the agent from the vertical position, the horizontal position, or their combination of the two objects, using their centroids. For example, the horizontal rule predicts the object on the right as the agent. Table B shows the results.

Table B: Accuracy (%) in PSG val

MethodCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Ours84.2684.9883.6685.85
Category rule72.9372.9372.9372.93
Vertical55.9955.9955.9955.99
Horizontal49.7749.7749.7749.77
Combination56.9556.9556.9556.95

Our probe outperforms all rules in all four models, so its accuracy cannot be explained by category and position shortcuts. Details are given in Appendix E.1.

Multiple same-category interaction pairs. To examine scenes such as the multiple riders and motorcycles, we selected positive pairs whose image also contains another pair with the same agent and patient categories, and measured probe accuracy on these pairs.

Table C: Probe accuracy (%) on the selected pairs and on all pairs on PSG val

PairsCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Selected pairs76.577.775.981.0
all pairs84.385.083.785.9

In such scenes, the accuracy of our probe is lower than on the full PSG val, but it remains well above the chance level of 50% for every model. This shows that the role-binding representation keeps the probe accurate even in complex scenes. Details are given in Appendix E.2. Together, these results show that the probe reads out more than category and interaction shortcuts, namely relational role binding.

[W2] The exchange experiment does not fully isolate moving role assignments from moving object identity or other semantic information.

To show that the unchanged components retain information about what each object is, we trained separate linear classifiers to predict the object categories from the full normalized patch representation xℓ,ix_{\ell,i}, the subspace coordinates Wℓxℓ,iW_\ell x_{\ell,i}, or the orthogonal component xℓ,i−Wℓ⊤Wℓxℓ,ix_{\ell,i}-W_\ell^\top W_\ell x_{\ell,i}. We used the exchange layers, where the preference-flip rate is highest. When the agent side and patient side exchanges peak at different layers, we report both layers, marked "agent side" and "patient side". Here are our results:

Table D: Category classification accuracy (%) on PSG val

ModelLayerFull representationSubspace coordinatesOrthogonal component
CLIP ViT-B8 (agent side)65.750.664.1
9 (patient side)66.651.565.4
CLIP ViT-L1464.846.063.4
OpenCLIP ViT-L1362.543.861.0
SigLIP27 (agent side)58.338.856.9
12 (patient side)67.046.565.5

In every model, the classifier on the orthogonal component is almost as accurate as the classifier on the full representation. Because the exchange does not change the orthogonal component, this category information stays at the original patch locations. We also found that the subspace coordinates contain some category information, although with lower accuracy. These results show that most of the category information remains at the original locations after the exchange, so object identity stays accessible and correctly localized. Further details are provided in Appendix G.

To address the concern that exchanging components could alter the category evidence at the interacting object's location and thereby favor the replacement caption, we also evaluated the exchange with two captions that both name all three objects aa, pp, and uu. For example, for agent-side exchange, the captions are "A photo of a {a} {r} a {p}. The photo also shows a {u}." and "A photo of a {u} {r} a {p}. The photo also shows a {a}." Because both captions mention all three categories, a change in category evidence can hardly affect the preference between them, and a change in role assignment is needed to flip it. We computed the preference-flip rate and compared the learned subspace with a random subspace. Here are our results:

Table E: Preference-flip rates (%) when both captions name all three objects

ExchangeSubspaceCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Agent-sidelearned25.1734.1516.905.93
random2.151.001.270.85
Patient-sidelearned12.0611.114.652.35
random1.951.710.970.86

The learned subspace gives higher preference-flip rates than the random subspace in every model and on both sides, most clearly for agent-side exchange. Because both captions mention the same three objects, these flips cannot be explained by a change in category evidence. Most of the category information lies in the orthogonal component, which the exchange does not change, and this task is insensitive to any category information that the exchange does move. The high flip rates under these conditions therefore support a contribution of the subspace to binding objects to roles. See Appendix G for details.

[W3] Reporting perturbation norms and removed variance, together with an appropriately matched control

To test whether the effect of the learned subspace comes from the magnitude of the intervention, we scaled the random subspace intervention to match the magnitude of the learned subspace intervention in the residual stream. Even at the same magnitude, the learned subspace has a much larger effect than the random subspace. Ablating the learned subspace lowers accuracy on the tasks that swap the roles of two objects much more. These results show that the consistent effects of the learned subspace do not come from the magnitude of the change in the residual stream. The results are reported in Appendix F.

[W4] The interpretation of the agent–patient asymmetry needs to account for differences in the initial caption preferences and the evaluated subsets.

To interpret the agent–patient asymmetry, we analyzed continuous caption margin changes in the subspace exchange. We computed the caption margin before the subspace exchange and its shift under the subspace exchange, over all items in PSG val. Each item has two captions: the original caption, such as "a person riding a horse", and a caption in which one object word is replaced with another object in the image, such as "a person riding a bicycle". The caption margin is the gap between the image's similarity to the original caption and its similarity to the other caption. A positive margin means that the model prefers the original caption. Here are our results:

Table F: Caption margin before the subspace exchange and its shift under the subspace exchange (values × 1,000)

ExchangeQuantityCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Agent-sideCaption margin22.8213.9413.911.49
Shift−14.88−26.53−16.65−3.65
Patient-sideCaption margin40.0045.9756.7541.16
Shift−13.85−18.92−12.73−2.98

The subspace exchange lowers the caption margin in every model and in both exchanges. In other words, after the exchange, the image becomes closer to the caption with the exchanged object, such as "a person riding a bicycle". Before the subspace exchange, the margin favoring the original caption is larger on the patient side, whereas the shift is larger on the agent side. Patient-side items therefore keep the original caption more often, which suggests that the asymmetry in the initial margins contributes to the agent–patient asymmetry in flip rates. Further details are provided in Appendix I.

Reviewer_MgEwOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This work studies whether vision transformers understand agent and patient relations between objects in a scene. It builds on prior work that found there is information in the embeddings indicating which two patches belong together. This project extends it further and asks whether vision transformers understand which patches belong to the object acting as the agent and which patches belong to the object acting as the patient in the asymmetric relationship. The authors train bilinear probes on the Panoptic Scene Graph dataset and show that at certain layers of different models, the probes achieve accuracies around 80%. The probing scores can be used for visualizing agent patches or patient patches in relation to some object's representation. The authors also define a relation-binding subspace learned from the probes. Performing mean ablation on this subspace reduces different models' performance on the SugarCrepe and VALSE benchmarks. Interventions that swap subspace components between two objects in an image flip the scores of what a model chooses as the closest caption. These experiments further support the authors' claim that the patch representations in CLIP-style models hold the information about the agent-patient relationship between objects.

Critical strengths

This work explores an interesting question: whether vision transformers 'understand ' the role-filler (agent-patient) relations. The authors find that vision transformers contain information about such relations, which might be surprising because VLMs are known to struggle to distinguish them. Overall, this work extends findings in prior literature that the embeddings might contain more information but are not properly utilized in CLIP-style models. The authors confirm their findings with several experiments (probing accuracy, mean ablation, and interventions) and provide intuitive qualitative visualizations.

Critical weaknesses

Limited evaluation. My main concern is a potential bias in the evaluations. Even though it seems like probes work well in identifying the agent-patient relationships, I wonder if there could be a natural bias that inflates the results. For example, it is more likely to see the relation 'baby holding an apple' than 'an apple holding a baby'. If the same probes the authors train are applied to the counterfactual (compositional yet OOD) data, would performance be the same? I also think the existing evaluations are somewhat limited: the authors consider one dataset and few models. Broadening the evaluations would significantly strengthen the authors' claims.

Explanation for the probe choice. This work follows closely Li's probing idea. However, compared to that work, the choice of bilinear probe is not fully explained or ablated. Is the proposed probing method the best choice for agent-patient relations, or could there be other alternative formulations?

Clarity and presentation. I also have some concerns about the clarity of explanations. For example, Figure 1 aims to explain what the probing score describes and what the positive or negative relation types are. This is an important figure, but it requires some time to process. Since the authors show 2 agent-patient relations, I suggest annotating them as (a1, p1) and (a2, p2) and then annotating each negative type like (a1, p2). Use the same notation in the paragraph at line 171. Similarly, at Line 353, using c both as an annotation for the third unrelated object and as a mean representation is confusing. I suggest the authors use a different letter for the unrelated object. I think working on these explanations could help readers better understand the work.

Questions for authors

Clarification questions:

  1. Are the probes used in Sections 3.2 and 3.3 the same as the ones trained on PSG?
  2. At Line 377, could the authors elaborate on the significance and the function of the no-exchange control? In Table 3, why is the preference-flip rate defined with respect to the no-exchange control and not the no-intervention?
  3. At Line 190, the authors mention that they probe over 5 seeds. However, Figure 2 only shows lines. Do the lines represent the average results over the seeds? Could the authors show the bands/ribbons?

Other questions:

  1. How does the model perform when one patient has several agents and when one agent has several patients? Does the probing performance degrade with increasing scene complexity?
  2. I wonder about the relationship between the authors' findings and text representations. Mean ablation and interventions show that the agent-patient relation between patches affects the results, but how? Can we use text representations instead of mean patch representations for finding regions of the agent or patient? For example, if the text says, 'a baby is holding ...', would there be a high probing score with patches belonging to an apple?
7 comments

Dear Reviewer MgEw,

Thank you for reviewing our paper and for recognizing our finding that Vision Transformers encode role–filler binding information. Your comments led us to run several additional experiments, and we believe the paper is considerably stronger as a result. We address each comment below.

[W1] Limited evaluation

To isolate the natural bias in object categories, we computed probe accuracy on synthetic datasets in which each negative patch has the same object category as the corresponding positive patch. That is, Swap is restricted to objects of the same category, and each replacing object in Agent-Rep, Patient-Rep, and Both-Rep must be of the same category as the corresponding original object. Because the positive and negative patches have the same categories, these datasets cannot be solved from the natural bias in object categories. Table A shows the results.

Table A: Probe accuracy (%) on PSG val with negatives of the same categories

ModelAvgSwapPatient-RepAgent-RepBoth-Rep
CLIP ViT-B69.0363.5267.7273.9870.91
CLIP ViT-L70.4263.7270.8272.7274.44
OpenCLIP ViT-L68.5261.5268.1071.4773.00
SigLIP70.7162.1273.3577.5669.82

Our probe remains well above the chance level of 50% in all four models and for every negative type. The probe therefore reads out role information that object categories alone do not provide. The results are reported in Table 11 of Appendix D.

We also applied our probe trained on PSG to counterfactual images, such as an apple holding a baby. The probe responses follow the agent–patient assignments shown in the images, wiht the apple detected as the agent and the baby as the patient. Details are given in Appendix C.

[W2] Explanation for the probe choice

The probes of Li et al. decode an undirected relation, whereas agent–patient relations are directed. In self-attention, the query–key score q⊤k=x⊤WQ⊤WKyq^\top k=x^\top W_Q^\top W_K y is a bilinear form in the two input representations. It is therefore natural to read out directed relations with a bilinear probe (Wx)⊤V(Wy)(Wx)^\top V(Wy) whose VV is left unconstrained, so that it can distinguish (x,y)(x,y) from (y,x)(y,x).

To further support this choice, we compared our probe with three other probes. The linear probe uses u⊤x+v⊤yu^\top x+v^\top y and has no interaction between the two patches. The low-rank quadratic probe uses (W1x)⊤(W2y)(W_1x)^\top(W_2y), so it also projects the patches to kk dimensions, but the two roles need not share a subspace. The full-rank quadratic probe uses x⊤Ayx^\top Ay without a rank constraint. We evaluated all probes on PSG val with unrestricted negatives and with negatives of the same categories in [W1]. Here are our results:

Table B: Probe accuracy (%) on PSG val with unrestricted negatives

ProbeCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Shared low-rank quadratic probe (Ours)84.2684.9883.6685.85
Low-rank quadratic probe83.4984.1883.1085.55
Full quadratic probe83.9385.2584.1883.31
Linear probe84.6485.7684.7884.71

Table C: Probe accuracy (%) on PSG val with negatives of the same categories

ProbeCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Shared low-rank quadratic probe (Ours)69.0370.4268.5270.71
Low-rank quadratic probe68.5669.8068.2769.92
Full-rank quadratic probe67.8470.2868.1066.10
Linear probe63.9964.9662.3862.73

The linear probe performs well with unrestricted negatives, but because it relies on category information, its accuracy drops when category information cannot be used, and our probe becomes the most accurate. These results support our choice of this probe. Details are given in Appendix D.

[W3] Clarity and presentation

Following your suggestions, we revised Figure 1 and the text to use indexed notation for agents and patients, as in (a1,p1)(a_1,p_1), and renamed the third object from cc to uu (see Section 3.1, 3.3 and Appendix A.2).

[Q1] Are the probes used in Sections 3.2 and 3.3 the same as the ones trained on PSG?

Yes. Sections 3.2 and 3.3 use the probes trained on PSG in Section 3.1. We added explanation to Section 3.2 and 3.3.

[Q2] Purpose of the no-exchange control

Because the two objects can differ in size, the exchange cannot swap their patches one by one. Instead, it gives each object the mean of the subspace coordinates over the patches of the other object. The exchange thus combines two operations: replacing the subspace coordinates of each object with their mean over its patches, and exchanging these means between the two objects. The no-exchange control applies only the first operation, so comparing the exchange with this control isolates the effect of exchanging the means of the subspace coordinates. Comparing the exchange with no intervention would mix the two effects. We added this explanation to Section 3.3.

[Q3] Do the lines represent the average results over the seeds? Could the authors show the bands/ribbons?

Yes, the lines show means over five seeds. Figure 2 already includes bands of ±1 standard deviation, but they are hard to see because the variation across seeds is very small. For each model, we now reports the mean and maximum of the standard deviations across layers.

Table D: Standard deviation of probe accuracy across five seeds

Standard deviationCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Mean across layers0.160.190.190.22
Maximum across layers0.230.340.280.44

[Q4] Does the probing performance degrade with increasing scene complexity?

We examine whether the probe still works when an object has several partners, for example, a single patient with several agents or a single agent with several patients. From PSG val, we select pairs (a,p)(a,p) of three groups: (i) an isolated pair of a single agent and a single patient, (ii) a single agent with several patients, and (iii) several agents with a single patient. We then evaluate the probe accuracy on each group.

Table E: Probe accuracy (%) for each group in PSG val

GroupCLIP ViT-BCLIP ViT-LOpenCLIP ViT-LSigLIP
Single agent, single patient84.8285.5884.2386.48
Single agent, several patients81.7484.7183.8084.91
Several agents, single patient79.5075.8773.9678.89

Accuracy decreases with increasing scene complexity, but it remains well above the chance level of 50% in all four models. These results show that the assignment of objects to roles remains decodable even in more complex scenes. Details are given in Appendix E.3.

[Q5] I wonder about the relationship between the authors' findings and text representations.

To our knowledge, this is the first work to probe role binding in Vision Transformers. Our findings indicate at which layers of the Vision Transformer these representations appear and are used, and they provide a starting point for interpretability studies that trace how such representations emerge. Our probe takes image representations as input and cannot use text representations directly. Studying role binding between image and text tokens in models that process both in one language model is also important, but it requires a separate study, so we leave it for future work.

I thank the authors for their detailed response. I think the clarity of the explanations has improved. I now understand better the significance of the no-exchange control. The results for increasing scene complexity and different probe options are interesting. And it makes sense to leave interaction with text representations for future work.

I have remaining questions for the same-category and counterfactual evaluations:

  1. Could the authors provide a few concrete same-category examples? I also noticed that probe accuracies drop substantially from unrestricted to same-category evaluations. Could the authors discuss whether this primarily reflects reliance on object-category priors or some other reasons? Given the magnitude of the drop, I think it would be useful to briefly discuss these findings in the main paper.

  2. Could the authors clarify how to interpret Figure 11? Each image seems to be repeated with reversed role assignments rather than showing a different scene. Also, probe predictions appear less clearly localized to the corresponding agents/patients compared to Figure 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.