acceptodds
Withdrawn from ICLR 2027

Intuition vectors

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large self-supervised vision models learn representations that support scene segmentation and the semantic decomposition of physical objects. We ask whether their representational geometry supports transfer to visual reasoning problems without any task-specific fine-tuning. We hypothesized that relational representations may bridge perception and abstract reasoning by encoding similarities and transformations among visual inputs such that an intuitive, implicit form of reasoning may be performed via latent vector arithmetic. Specifically, we examine DINOv3, MAE, and random pixel projections on abstract and naturalistic Bongard problems, ARC-AGI-1 and ARC-AGI-2, and novel ARC-GEN instances. On both Bongard benchmarks, the accuracy of a simple nearest-centroid readout of frozen visual embeddings is within four percentage points of the task-specific baselines reported with the original benchmarks. In ARC, latent difference vectors summarizing demonstration input–output transformations, which we refer to as *intuition vectors*, show greater alignment with test vectors from the same task, whereas those from unrelated tasks are near orthogonal. This latent geometry is operational: transporting a query along its intuition vector consistently improves exact-output retrieval, reaching on ARC-AGI-2 evaluation. Across 397,000 ARC-GEN instances from 794 tasks, single-pair intuition vectors identify the generating task with approximately 87% leave-one-out accuracy. These findings suggest that latent vector arithmetic over frozen visual representations supports implicit rule inference across varied problem domains without a generative model component, indicating that inferring an abstract transformation and generating its instance-specific consequence may be separable capacities.

Primary area:
interpretability and explainable AI
Keywords:
Relational representations, abstract visual reasoning, pretrained vision models, representation geometry, ARC-AGI, Bongard problems

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_BxJ1OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors use nearest-centroid classificationAuthors derived the "intuition vector" as the difference of positive and negative support centroids, which they then used to retrieve the corresponding answer for query images using a nearest neighbour retrieval.

Critical strengths

The intuition vector of answer = query + (prototype out - prototype in) matches the queen = king + (woman - man) concept. Although concepts are well established, it's interesting to see it applies on ARC dataset.

illustrations and figures help with understanding.

Critical weaknesses

Theoretical grounding is weak. What is this "structure" in "systematic geometric organization", "plethora of evidence for emergent, informative structure across ...", "structure is coherent and functionally meaningful", etc?

The paper is difficult to understand with all the jargon. The analyses do not clearly explain why the method works or the assumptions it makes. Why are we computing prototype vectors and doing vector algrebra with it?

Only two models are investigated. CLIP trained vision encoders have been shown to give better results on Bongard tasks when using the same prototype vector methodology [1]. Is this "best published average" really the best?

The analysis on page 7 onwards, for the most part, is unclear, and should be in the appendix with only clear takeaways in the main text. The claim "representing a transformation and executing its consequence thus appear to be distinct capacities" is not sufficiently supported. Comparing classification against retrieval on pixel spaces (values) or embeddings does not support that. To do so, one should first define what "representing a transformation" and "executing a consequence" are. Only claim I can see this supporting is that dataset can't be solved using pixel similarity, supporting "beyond pixel similarity" claim, though, here, similarity is only one metric and doesn't prove solving by some measure/approach is not possible.

[1] Vompa et al., 2026, Beyond the Linear Separability Ceiling: Aligning Representations in VLMs

Questions for authors

Could you add a list with contributions so it's easier to evaluate the work done?

Reviewer_CUpGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether frozen self-supervised vision representations support visual reasoning through simple geometric operations. It evaluates nearest-centroid classification on Bongard problems and defines an ARC intuition vector as the average support-output embedding minus support-input embedding. The analysis measures alignment with held-out transformations, retrieval after adding the vector to a query embedding, and leave-one-out identification of ARC-GEN tasks from single-pair difference vectors. Experiments compare DINOv3, MAE, and random-projection pixel representations, and include generated instances, perturbed candidates, and exploratory local-composition analyses. The paper reports substantial retrieval gains and approximately 87% task-identification accuracy, while explicitly separating output retrieval from full ARC output generation.

Critical strengths

The paper tests whether the geometry has an operational effect, with substantial validation on new instances. Adding the support-derived vector improves retrieval over the untranslated query. The reported DINOv3-224 results increase from 32.0% to 63.2% on ARC-1 evaluation and from 56.3% to 70.7% on ARC-2 evaluation. On ARC-GEN, the analysis covers 79,400 generated query–answer pairs from 794 tasks against 368,442 unique candidate outputs, with encoder layers fixed using the original ARC data. Transport gains of 11.6–21.5 percentage points provide meaningful evidence that the support-derived statistic contributes useful predictive information.

The comparison spans two encoders, two resolutions, random pixel projections, cross-task and shuffled-output controls, full layer profiles, and near-miss candidates. The paper also acknowledges the strong absolute pixel-retrieval performance and shows that local addition vectors can be directionally aligned while failing magnitude additivity. These findings help define the limits of a simple global linear-transformation account.

Critical weaknesses
  1. The proposed vector is invariant to permutations of the support input–output pairing. This is compatible with useful latent translation, but it makes the interpretation of the measured gains important. Comparing transported queries with untranslated queries establishes the usefulness of the support statistic. It does not establish how much of that usefulness comes from locating the task’s output distribution. The paper does not report retrieval using the support-output centroid alone. Similarly, the approximately 87% leave-one-out task-identification result uses centroids constructed from many other instances of each generating task, without input-only or output-only identification controls. Stable endpoint appearance could therefore explain part of the task separation.

  2. [1] uses pretrained visual embeddings to compute E(B)+E(A′)−E(A)E(B)+E(A')-E(A) and retrieve an answer candidate. [2] investigates combining ARC example differences by averaging and adding the result to a query embedding, using an ARC-trained VAE. These close precedents are not discussed or compared.

[1] Bitton et al., VASR: Visual Analogies of Situation Recognition (2023)

[2] Thoms et al., Solving ARC visual analogies with neural embeddings and vector arithmetic: A generalized method (2023)

Questions for authors
  1. Under the same layers and candidate pools, how does transported-query retrieval compare with retrieval using only the support-output centroid μy\mu_y?

  2. Under the identical leave-one-out protocol, what are the task-identification accuracies of E(x)E(x), E(y)E(y), and E(y)−E(x)E(y)-E(x)? Does the difference representation retain an advantage when tasks or instances are matched more closely for simple appearance statistics?

  3. Do the main retrieval gains and Bongard comparisons persist when the layer is selected on separate training/validation tasks and then fixed for evaluation?

Reviewer_XrVvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes intuition vectors, which are essentially the difference between the answer embedding and the input embedding. They show that one can use intuition vectors to infer correct outputs on two self-supervised vision models and two datasets.

Critical strengths

The idea of finding a model embedding that can yield correct outputs is interesting.

Critical weaknesses

The overall writing needs improvement. The experiments are limited.

Questions for authors
  1. It would be appreciated if the authors could clarify the contributions, novelties, and main findings of this paper.

  2. The experiments conducted in this paper are very limited. The paper claims to solve abstract reasoning for self-supervised vision models. But they only examine two models and two datasets, and they only focus on Bongard problems.

  3. Can you explain why you only use these two models: DINOv3-Large/16 and ViT-MAE-Large/16 encoders? Can you explain why you only use these two datasets: ARC-1 and ARC-2?

  4. L177 writes that the proposed intuition vector implicitly represents the underlying transformation rule. Then, what if a dataset contains various tasks that require different inference rules? Is this single intuition vector still capable of representing the underlying transformation rule? A broader question is: how is the performance of the proposed intuition vector on OOD tasks?

  5. What does the “held-out pair” in L180 refer to? And what does the “cell-shuffled answer grids” in L181 mean?

  6. Throughout the paper, many parameters are directly specified in the method section, which seems to be some random choices without justification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.