acceptodds
Under review as a conference paper at ICLR 2027

PRISM: Progressive Reasoning through Iterative Slot Memory for Vision

Abstract

Modern vision models process images in a single feed-forward pass, which limits their ability to recover missing evidence or refine uncertain representations under incomplete observations. Inspired by the iterative nature of human perception, we introduce **PRISM** (**P**rogressive **R**easoning through **I**terative **S**lot **M**emory), a pyramid vision architecture that reasons over images through iterative refinement. At a high level, PRISM groups visual features into object-centric representations, retrieves relevant patterns from a learned memory, and iteratively refines the representation to resolve ambiguity and recover missing information. This organize–recall–refine process operates recurrently across multiple scales, enabling progressive improvement of visual representations. Across standard vision tasks, including image classification, object detection, and semantic segmentation, PRISM achieves competitive performance while demonstrating improved robustness under incomplete observations such as occlusion. These results suggest that iterative reasoning with structured representations and memory is a promising direction for building more resilient and adaptive vision models. Source code and models will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Iterative visual reasoning, memory-augmented vision, occlusion robustness.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wmBLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to enhance the robustness of visual models under occlusion when inferencing. The authors propose a method to reconstruct occluded parts by leveraging prior memory of complete objects through iterative inference. The PRISM method integrates three established components into a unified backbone. Slot attention aggregates image features into a set of slots, effectively decomposing the scene into a structured representation rather than relying on a dense grid of tokens. Slot-level VQ-VAE applies standard VQ-VAE vector quantization at the slot level. During inference, each slot is used to retrieve its nearest prototype, then incorporate the prototype information back into the representation. ACT dynamically determines the number of recurrent steps based on the complexity of the input image. The author demonstrates the effectiveness of this method through various experiments and ablation studies.

Critical strengths
  1. Occlusion robustness is a real problem in the field of vision and is often overlooked by mainstream benchmarks. Most SOTA methods compare accuracy on clean ImageNet, while in real-world deployments, objects occluding each other is almost the norm.
  2. The experiments cover classification, detection, and instance segmentation. A benchmark for occlusion has also been designed, featuring five distinct types to stress-test the models from various perspectives.
  3. The authors provide visualizations in the appendix, showing how attention gradually focuses on the object and suppresses the obstructions in each loop, moving from an occluded state. This demonstrates the effectiveness and interpretability of this method.
  4. The correspondence between the motivations mentioned by the authors and the design of their methodological framework is very clear. The intuitive approach of filling in the obscured parts based on memory, combined with repeated deduction, was precisely translated into three modules.
Critical weaknesses
  1. The Slot Attention, VQ-VAE, and ACT components mentioned in the methodology section of the article are all existing methods. The PRISM's main contribution is to combine them into the backbone and make a loop. This leads the reviewer to question whether the work offers sufficient substantive contribution and innovation?
  2. All experiments are limited to the tiny scale, and the models used for comparison are also small models. It remains unclear whether the robustness gains provided by VQ prototype memory will persist or vanish as the model size increases?
  3. The test datasets consist entirely of synthetic occlusions. These types of occlusions produce pixel statistics that fall outside the original distribution, rather than representing realistic occlusions. Are there any test results based on real-world datasets?
  4. Table A7 shows that MAE ViT-B outperforms PRISM across all metrics, though this may be due to the fact that MAE has 86M parameters. However, if the number of parameters were held constant, would PRISM be able to surpass MAE?
  5. The loss weight of ACT ponder = 0.005 is a small value that it raises doubts about the true contribution or influence of ACT to the overall loss calculation process? Furthermore, the ACT is unstable during training, and the paper does not provide any sensitivity analysis.
Questions for authors
  1. Could you add a variant that applies the same VQ memory to (a) dense token features or (b) a single globally pooled feature instead of object-centric slots?
  2. Table A7 compares an 11M model against an 86M MAE that also received 800 epochs of self-supervised pretraining. Could you report a MAE at a comparable scale (ViT-Ti/16 at ~5M, or a width-reduced MAE at ~11–13M) under the same protocol?
  3. Could you add at least one evaluation with real occlusions or an existing real-occlusion benchmark such as Occluded-ImageNet or COCO-O if applicable?
Reviewer_SUmBOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes PRISM, which employs slot attention, prototypical memory, and adaptive iteration to progressively update image features, thereby enhancing recognition capabilities when visual information is incomplete. Experimental results demonstrate that this method achieves strong performance across image classification, detection, and segmentation tasks, while also exhibiting robust performance under conditions of artificial occlusion.

Critical strengths
  1. By integrating slot attention, prototypical memory, and adaptive recurrence, this approach provides a unified framework for the aggregation and iterative updating of incomplete visual information.

  2. The method yields performance improvements across classification, detection, and segmentation tasks under scenarios involving artificial occlusion; furthermore, results from a VQ-free baseline (trained from scratch) provide additional evidence for the contribution of prototypical memory to model robustness.

Critical weaknesses
  1. The existing experimental results are insufficient to prove that VQ retrieval actually recovers missing object information; furthermore, they fail to distinguish between the gains derived from prototype recovery and those resulting from quantization regularization.

  2. The fixed-step experiments primarily involve altering inference settings within a dynamic model. The absence of fixed-iteration models, trained independently under an equivalent computational budget, makes it difficult to ascertain the specific contribution of the adaptive stopping mechanism.

  3. The experiments rely predominantly on synthetic patches and rectangular occlusions, lacking validation in scenarios involving real-world occlusions. Consequently, it remains unclear whether the approach can generalize to complex occluders and natural scenes.

  4. There are inconsistencies in the reported experimental results, and the information provided is insufficient for reproduction. For instance, the instance segmentation score is cited as 41.1 in the main text, yet it appears as 40.8 in Table 2. Additionally, the semantic segmentation experiments do not specify the framework employed or the key training configurations used.

  5. Robustness gain comes solely from VQ. In Table A3, the variant trained without VQ keeps slot attention, recurrence and ACT. On PatchMask0.8 it falls below PVTv2-B1 (37.2 vs. 39.6), and on BlockMask112 (75.2 vs. 74.6) and OnlyOne112 (63.1 vs. 62.8) it is roughly on par. Recurrence and slot grouping therefore bring no measurable robustness on their own. This leaves the paper’s central narrative unsupported: the “progressive reasoning” in the title, the recurrent organize–recall–refine mechanism, and the claim that object-centric representations improve robustness.

  6. Limited novelty. MetaSlot has already applied vector quantization to slot representations, which is the core idea of this paper, yet it is discussed only in the appendix. Hierarchical backbones that group tokens into slots or clusters also already exist, e.g., PaCa-ViT and ClusterFormer, both cited in the paper. The halting mechanism is standard ACT. Taken together, the method is mainly a combination of existing components. Since the robustness gain comes only from slot-level VQ (point 3), the actual technical contribution beyond MetaSlot is unclear.

Questions for authors
  1. How can it be demonstrated that the gains from VQ stem from "recovering missing information" rather than from the regularization effect introduced by quantization?

  2. Does the adaptive iteration scheme still outperform a fixed iteration scheme when both are constrained by the same computational budget?

  3. Could you clarify the inconsistencies regarding the experimental setup and the reported numerical values? Please specify the framework and training configuration used for semantic segmentation. Additionally, please explain why the instance segmentation results are reported as 41.1 in the main text but as 40.8 in Table 2, and clarify the discrepancy in the average number of iterations for "Stage 3" between the main text and the efficiency analysis.

Reviewer_K7aoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PRISM is a four-stage pyramid backbone in which each stage holds a single weight-shared block applied recurrently. Each recurrent step performs an organize–recall–refine cycle: slot attention groups patch tokens into 8 slots using position-modulated keys; the grouped key/value pair passes through a per-stage VQ codebook of 64 prototypes; cross-attention writes the recovered information back to spatial positions via two independent FFNs. An ACT head emits a single global halting probability, making the iteration count input-dependent (T_max = 5). Trained from scratch for 300 epochs on ImageNet-1K and transferred to COCO and ADE20K, PRISM reports 80.3% clean top-1 at 11M parameters and substantially stronger accuracy under five synthetic occlusion regimes, with no occluded images seen during training.

Critical strengths

Robustness gains are substantial across tasks. Under PatchMask0.6, PRISM reaches 69.1% accuracy versus BiFormer-T’s 60.4%; On occluded ADE20K, mIoU is 39.7 versus PVTv2-B1’s 34.8 These results support practical value within the evaluated synthetic-occlusion settings.

Critical weaknesses
  • Figure 3B's metric cannot be compared across models. Feature similarity is computed as cosine within each model's own feature space. Quantizing to ~26–41 centers mechanically raises the similarity between any two inputs, including a clean/occluded pair, with no information recovery required. The figure therefore cannot distinguish "PRISM recovered the missing evidence" from "PRISM's features are more collapsed." The needed control is simple and absent: normalize by inter-image similarity. Since this is the only mechanistic evidence offered, the explanation of why PRISM is more robust is unsupported.
Questions for authors
  1. Please clarify the masking, augmentation, label-scoring, and segmentation configurations. Do all compared models use the same evaluation masks and scoring rules?

  2. Please identify the result/checkpoint sources and run counts, substantiate or correct the PaCa-Ti and DeBiFormer statements, and reconcile the clean mask AP discrepancy (41.1 versus 40.8).

  3. Please provide the missing architecture widths and a per-component parameter/MAC/FLOP breakdown, distinguishing slot-level from spatial-token computation.

  4. Please reconcile the halting averages, specify the maximum-step cap and remainder handling, and clarify how reported costs account for actual executed steps under clean/occluded conditions and batched execution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.