- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims to enhance the robustness of visual models under occlusion when inferencing. The authors propose a method to reconstruct occluded parts by leveraging prior memory of complete objects through iterative inference. The PRISM method integrates three established components into a unified backbone. Slot attention aggregates image features into a set of slots, effectively decomposing the scene into a structured representation rather than relying on a dense grid of tokens. Slot-level VQ-VAE applies standard VQ-VAE vector quantization at the slot level. During inference, each slot is used to retrieve its nearest prototype, then incorporate the prototype information back into the representation. ACT dynamically determines the number of recurrent steps based on the complexity of the input image. The author demonstrates the effectiveness of this method through various experiments and ablation studies.
Critical strengths
- Occlusion robustness is a real problem in the field of vision and is often overlooked by mainstream benchmarks. Most SOTA methods compare accuracy on clean ImageNet, while in real-world deployments, objects occluding each other is almost the norm.
- The experiments cover classification, detection, and instance segmentation. A benchmark for occlusion has also been designed, featuring five distinct types to stress-test the models from various perspectives.
- The authors provide visualizations in the appendix, showing how attention gradually focuses on the object and suppresses the obstructions in each loop, moving from an occluded state. This demonstrates the effectiveness and interpretability of this method.
- The correspondence between the motivations mentioned by the authors and the design of their methodological framework is very clear. The intuitive approach of filling in the obscured parts based on memory, combined with repeated deduction, was precisely translated into three modules.
Critical weaknesses
- The Slot Attention, VQ-VAE, and ACT components mentioned in the methodology section of the article are all existing methods. The PRISM's main contribution is to combine them into the backbone and make a loop. This leads the reviewer to question whether the work offers sufficient substantive contribution and innovation?
- All experiments are limited to the tiny scale, and the models used for comparison are also small models. It remains unclear whether the robustness gains provided by VQ prototype memory will persist or vanish as the model size increases?
- The test datasets consist entirely of synthetic occlusions. These types of occlusions produce pixel statistics that fall outside the original distribution, rather than representing realistic occlusions. Are there any test results based on real-world datasets?
- Table A7 shows that MAE ViT-B outperforms PRISM across all metrics, though this may be due to the fact that MAE has 86M parameters. However, if the number of parameters were held constant, would PRISM be able to surpass MAE?
- The loss weight of ACT ponder = 0.005 is a small value that it raises doubts about the true contribution or influence of ACT to the overall loss calculation process? Furthermore, the ACT is unstable during training, and the paper does not provide any sensitivity analysis.
Questions for authors
- Could you add a variant that applies the same VQ memory to (a) dense token features or (b) a single globally pooled feature instead of object-centric slots?
- Table A7 compares an 11M model against an 86M MAE that also received 800 epochs of self-supervised pretraining. Could you report a MAE at a comparable scale (ViT-Ti/16 at ~5M, or a width-reduced MAE at ~11–13M) under the same protocol?
- Could you add at least one evaluation with real occlusions or an existing real-occlusion benchmark such as Occluded-ImageNet or COCO-O if applicable?