- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors revisit jigsaw-style self-supervised objectives in a modern DINOv3 self-distillation framework. They compare relative position, absolute position, and permutation classification objectives, and introduce two new formulations: blockwise relative position and region matching. The approach is evaluated on a broad suite of frozen-feature benchmarks spanning dense prediction, correspondence, tracking, object discovery, saliency, boundary detection, anomaly localization, and global recognition. The results show that explicit spatial supervision can improve spatially sensitive tasks, with region matching being the only objective that improves both spatial and semantic performance relative to DINOv3.
Critical strengths
- The paper addresses an important question in self-supervised representation learning: whether explicitly supervising spatial arrangement, rather than relying only on semantic invariance or masked prediction, improves fine-grained and spatially structured representations.
- The study is well motivated and provides a useful controlled comparison of several jigsaw formulations under a common DINOv3 training setup, backbone, pretraining checkpoint, refinement schedule, and evaluation protocol.
- The two proposed objectives, blockwise relative position and region matching, are interesting. Region matching in particular is a sensible formulation because it uses soft spatial targets over image regions rather than requiring exact tile-to-cell assignment.
- The evaluation is broad and thus convincing, covering object discovery, saliency, boundary detection, anomaly localization, segmentation, depth, correspondence, video segmentation, point tracking, and global recognition.
- The finding that different jigsaw formulations favor different capabilities is valuable: permutation classification appears strongest for figure-ground separation and object discovery, while relative-position variants are stronger for boundaries, anomaly localization, and tracking.
Critical weaknesses
- Although the paper introduces two new objectives, the technical novelty is moderate. The core idea is a careful modernization and systematic empirical study of jigsaw objectives within DINOv3, rather than a fundamentally new learning principle. The contribution would be strengthened by clearer analysis of why each objective produces its observed representational trade-off.
- The paper claims that jigsaw objectives improve fine-grained visual representations, but the mechanisms behind the improvements are not directly validated. For example, it remains unclear whether gains arise from better boundary sensitivity, better local correspondence, reduced feature collapse, or simply an additional dense prediction signal. Additional quantitative probing to clarify the mechanisms would be helpful.
- The comparison is limited to a ViT-L/16 backbone, ImageNet-1k pretraining, and a relatively short 20k-iteration refinement phase. It is unclear whether the conclusions generalize to other backbone scales, longer training schedules, larger datasets, or models where the dense-feature degradation problem may be more or less pronounced.
- Some results appear mixed and deserve more discussion. For example, several jigsaw variants clearly improve object discovery and anomaly detection but underperform DINOv3 or NeCo on SPair correspondence and depth estimation. The paper would benefit from a clearer explanation of when spatial supervision helps and when it harms downstream transfer.
- The shortcut-suppression analysis in Table 5 is interesting but potentially confusing: adding augmentations changes the effective task difficulty, so it is difficult to isolate whether the observed shifts are due to removal of low-level shortcuts or to a change in the optimization landscape.
Questions for authors
- What is the main source of the observed improvement: stronger boundary information, improved local feature discriminability, better object/background separation, or reduced degradation of dense features during refinement? Could this be disentangled using feature-similarity, boundary-localization, or patch-correspondence diagnostics?
- Why does region matching improve both semantic and spatial benchmarks, whereas the other jigsaw objectives show a clearer trade-off? Is this because its soft region-level target avoids overfitting to exact tile positions, or because it preserves more global context?
- How sensitive are the conclusions to the refinement duration and loss weight λ=0.3? In particular, would longer refinement cause the jigsaw objectives to over-suppress semantic invariance or eventually recover the semantic performance of DINOv3?
- For the shortcut-suppression experiments, could you share the results for separating the effect of grayscale conversion, chromatic shifting, and per-tile normalization in order to better understand the effects of each?