- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper investigates whether DINOv3 patch tokens retain semantic and spatial information below their native 16×16-pixel resolution. Using ADE20K compositions and synthetic shape–colour mixtures, it studies feature anisotropy, direction versus magnitude, within-patch location, foreground–background entanglement, and foreground retrieval through Fisher-style supervised projections. Based on these observations, it proposes an ADE20K-trained prototype-conditioned upsampler that consumes four layers of frozen DINOv3 features, predicts 16 child tokens per parent patch without RGB guidance at inference, and separates selected semantic directions from a complementary detail path. Evaluation covers synthetic retrieval, reconstruction of high-resolution DINOv3 features, ADE20K segmentation with linear and DPT-style heads, and transfer of the frozen upsampler to OpenEarthMap.
Critical strengths
Recovering dense features without running the ViT at substantially higher resolution is important for segmentation and other dense tasks
Critical weaknesses
-
The main comparison confounds architecture with substantially stronger semantic supervision. The proposed upsampler is trained on ADE20K with high-resolution DINO targets, child semantic labels, class prototypes, routing cross-entropy, InfoNCE, a class-margin loss, an image-presence loss, and 5× weighting of minority children (pp. 4–5 and 16–17; Figure 1; Appendices C.1–C.3). The paper does not state that JAFAR or AnyUp was retrained with equivalent semantic supervision; Appendix C.4 says only that the downstream heads share a recipe. The cited JAFAR and AnyUp methods are task-agnostic and annotation-free in their original formulations.
-
No ablation establishes that the representation findings cause the reported gains. The method combines four DINOv3 layers, 5×5 context, child-position queries, trainable Fisher-initialized directions, hard top-k selection, multiple prototypes, an image-level prior, semantic/detail paths, and seven losses (Section 3.3; Appendices C.1–C.3). No component ablation is reported, and W is optimized after initialization.
Questions for authors
-
What exact data, supervision, checkpoints, losses, and tuning budgets were used for JAFAR and AnyUp? Can you provide same-supervision baselines and a reconstruction-only variant?
-
Was the ADE20K validation set used both to select epoch 13 and to report Tables 2–3? What encoder and RGB-guidance resolution was used for every row?
-
Which improvements remain after ablating Fisher initialization, top-K selection, prototypes, semantic losses, multilayer context, image-presence prediction, and the detail path?
-
How are 128 Fisher directions obtained from the stated 64-dimensional search space, how is Q constructed relative to learned W, and how are hard selection and prototype assignment trained?
-
Do the conclusions hold for natural mixed patches and unseen foreground identities, and against fitted affine mixtures and standard supervised linear classifiers?
Response to
I appreciate the author’s detailed response and additional experiments, which have addressed some of my concerns. However, having read the comments from other reviewers and the corresponding responses, I remain concerned about the paper’s contribution; therefore, I am maintaining my original score.