acceptodds
Under review as a conference paper at ICLR 2027

LinPart: Unsupervised Part Discovery with a Single Linear Head

Abstract

Unsupervised part discovery segments objects into parts and assigns consistent identities across images without part annotations. Recent methods build on self-supervised features by adding trainable modules or partially fine-tuning the backbone. Two analyses of frozen DINOv2 features motivate a simpler design. First, nearest-centroid clustering already recovers part structure using a linear assignment rule. Second, cross-image feature neighbours agree on part identity more often than randomly reconnected pairs. Based on these observations, LinPart trains only a shared linear head on the frozen features. Its objective combines feature reconstruction and regularization to guide foreground selection, division into parts, and consistent part assignments across images. With at most 39,270 trainable parameters, LinPart outperforms prior methods. We also derive a merge score by relating the expected correspondence loss to graph modularity and incorporating changes in reconstruction and warp consistency. Applied to a trained head's predictions, the score produces partitions with fewer parts without annotations or retraining. In multiple evaluated settings, these partitions achieve higher mean agreement with annotated parts than direct training at the target part count.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
unsupervised part discovery, part segmentation, frozen self-supervised features, linear head, cross-image correspondence, graph modularity

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_H8xyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a simple linear head for unsupervised part discovery on top of a frozen DINOv2 backbone as well as a merging criterion that allows changing the partition granularity after training the head. The method is evaluated on three benchmarks across three granularities with mixed results compared to prior works, which however have more complex architectures.

Critical strengths

The paper shows that unsupervised part discovery is possible based on strong frozen features with only a lightweight head (Table 2) and that the granularity of the resulting partitions can be adapted post-hoc (Table 6). This is relevant as prior works often employ more complex architectures (Table 3) and operate on fixed partition granularities (see e.g. Aniraj et al. 2024).

Critical weaknesses

The method relies on a fairly complex loss function with 6 high-level terms which can further contain multiple sub-components. However, the ablation shows that the different loss terms have very different impact depending on the dataset (Table 5). As an example, removing the foreground anchor loss slightly improves performance on CUB (K=16) while leading to a complete collapse on OOD (K=50). This draws into question how general and robust the loss function is. The complexity is further increased by having different weights for the individual loss terms which are set differently across benchmarks (Table 10) but never ablated. This makes it difficult to assess how general the objective is or how much it was tailored towards the exact benchmarks considered in the paper.

Questions for authors

The paper uses bfloat16 mixed precision (l.802) but compares memory usage using fp32 (l.353) which might skew the results. Is there a reason why the memory comparison does not use the experiment setup used in the rest of the paper?

Reviewer_uZw9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses unsupervised part discovery and argues, from a linear-probe and a cross-image neighbour analysis, that frozen DINOv2 features already contain the needed part structure. LinPart trains only a shared linear head with at most 39,270 parameters on frozen DINOv2-B/14 tokens, using six losses for foreground selection, reconstruction, cross-image correspondence, smoothness, slot usage and warp consistency. A merge score derived from the objective coarsens a trained head to fewer parts without retraining. LinPart reports the highest NMI at K=16 on CUB, K=11 on NABirds and all K on PartImageNet-OOD, and its merged partitions beat direct training in five of six settings.

Critical strengths
  • Motivates the design with two direct analyses of frozen DINOv2 features, a supervised linear probe reaching 88.1 NMI against 66.1 for clustering and cross-image neighbours agreeing on part identity far above a degree-preserving control.

  • Trains only a linear head with at most 39,270 parameters, with 226× fewer trainable parameters and 15 to 19× less memory than MPAE, yet reaches the highest NMI at K=16 on CUB, K=11 on NABirds and all K on PartImageNet-OOD at matched resolution.

  • Reports losses, harmful ablation terms and limitations openly, with three seeds, per-seed appendices and a complete reproducibility statement, and derives a merge criterion from the objective that coarsens a trained head without retraining.

Critical weaknesses
  • OOD results do not test the reported part counts. At K=50 each OOD image uses only 4.7 to 6.0 slots, so the coarsening to 25 parts changes 3.5 to 5.6 percent of labels, and the two OOD coarsening “wins” compare nearly identical partitions.

  • The headline OOD margin rests on a collapsed baseline. The 10.49 to 18.91 point claim at 448 pixels uses a retrained MPAE that scores 16.25 against its published 32.90. At matched 224 pixels the margins are smaller and MPAE leads ARI at K=50.

  • The objective is a union of benchmark-specific needs. Removing reconstruction helps OOD, removing the foreground anchor helps CUB, warp consistency matters only on birds, coefficients differ by benchmark, and on OOD every single-layer MLP beats the linear head.

  • Constant selection is unreported. Thresholds, margins, neighbour counts, usage bounds and the Sinkhorn temperature are fixed without any stated procedure or validation protocol, so tuning on evaluation annotations cannot be ruled out.

  • Limited novelty. Linear foreground heads, correspondence losses and modularity merging all exist in cited work. The contribution is their combination and a sufficiency result that holds on two of three benchmarks.

Questions for authors
  1. No code is provided. Will the training code, the trained heads and the retrained baselines be released?
  2. How were the thresholds, hinge margin, neighbour counts, usage floor and cap and Sinkhorn temperature chosen, and was any annotated data used in that process?
  3. On PartImageNet-OOD at K=50 each image uses about five slots. How many slots are used across the whole dataset, and what do NMI and ARI become when background pixels are excluded?
1 comment

Thank you for the response, which addresses some of my concerns. However, the rebuttal regarding novelty is not fully persuasive and the code is still not yet available, making it difficult to verify the implementation details.

Considering both the whole paper quality and the rebuttal, I still believe the paper merits a Weak Accept.

Reviewer_NGYbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LinPart performs unsupervised part discovery by training only a shared linear head on frozen DINOv2 features with six loss terms. It also introduces a criterion for merging parts without retraining. Experiments on three benchmarks demonstrate competitive performance and low training costs.

Critical strengths
  1. Simplicity and efficiency. With at most 39,270 trainable parameters, LinPart achieves strong results in several settings while substantially reducing training-step time and memory usage under matched measurement conditions.
  2. Reasonably thorough validation. The paper includes multiple seeds, loss ablations, head-capacity experiments, and merge-criterion comparisons, while explicitly reporting settings where the method is less effective.
Critical weaknesses
  1. The sources of the main performance gains are insufficiently disentangled. PartImageNet-OOD metrics include background, leaving unclear how much improvement over competing methods comes from foreground localization versus semantic part grouping. The foreground-anchor ablation establishes its importance within LinPart, but does not explain differences between methods.
  2. The additional value of the full merging score is setting-dependent. Table 6 supports effective coarsening without retraining, but the full score achieves the best NMI and ARI in only three of six settings. Its advantage over simpler criteria is therefore inconsistent.
Questions for authors
  1. Could you report foreground-only NMI/ARI on PartImageNet-OOD and diagnostic comparisons using identical foreground masks to separate foreground localization from part grouping?
  2. Could you explain the large NMI difference between the published 224-resolution and retrained 448-resolution MPAE results, and establish that the main comparisons are robust to preprocessing and evaluation protocol differences?
  3. Are the advantages of the full merging score over simpler criteria stable across different training-image subsets used to estimate merge statistics?

Comments on the paper

1 comment

Dear Reviewers,

The authors have submitted detailed responses to your reviews.

Please review their feedback, engage with the authors if any points remain unclear, and update your evaluation and score if their response has addressed your concerns.

Thank you

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.