acceptodds
Withdrawn from ICLR 2027

CLAP: Classification and Localization with Object-Aligned Prototype Evidence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Part-prototype networks make image classification transparent by expressing predictions through spatial matches to learned prototypes. Yet classification supervision rewards any discriminative cue, so prototypes may respond to a small object part or correlated background rather than faithfully represent the target object. This limits the use of their spatial responses for object localization. We introduce CLAP, a training framework that makes prototype evidence object-aligned using only image-level labels. A lightweight class-agnostic foreground–background head produces pseudo-masks, and two alignment losses encourage collective coverage of the predicted foreground while suppressing responses on predicted background. Classification and localization are then computed from the same class-associated spatial prototype responses, allowing the prototype pathway to retain its discriminative role while gaining localization capability. We instantiate CLAP with ProtoPNet and ProtoPool. On part-prototype benchmarks, CLAP substantially improves GT-Known localization while preserving classification accuracy with no additional cost at inference. On standard WSOL benchmarks, it achieves 97.5% GT-Known localization on CUB-200-2011 and 74.9% on ILSVRC, remaining competitive with specialized WSOL methods. These results show that part prototypes can provide a unified, intrinsically traceable basis for recognition and localization, establishing prototype-based WSOL as a promising direction for future part-prototype architectures.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
weakly supervised object localization, prototype learning, part-prototype networks, object localization, interpretable deep learning, object-aligned representations

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_m1ECOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CLAP extends part-prototype classifiers to weakly supervised object localization by aligning their spatial responses with foreground–background pseudo-masks learned through a C2AM-style contrastive objective. Background-exclusion and foreground-coverage losses supervise the collective responses of ground-truth-class prototypes. A fixed sum of max-pooled prototype responses replaces the learned classifier, while normalized spatial responses produce localization maps. The auxiliary mask head is discarded at inference.

The framework is instantiated with ProtoPNet and ProtoPool using DeiT-S and DINOv2-S backbones. Experiments show substantial localization improvements over prototype baselines, generally comparable classification accuracy, and competitive WSOL results on CUB and ILSVRC. Loss ablations demonstrate complementary benefits from background exclusion and foreground coverage.

Critical strengths
  1. Substantial and consistently demonstrated localization improvements. CLAP markedly improves localization over ProtoPNet and ProtoPool across the evaluated datasets. The controlled loss ablation shows that background suppression and foreground coverage provide complementary benefits. This supports the practical usefulness of the proposed alignment mechanism.

  2. A compact integration of prototype classification and localization. Both outputs are computed from class-associated prototype responses, and the auxiliary supervision head is removed at inference. This provides explicit traceability to prototype activations while retaining competitive localization performance, making the framework a useful extension of interpretable prototype classifiers.

Critical weaknesses
  1. The central claim of faithful spatial correspondence is insufficiently validated. Classification sums raw max-pooled responses, whereas localization takes a spatial maximum over individually normalized response maps. Consequently, non-maximal regions can affect localization without directly contributing to the class logit, and weakly contributing prototypes can become prominent after normalization. Shared responses establish computational traceability, but do not by themselves establish that the full localization map faithfully represents classification evidence. Figure 4 illustrates the architecture rather than resolving this concern. This matters because such correspondence is the paper’s principal advantage over specialized WSOL methods.

  2. The experiments do not sufficiently isolate the distinctive contribution of the proposed design. The ablation establishes the value of the alignment losses, but does not separate their effects from those of removing the learned classifier or assess whether comparable benefits arise with a conventional class-response head. The pseudo-mask generator’s own localization quality is also unreported. These omissions make it difficult to determine how much benefit comes specifically from prototype-based integration. Unclear ILSVRC threshold calibration further limits confidence in the comparisons.

Questions for authors

The following are clarification requests that could affect my assessment:

  1. What data and procedure are used to select localization thresholds on ILSVRC and in Table 1? Please clarify whether evaluation images or their bounding boxes influence threshold selection, and whether compared methods use the same box-extraction procedure.

  2. Is the claimed classification–localization correspondence intended to mean prototype-level computational traceability, or faithfulness of the entire spatial localization map? How does the latter interpretation account for non-maximal responses and the removal of response magnitude through per-prototype normalization?

  3. Do you have controlled results for retaining the learned classifier while applying the alignment losses, and for directly localizing with the learned foreground pseudo-masks? These comparisons would materially affect my assessment of the necessity and added value of the proposed integration.

Reviewer_AvzAOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a new methods for weakly supervised object localization named CLAP, which extends part-prototype classifiers to weakly supervised object localization. A C²AM-style foreground/background head produces class-agnostic pseudo-masks, which supervise background exclusion and a one-sided foreground-coverage objective on class-associated prototype responses. Class logits sum the rectified peaks of the corresponding prototypes, while localization aggregates their normalized spatial response maps. The auxiliary head is removed at inference. The main contribution is therefore the integration of established objectness supervision with a prototype-based prediction pathway. Experiments with ProtoPNet and ProtoPool report substantial localization gains and competitive WSOL performance.

Critical strengths
  1. The author utilizes prototype classifiers to improve the localization performance. Table 1 reports GT-known localization gains of 35.9–65.8 percentage points, while maintaining the same classification accuracies.

  2. The inference framework is somehow easy and straightforward.

Critical weaknesses

However, I have severe concerns about the paper.

  1. Compared to existing baseline methods like C^2AM and the earlier baseline like PSOL, I think the novelty is limited. Foreground/Background separation and pseudo-labels were already explicitly discussed in PSOL. C²AM subsequently developed class-agnostic foreground/background separation through contrastive learning, which is adopted by CLAP. I think the contribution of CLAP lies in the prototype part. Thus, I think the novelty is minor.

  2. I think the performance boost over existing methods is minor. CLAP–ProtoPNet improves GT-known localization over TriLite with DeiT-S, but under DINOv2-S it trails TriLite on all reported localization metrics on both datasets. Thus, I have severe concerns about its generalization ability.

Questions for authors

My most questions lie in the weaknesses part:

  1. Can you claim the differences/novelty compared to existing baseline methods?

  2. The performance boost is somewhat minor, and can the proposed method be extended to CNN-based backbones?

Reviewer_XQdmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

this work tackles the task of image classification using part-prototypes networks. authors argue that these models fails in terms of part-localization to cover objects: either it covers only part of the objects, or it correlates with background, limiting its interpretability. to deal with this, authors borrows strategies from wsol field. in particular, they used a fg/bg tiny classifier at the feature outputs, in a first training stage. this allows to build class-agnostic masks to separate fg/bg. this is achieved using pretrained dinov2 models, without supervision using c2am method. in a second training phase, they train the part-proto model by aligning the aggregated map from the prototypes with the fg/bg masks to promote parts to highlight fg while skipping bg.

results are reported over both domains: part-proto models (cub+stanfard cars), and wsol (cub+imagenet). ablation about fg/bg alignment is provided.

Critical strengths
  • the writing is good.
  • the paper aims to improve the interpretability of part-proto models.
  • results are reported.
Critical weaknesses

method:

  • lack of novelty. to improve interpretability of part-proto models, authors borrow common approach in wsol: predict fg/bg using a tiny classifier over features. this is well known in wsol. the methods requires 2-stage training. in phase 1, dino is needed to be able to build fg/bg maps. other backbones can not be used. in addition, authors borrow this learning from c2am paper. this is critical in the method. without it, there is no stage 2. in stage 2, authors align aggregated map from prototypes with the fg/bg masks estimated in phase 1. in short, a class-agnostic mask is first estimated, then used for alignment to guide object-parts. the issue here is: alignment between class-agnostic mask vs discriminative map from prototypes. this may work in cub, but not in images with complex scenes (where the salient object in the class-agnostic mask may not be the the object targeted in the part-proto/image class). there was no check here that the fg mask captures the real class in the image. in addition, the class-agnostic mask is considered ground truth (fully trusted) without any uncertainty consideration as the part-proto mask is directly aligned to it (eq.2).

experiments:

  • the paper sits on 2 fields: part-proto classification models, wsol. in terms of performance, the proposed model fails both domains. in tab.1 for the first domain, the propose method falls behind in terms of classification - the main task - compared to sota. it only improves "interpretability" at the expense of classification. in wsol, the propose method fall way behind sota.
  • very limited comparison to existing works in wsol (tab.2); extremely limited comparison to part-proto methods in tab.1
  • lack of ablations: encoder (only dinov2, deit). part-proto models (proto-pool/net).
  • sec.5.1 - line 367: in tab.1 "CLAP improves .... with classification changes ranging from −2.1 to +1.9 points": it should be clear here that you mean w.r.t to the prototype models (protopool/net) and not w.r.t sota in that table. it is confusing. w.r.t sota, CLAP is behind always (-).
  • no clear discussion how hyper-parameters were obtained.

in short, the method lacks novelty. it only borrows common approach in wsol for part-proto model to improve interpretability. at the end, the method fails to compete in both domains while improving interpretability in part-proto models.

please stop using ai.

Questions for authors

please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.