acceptodds
Withdrawn from ICLR 2027

Hidden Label Distinctions Change the Gain from Representation Rate

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

How much accuracy does a fixed increase in representation rate buy under ambiguous supervision? We study whether this marginal gain changes when candidate-label channels have equal ambiguity and spectrum but hide different class distinctions. Using frozen ImageNet features, we compare a fixed increase from 32 to 64 nominal bits per feature vector. Separate DESIGN measurements select the class pairs with the largest and smallest discrimination gains, fixing two matched hiding interventions and their expected gain ordering before downstream fitting and TEST. Across 68 parent groups and 340 classes, hiding the largest-gain pairs yields 0.702 percentage points less mean fine-class gain (95% interval [0.088, 1.345]); the fine-minus-coarse contrast is 0.752 points. The interaction persists with each high-rate-trained classifier held fixed; fixed-low persistence remains unresolved. The higher final-accuracy arm has the smaller marginal gain. Difficulty and headroom remain possible explanations; the contrast does not isolate a unique mechanism. Concept-preserving EEG correspondence controls provide supporting discovery. In the tested system, matched channel ambiguity and spectrum do not equate the marginal value of the same representation increment.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
representation learning, partial-label learning, representation compression, supervision ambiguity, rate-utility trade-offs

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ZAdVOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates whether the marginal utility of increasing representation rate depends on which class distinctions are obscured by ambiguous supervision. Using frozen ImageNet features, the authors construct two candidate-label channels with matched ambiguity and singular-value spectra but different hidden class pairs. They find that hiding pairs with larger rate-dependent discrimination gains leads to smaller downstream accuracy improvements. EEG experiments provide additional supporting observations.

Critical strengths
  1. The paper studies a well-defined question at the intersection of representation compression and weak supervision.
  2. The experimental design is relatively careful, with separate DESIGN and TEST splits, matched supervision channels, and predefined hypotheses.
Critical weaknesses
  • The central observation is not particularly surprising: matching the entropy and spectrum of two supervision channels does not imply equal downstream utility when their relationships to the underlying feature distribution differ. The paper does not establish a new theoretical characterization or provide a predictive framework beyond this observation.
  • The HIGH and LOW pairs differ substantially in baseline classification difficulty (80.49% vs. 91.04% Q4 accuracy). Consequently, the observed gain difference may reflect baseline difficulty or available headroom rather than the proposed interaction between hidden distinctions and newly accessible representation information. The independent contribution of the DESIGN selection criterion remains unestablished.
  • The primary effect is relatively small (0.702 percentage points) and heterogeneous across class groups. The main evidence is restricted to a single pretrained feature geometry and representation-rate increment, while the supplementary experiments do not independently validate the same mechanism.
  • The paper introduces extensive terminology, experimental qualifications, and auxiliary analyses before clearly establishing its central intuition. The complexity of the presentation appears disproportionate to the underlying research question and empirical findings.
Questions for authors

see weakness

Reviewer_bfosOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how much accuracy a fixed increase in representation rate buys, and whether that depends on which class distinctions the supervision hides. The main experiment uses frozen ResNet18 ImageNet features compressed with a nested residual vector quantizer at 32 and 64 bits per vector. In each of 68 five-class parent groups, a separate design split picks the class pair that gains the most from the extra rate (HIGH) and the pair that gains the least (LOW). Two candidate-label channels are then built that hide one of these pairs. The channels match in ambiguity, rank and spectrum. On held-out test data, hiding HIGH gives a 0.70 point smaller gain in fine accuracy than hiding LOW (95% interval 0.09 to 1.35), even though hiding HIGH has the higher final accuracy. An EEG retrieval study with correct and within-concept swapped supervision is offered as earlier supporting evidence.

Critical strengths
  1. Locking the HIGH/LOW map, the channels and the direction of the hypothesis before any reader fitting or test scoring is a strong guard against tuning on the test set.

  2. Building both arms as permutations of the same channel with the same spectrum, rank, objective and exposure isolates which distinction is hidden in a clean way.

  3. The fine-minus-coarse contrast helps rule out a simple shift in accuracy that would affect both task levels equally.

  4. The fixed-reader crossing shows the effect persists for high-rate-trained readers without refitting at each rate.

  5. Failed and unresolved results, such as fixed-low persistence, the EASY/MID/HARD order, the EEG specificity test and the NOD replication, are reported openly.

Critical weaknesses
  1. Everything rests on one ResNet18 backbone, one 32 to 64 bit step, one linear reader and one hierarchy, so it is unclear whether the result holds for stronger features or other rate steps.

  2. There is no breakdown of the 0.70 point gap between the two hidden classes and the remaining 338 classes, so it is unclear whether the effect is more than the direct loss on the hidden pair.

  3. The 10 point baseline difficulty gap between HIGH and LOW pairs is acknowledged but not controlled with a difficulty-matched selector.

  4. The effect is small and fragile, with a lower bound of 0.09 points and only 42 of 68 groups going the predicted way, and the negative design gain of LOW pairs suggests some regression to the mean.

  5. The EEG study uses a different task, rate unit and type of supervision with only 8 participants, so it does not test the orientation hypothesis and mostly adds length.

Questions for authors
  1. How much of the 0.70 point fine contrast comes from the two hidden classes in each parent and how much from all other classes?

  2. Does the contrast survive if pairs are chosen by baseline Q4 difficulty alone, or if HIGH and LOW pairs are matched on Q4 accuracy?

  3. Where does the observed 0.70 points fall in a null distribution built by hiding a random pair in each parent many times?

  4. Does the ordering still hold when the gain is measured relative to headroom, such as the fraction of remaining error removed, since HIDE LOW starts lower?

  5. Does the effect appear with at least one other backbone such as ViT or CLIP and one other rate step such as Q8 to Q16? This could be a future work.

Reviewer_r27ROpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper appears to address how bitrate of representations results in different abilities to accurately decode images associated with the input representations (imagenet, EEG data acquired while participants view natural images) through a variety of analysis approaches. There seems to be some evidence that selecting a subset of images for decoder refinement based on their expected gain can improve performance. The paper is not sufficiently clear to a broad audience to extract further information at present.

Critical strengths

None

Critical weaknesses
  1. As written, the paper is very hard to read. Details are presented one after another with no introduction, justification, or narrative. By the end of the first section there is barely any information that would be accessible to the general ICLR audience as to what the goals of the project are, or what the team accomplished
  2. Overall goals of the work are unclearly specified. It appears the aim is to test how changing the bit depth of representations may impact decoding/classification performance, but I don’t have much confidence in my ability to understand the manuscript as written
  3. Use of EEG data is not justified or well-described, so its contribution cannot be assessed
Questions for authors
  1. What is the goal of the work? Does this support improved efficiency, generalizability, etc, of decoding methods? What has changed about our understanding with this work?
  2. Why is EEG data used? What impact does the described work have on EEG decoding, and why does this matter practically?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.