- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates whether the marginal utility of increasing representation rate depends on which class distinctions are obscured by ambiguous supervision. Using frozen ImageNet features, the authors construct two candidate-label channels with matched ambiguity and singular-value spectra but different hidden class pairs. They find that hiding pairs with larger rate-dependent discrimination gains leads to smaller downstream accuracy improvements. EEG experiments provide additional supporting observations.
Critical strengths
- The paper studies a well-defined question at the intersection of representation compression and weak supervision.
- The experimental design is relatively careful, with separate DESIGN and TEST splits, matched supervision channels, and predefined hypotheses.
Critical weaknesses
- The central observation is not particularly surprising: matching the entropy and spectrum of two supervision channels does not imply equal downstream utility when their relationships to the underlying feature distribution differ. The paper does not establish a new theoretical characterization or provide a predictive framework beyond this observation.
- The HIGH and LOW pairs differ substantially in baseline classification difficulty (80.49% vs. 91.04% Q4 accuracy). Consequently, the observed gain difference may reflect baseline difficulty or available headroom rather than the proposed interaction between hidden distinctions and newly accessible representation information. The independent contribution of the DESIGN selection criterion remains unestablished.
- The primary effect is relatively small (0.702 percentage points) and heterogeneous across class groups. The main evidence is restricted to a single pretrained feature geometry and representation-rate increment, while the supplementary experiments do not independently validate the same mechanism.
- The paper introduces extensive terminology, experimental qualifications, and auxiliary analyses before clearly establishing its central intuition. The complexity of the presentation appears disproportionate to the underlying research question and empirical findings.
Questions for authors
see weakness