acceptodds
Withdrawn from ICLR 2027

Partition Recovery for Structured Representation Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Representation learning is typically evaluated through predictive accuracy, reconstruction quality, or information-theoretic objectives. However, in many scientific and decision-making domains, the true objective is to preserve latent distinguishability structure rather than labels alone. A representation can therefore be predictively sufficient while collapsing meaningful latent modes. We formalize this problem as partition recovery: recovering the latent equivalence relation induced by task-relevant distinguishability. We develop a statistical theory of partition recovery, establishing consistency guarantees, finite-sample recovery bounds governed by representation complexity and partition margin, and a characterization showing that recovery under imbalance is controlled by the least-sampled latent mode. These results motivate a stronger notion of representation quality, task-structure sufficiency, which requires preservation of the underlying partition rather than predictive performance alone. To optimize this objective, we propose a differentiable representation-learning framework (DPT) that directly targets partition preservation. DPT can be trained using either domain-derived partitions or partitions statistically bootstrapped from data. Across synthetic and real-world benchmarks, we show that conventional supervised objectives can preserve coarse predictive performance while destroying genuine latent distinctions, whereas DPT retains the underlying structure and reveals representational failures that remain invisible to standard accuracy metrics. Together, these results suggest that representation learning should be evaluated not only by what predictions are preserved, but also by what latent structure survives compression.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
representation learning, hierarchical classification

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_yYHvOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the tension between downstream predictive sufficiency and the preservation of latent equivalence structure in representation learning. The authors argue that in scientific and engineering domains such as fault diagnosis, representations must retain the underlying distinguishability partition rather than merely fitting coarse task labels.

The main contributions include:

  • Partition Recovery Theory: Formalizes latent mode recovery using signature metrics and a partition margin γ\gamma, proving consistency (Theorem 1), finite-sample recovery bounds (Theorems 5 & 6), and an imbalance recovery bound governed by the least-sampled mode min⁡fnf\min_f n_f rather than the total sample count nn (Corollary 2).
  • Task-Structure Sufficiency: Formally proves that representations achieving Bayes-optimal classification risk or maximizing label mutual information I(ϕ(X);Y)I(\phi(X); Y) can still collapse fine-grained latent modes that share a coarse label (Theorem 2, Corollary 1).
  • Differentiable Partition Training (DPT): Introduces a differentiable objective optimizing a soft partition matrix via a soft Variation of Information (Soft VI) bottleneck loss, trained against symbolic priors or data-bootstrapped reference partitions.
  • Empirical Validation: Evaluates partition recovery on a synthetic two-tank process, an IEEE 7-bus radial feeder, and the CWRU bearing dataset, reporting metrics such as Fibre Purity (monitoring false merges) and Fibre Completeness (monitoring spurious fragmentation).
Critical strengths
  • Thought-Provoking Conceptual Motivation: Through Theorem 2 and Corollary 1, the authors provide clear constructive counterexamples illustrating that representations achieving downstream predictive sufficiency or Bayes-optimal risk on coarse labels can still collapse distinct fine-grained latent modes. This formalizes an important failure mode of conventional supervised representations in scientific discovery.
  • Granular Metric Decomposition: Decomposing Variation of Information (VI) into Fibre Purity and Fibre Completeness is insightful for isolating false merges from spurious fragmentation. Furthermore, the paper transparently reports the optimization variance of DPT under constrained bottleneck widths (Figure 5(a)) and shows that reference partition errors induce an irreducible bias (Figure 2).
Critical weaknesses
  1. Foundational Mathematical Gaps in Partition Definition:

    • Non-Transitivity of ϵ\epsilon-Confusability: Definition 1 defines fi∼ϵfj  ⟺  d(Φ(fi),Φ(fj))≤ϵf_i \sim_\epsilon f_j \iff d(\Phi(f_i), \Phi(f_j)) \le \epsilon and directly forms the quotient Πϵ∗=F/∼ϵ\Pi_\epsilon^* = F / \sim_\epsilon. However, thresholded metric distance is non-transitive: d(A,B)≤ϵd(A, B) \le \epsilon and d(B,C)≤ϵd(B, C) \le \epsilon does not imply d(A,C)≤ϵd(A, C) \le \epsilon. Thus, ∼ϵ\sim_\epsilon is generally not an equivalence relation and cannot induce a valid quotient partition without taking an explicit transitive closure.
    • Proof Inconsistency: Although Appendix I.4 briefly mentions a transitive closure, the proof of Theorem 1 (Appendix G.1) explicitly relies on every pair in the same population block having distance ≤ϵ\le \epsilon, which fails under transitive closure chaining.
    • Logical Jump in Theorem 14: The proof of Theorem 14 (Appendix E.9) asserts that HD(ΠO∣ΠZ)=0H_D(\Pi_O \mid \Pi_Z) = 0 implies ΠO=ΠZ\Pi_O = \Pi_Z. In information theory, zero conditional entropy only implies that ΠZ\Pi_Z is a refinement of ΠO\Pi_O (i.e., ΠZ≤ΠO\Pi_Z \le \Pi_O), not that the two partitions are equal.
  2. Contradiction in Supervisory Budget (Fine-Grained Labels vs. DPT):

    • In Section 7.3, DPT is presented as a method that operates without fine-grained class labels. However, the training algorithm requires sampling a "per-fault mini-batch" at each step to compute empirical class signatures Φ^(fi)\hat{\Phi}(f_i). Appendix I.11 explicitly concedes that grouping samples by fault before forming signatures is a "labeling requirement".
    • If sample-level fault identities are already known during training, the claim that DPT requires no fine-grained labels is misleading. In scenarios where the target partition P∗P^* consists of singleton fibres (e.g., in Figure 2 and Appendix H.12.1), this requirement is practically equivalent to full fine-grained supervision.
  3. Direct Discrepancy Between Hypothesis H6 and Empirical Findings:

    • Section 8.2 and Appendix H.10.6 explicitly state Hypothesis H6: "imbalance and noise degrade purity more than completeness (false merges)".
    • However, Figures 1(b)-(c) and Figures 4(a)-(b) show the exact opposite: Fibre Purity remains nearly 1.0 across all noise and imbalance levels, whereas Fibre Completeness degrades severely (indicating spurious fragmentation/false splits, not false merges).
    • In Section 8.3, the text erroneously asserts that this pattern is "confirming H6 precisely". Claiming empirical confirmation for a result that contradicts the formal hypothesis represents a severe disconnect between hypotheses and evidence.
  4. Missing Empirical Baselines for Core Criticized Paradigms:

    • The paper strongly positions DPT against modern self-supervised learning (SimCLR, TS2Vec, SwAV), arguing that optimizing continuous embedding geometry inevitably fails to preserve discrete partition structures (Sections 1, 2, and Appendix J.8).
    • Yet, the empirical evaluation (Figures 1, 5, 6) contains no modern SSL or contrastive learning baselines, comparing DPT only against raw features, supervised classifiers, and simple anomaly detection. The claim that DPT outperforms modern SSL in partition preservation remains unsupported by empirical data.
  5. Theory-Practice Disconnect and Architecture Scale:

    • The sample complexity bounds in Theorem 6 rely on an abstract capacity measure C(H)\mathcal{C}(\mathcal{H}). As the authors acknowledge in Appendix I.6, classical VC-dimension or Rademacher complexity bounds are notoriously vacuous for modern overparameterized neural networks, and no instantiated bounds are provided for the evaluated models.
    • Theorem 7 characterizes only global minimizers and provides no convergence guarantees for gradient-based training on non-convex objectives (Remark 1). In practice, DPT displays substantial optimization variance (Figure 5(a), average ARI ≈0.80\approx 0.80).
    • Empirical architectures are restricted to micro-scale models: a 2-layer MLP (16 hidden units) and a shallow 1D-CNN (8/16 channels) on 8 to 16 classes. Broader scalability remains unverified.
  6. CWRU Real-Data Experiment Does Not Demonstrate Structure Collapse:

    • In Figure 6(a), all realizations (including coarse supervision) achieve an overall ARI of 1.0 on the 16-condition CWRU bearing dataset.
    • While Figure 6(b) shows that coarse supervision compresses the pairwise MMD between IR007 and IR028 from 0.69 to 0.14, the discrete partition is preserved under the calibrated threshold. This demonstrates margin reduction in continuous space, but falls short of proving an actual failure of partition recovery.
Questions for authors
  1. Transitivity in Partition Definition: How is the ϵ\epsilon-confusability relation in Definition 1 guaranteed to be transitive? If an explicit transitive closure is required to form equivalence classes, how do you resolve the fact that elements within the same block can have distances exceeding ϵ\epsilon, which contradicts the premise in the proof of Theorem 1 (Appendix G.1)?
  2. Supervision Budget and Batch Construction: How can per-fault mini-batches be constructed to evaluate Φ^(fi)\hat{\Phi}(f_i) without sample-level fault identities? If sample identities are known, what is the exact operational distinction in supervisory information between DPT and fine-grained supervised learning when the reference partition consists of singleton clusters?
  3. Inconsistency in Hypothesis H6: Why does Section 8.3 claim to "confirm H6 precisely" when the experimental results in Figures 1 and 4 show that Purity remains at 1.0 while Completeness breaks, which is the exact opposite of H6's stated claim that "imbalance and noise degrade purity more than completeness"?
  4. Comparison with SSL Baselines: Given that SimCLR and TS2Vec are cited as the core paradigms failing to preserve discrete partition structures, can you provide benchmark comparisons against TS2Vec or SimCLR on the CWRU or IEEE feeder datasets under matched embedding dimensions?
  5. Threshold Sensitivity on CWRU: In Figure 6(a), coarse supervision achieves an ARI of 1.0 despite the reduced MMD between IR007 and IR028. Under what calibrated decision threshold τ\tau does this distance reduction actually manifest as an empirical false merger?
Reviewer_SRu3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether a representation preserves distinctions among latent modes beyond those needed to predict a coarse label. Each mode is associated with a conditional distribution of representations, called a signature. The authors formulate partition recovery using distances between these signatures and define task-structure sufficiency as equality between observation-space and representation-space partitions. They present a counterexample separating predictive sufficiency from structural preservation, consistency and finite-sample recovery statements, and an imbalance-dependent sufficient condition involving the least-sampled mode.

The proposed Diagnostic Partition Training (DPT) method matches a differentiable pairwise partition matrix to a reference partition derived from domain knowledge or pilot data. Experiments on two-tank and power-feeder benchmarks, with a CWRU bearing-data example, investigate recovery under noise, imbalance, coarse supervision, and compression. Reported results suggest that coarse supervision can reduce selected fine-grained separations and that a trained low-dimensional representation can improve subsequent recovery from small evaluation batches.

Critical strengths
  1. A concrete and relevant representation-quality question: The paper makes explicit why coarse-label accuracy alone cannot certify retention of finer distinctions. In diagnostic settings, losing a distinction between two physical fault modes can matter even when the existing classifier remains accurate. Making that failure visible is a useful contribution to evaluation.

  2. Controlled experiments with explicit structural targets: Analytically specified synthetic partitions permit direct recovery evaluation, and the wrong-reference-partition ablation exposes the bias inherited from incorrect structural supervision. Reporting VI together with merge/fragmentation measures is more informative than relying on one aggregate score. The bearing-data example provides an additional practical setting, although its interpretation needs qualification below.

Critical weaknesses
  1. The population partition, the proof assumptions, and the differentiable target are not consistently aligned:

Definition1 defines fi∼εfjf_i\sim_\varepsilon f_j by d(Φ(fi),Φ(fj))≤εd(\Phi(f_i),\Phi(f_j))\leq\varepsilon and immediately takes a quotient. Thresholded metric proximity need not be transitive. For example, point masses at 00, 0.75ε0.75\varepsilon, and 1.5ε1.5\varepsilon, with Wasserstein distance, form a chain whose endpoints fail the threshold test.

Appendix I.4 mentions transitive closure, and partition_recovery.py implements it using union-find. However, the proofs in G.1 and G.3 assume every same-block pair has distance at most ε\varepsilon, which is false for connected-component blocks. An edge-connectivity proof may repair those results, but the manuscript must state and analyze the same construction throughout. Relatedly, G.2 argues using an empirical threshold ε\varepsilon, whereas Section~5 defines the empirical indicator with τ\tau.

The optimization claim has a related mismatch. Proposition 1 takes the soft-matrix limit to exact signature equality, rather than positive-threshold connectivity. Theorem 7 concerns a soft VI objective, but Section 7.2 also introduces matrix MSE/BCE, and the inspected implementation optimizes pairwise BCE. The relationship between the pairwise matrix, the memberships qikq_{ik}, and hard-partition recovery needs an explicit definition and justification. A theorem about an unregularized soft VI optimum also does not automatically cover an objective with a positive compression penalty. These issues affect the central theoretical justification, not just notation.

  1. The supervision and data budgets limit the claimed empirical advantage, and the supplied implementation does not yet make that advantage reproducible:

Section 7.3 samples per-fault batches, and the code takes dictionaries indexed by fault identity. Appendix I.11 explicitly acknowledges this grouping requirement. Avoiding a multiclass classification loss is therefore not equivalent to avoiding mode-identity supervision. Moreover, a reference partition that separates same-coarse-label modes supplies information that the coarse-label baseline is not trained to preserve. The comparison demonstrates the effect of supplying a different target, but does not by itself establish superiority under matched information.

The small-nn comparison also needs its scope stated precisely. In run_bootstrap_dpt.py, the reference construction uses a 500-per-fault pilot parameter, training uses an additional 200-per-fault pool, and recovery calibration uses 200-per-fault pilot samples. Thus n∈{5,10,20}n\in\{5,10,20\} denotes the evaluation batch size, not the total observations consumed. Section 7.1 acknowledges the large pilot and an amortized compression benefit is plausible; the missing evidence is a comparison against alternatives allowed the same pilot, training data, and grouping information.

On CWRU, Section 8.3 and Figure 6 report ARI =1=1 for all four realizations. The reduced MMD of one pair is evidence of changed separation under the chosen kernels, not demonstrated failure of the reported discrete partition. Its interpretation requires bandwidth/scale controls and a corresponding robustness or downstream consequence. Finally, the supplied folder lacks modules imported by principal scripts, including verify_diagnosability.py, verify_power.py, and dpt_training.py. A related training file exists under a different name in the ablation subfolder, but that alone does not resolve the imports. These unresolved dependencies and protocol questions leave the strength and reproducibility of the central empirical claims uncertain.

Questions for authors
  1. Is the target a threshold graph's connected-component partition or the raw pairwise threshold relation? Please give one formal construction and reconcile G.1/G.3, the ε\varepsilon versus τ\tau thresholds in G.2, and the exact-equality limit in Proposition~1. Which existing guarantees survive for the actual implementation?

  2. What exact soft VI functional and memberships qikq_{ik} are used? How does minimizing the implemented BCE on PθP_\theta imply recovery of the thresholded/connected-component partition, and which part of that implication remains valid for β>0\beta>0? Please identify the implemented objective and corresponding guarantee for each reported experiment.

  3. What information is required to form per-mode batches without fine-grained labels? Can the small-nn advantage be demonstrated with an explicit budget table for reference construction, encoder training, calibration, and evaluation, together with baselines given the same available data and grouping information?

  4. For CWRU, does the pairwise degradation persist after controlling representation scale and RBF bandwidth and under the contiguous split included in the updated code? Does it produce a threshold-recovery or robustness consequence beyond a change in MMD? Please identify which split and source files generated Figure~6 and provide the missing modules needed to run the principal scripts.

Reviewer_5ufoOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that classification objectives can fail to preserve the distinguishability between finer-grained classes. As a remedy, the authors propose a differentiable partition-learning objective (DPT) with supporting theory, evaluated on two synthetic benchmarks and the CWRU dataset and compared against authors' own baselines.

Critical strengths

None

Critical weaknesses
  • The paper motivates the idea with an example on the CIFAR-100 dataset, where a classifier is trained to predict coarse "superclass" labels, and argues that it can lead to completely destroying the subclass structure. This setting is never evaluated, and modern self-supervised and contrastive methods that preserve the embedding structure are excluded.
  • The existing evaluation uses two synthetic benchmarks and the CWRU dataset (which is not cited in the paper), with no competitors from the representation learning literature (e.g., SimCLR, reconstruction objectives, GANs, or deep clustering methods) and none of the commonly used datasets (e.g., MNIST, CIFAR, iNaturalist).
  • The paper is hard to read: different terms are used for the same ideas (e.g., labels and latent modes / task partition, confusable-fibre partition, and population partition), and evaluating the theoretical contributions is difficult due to notation and terminology inconsistencies.
  • The provided codebase cannot be run (most scripts import modules that are missing), refers to non-existing sections of the paper, uses hardcoded absolute paths ("/home/claude/..."), and lacks a README, which makes the work non-reproducible.
Questions for authors

None. The concerns listed under Weaknesses require major revisions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.