acceptodds
Under review as a conference paper at ICLR 2027

The Credal Information Bottleneck

Abstract

The information bottleneck (IB) finds a compressed representation that preserves information relevant to a task. Traditional IB defines this compression–prediction tradeoff for a fixed joint input–output distribution. While robust IB variants account for perturbations in that distribution, they do not explicitly preserve the information needed to estimate the uncertainty at the output of the bottleneck. To address this fundamental limitation, we propose the credal information bottleneck (CIB), which for the first time represents uncertainty in the data-generating distribution and the data processing, and propagates it over the bottleneck, enabling its estimation and categorization at the bottleneck output. Importantly, CIB disentangles aleatoric, data-induced and processing-induced uncertainty. This knowledge about the sources of uncertainty allows performing specific and selective interventions at deployment-time. To do so, CIB leverages the formalism of credal sets, i.e., convex combinations of probability distributions, to characterize and propagate the uncertainty and the information about the uncertainty sources through the bottleneck. By replacing point-valued representations with credal sets, CIB fundamentally changes the very definition of information object and information distortion. We introduce variational CIB (VCIB) to make CIB differentiable, and prove that the VCIB objective upper-bounds the CIB up to an additive term. We further prove that every pooled-efficient IB representation incurs a strictly positive credal-distortion floor, which CIB provably overcomes. On three colored benchmarks with environment conflict, VCIB improves post-deployment worst-case accuracy by up to 6.4 percentage points over the strongest baseline.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
information bottleneck, imprecise probability, credal sets, worst risk optimization, robustness, uncertainty quantification, covariate shift, information theory, bayesian deep learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PBerOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes the Credal Information Bottleneck (CIB) for representation learning under uncertainty over data-generating environments and model parameters. It defines environment-wise rate and distortion objectives and establishes a distortion floor for pooled-efficient IB representations. A variational multi-head implementation, VCIB, estimates aleatoric, environment-related, and model-related uncertainty and uses unlabeled test batches to reweight environments. Experiments on three colored classification benchmarks report reduced credal distortion and improved worst-case accuracy after adaptation.

Critical strengths
  1. Preserving environment-specific predictive information through compression is a meaningful problem. The stated separation result clearly identifies when a representation sufficient for pooled prediction may lose information needed to recover environment-specific labeling laws.
  2. The evaluation includes ten seeds, statistical comparisons, and ablations separating model-level credal sets from environment-specific modeling. The paper also reports limitations of its model-uncertainty estimates and selective-prediction performance.
Critical weaknesses
  1. The deployment gains are not fully isolated from the benefits of environment-specific prediction and batch adaptation. The rate-free VCIB ablation provides relevant evidence, but a standard multi-head classifier using the same environmental information and batch-weighting procedure would help determine whether the gains require the proposed credal bottleneck.
  2. The compression comparison matches per-code rates, while predictions combine multiple encoder heads. Appendix A.5 reports joint-rate bounds, but the methods are not compared at matched total rates. Such comparisons, including the rate-free variant, are needed to assess the compression advantage of the complete predictive representation.
  3. The uncertainty experiments mainly assess detection proxies rather than the accuracy of the claimed decomposition. Appendix D provides analytical uncertainty components and qualitative comparisons, but quantitative agreement with these components is not established. Detecting underrepresented inputs or OOD inputs does not directly validate model or aleatoric uncertainty. The weak model-uncertainty results further limit the general disentanglement claim.
  4. All three benchmarks use the same two-environment binary shape–color construction. Changing the underlying images provides some variation, but does not test richer uncertainty structures or scaling with the number of environments. The practical scope of the method remains unclear beyond this construction.
  5. Equation 16 assigns one environment-weight vector to an entire test batch. Deployment is evaluated on homogeneous batches of 100 inputs from known environments, without establishing sensitivity to batch size or mixed-environment batches. These conditions matter because inaccurate environment inference may undermine the reported adaptation advantage.
  6. Appendix A.7 describes the reported distortion as expected excess log-loss, whereas the theoretical distortion measures information lost through the representation. The paper should clarify whether the reported quantity includes decoder approximation error and explain how it is computed consistently across methods. Without this distinction, the results do not clearly isolate representation-level information preservation.

Minor:

  1. Statements about pooled representations should consistently retain the pooled-efficiency qualification required by Theorem 2. Environment-specific decoders trained on frozen pooled features would provide useful empirical context.
  2. Proposition 15 distinguishes control over the generating environments from control over their mixtures. The implications of this distinction for deployment under mixed or previously unseen environments deserve clearer discussion.
  3. Appendix A.6 replaces the learned mixture priors with moment-matched Gaussian priors for environment inference. Report how this choice affects environment identification and deployment accuracy.
  4. Appendix A.2 uses moving-average environment weights and tempered weight-KL terms. Clarify how these implementation choices relate to the stated variational objective and its guarantees.
  5. Report parameter counts and computational costs by method, including the inference cost of multiple hypotheses and batch adaptation.
  6. Clarify the relationship to Group DRO [1], adaptive risk minimization [2], and prior uncertainty-disentanglement benchmarking [3], particularly the distinctions in objectives, adaptation procedures, and validation.

References:

[1] Sagawa, Shiori, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In ICLR, 2020.

[2] Zhang, Marvin, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, and Chelsea Finn. Adaptive risk minimization: Learning to adapt to domain shift. In NeurIPS, 2021.

[3] Mucsányi, Bálint, Michael Kirchhof, and Seong Joon Oh. Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks. In NeurIPS, 2024.

Questions for authors
  1. Does the deployment advantage remain against standard environment-specific classifiers using the same unlabeled batches and weighting procedure?
  2. Do the improvements persist when methods are compared at matched joint representation rates?
  3. How accurately do the estimated uncertainty components match the analytical quantities in Appendix D?
  4. How does adaptation behave with smaller or mixed-environment batches, and how sensitive is it to the post-training choice of code priors?
  5. Does the reported distortion include decoder approximation error? How is it computed consistently across methods?
6 comments
AuthorsOpenReview ↗

Response to PBer (1/6)

Q1/W1. Multi-head classifiers

Comment: Does the deployment advantage remain against standard environment-specific classifiers using the same unlabeled batches and weighting procedure?

Response: We thank you for suggesting this baseline, which we added: a standard deterministic classifier with the same encoder backbone and one output head per environment, trained on the average of the environments' losses, with no Bayesian weights, no stochastic code and no rate term. At deployment, it uses the same unlabeled batches and Eq. 16 as VCIB, with one Gaussian fitted to its features per environment. The comparative results are in Table R11.

Table R11. Multi-head classifier against VCIB at 22 bits and at the next point of its β\beta grid (β=24\beta=24). Rate: R as in Table 1 (bits); ∞\infty: a deterministic code has no finite rate. AUROC_conflict: EUinput\mathrm{EU}_{\mathrm{input}} on classes 0−40-4 against 5−95-9 (Table 2). AUROC_OOD,model: EUmodel\mathrm{EU}_{\mathrm{model}} on the out-of-support inputs of Table 2 against the test inputs. Best pooled: best value among VIB, Ens-VIB, CBDL and VCIB_mod, which use a single epistemic score for both AUROC columns; its D, WC-Acc_int and AUROC_conflict, and those of VCIB (2 bits), are the values of Tabs. 1 and 2. Mean over 10 seeds. ▲/▼: significantly better/worse than the multi-head classifier (seed-paired Wilcoxon signed-rank test, Holm-Bonferroni over the VCIB variants of each dataset and metric, p<0.05p < 0.05).

DataMethodRateD ↓WC-Acc_int ↑AUROC_conflict ↑AUROC_OOD,model ↑
C-MNISTBest pooled1.10.2400.8120.8270.671
Multi-head∞\infty0.0450.8820.9940.507
VCIB(β→∞\beta\to\infty)∞\infty0.0430.8830.9940.836▲
VCIB (2 bits)2.070.055▼0.873▼0.992▼0.689▲
VCIB (β=24\beta=24)2.260.039▲0.8810.9950.774▲
C-KMNISTBest pooled1.3-1.40.3120.7780.7770.619
Multi-head∞\infty0.1510.8320.9570.504
VCIB(β→∞\beta\to\infty)∞\infty0.1570.8300.949▼0.680▲
VCIB (2 bits)2.040.177▼0.821▼0.935▼0.612▲
VCIB (β=24\beta=24)2.600.1430.8340.943▼0.653▲
C-FashionBest pooled1.20.3100.7700.7940.650
Multi-head∞\infty0.1490.8380.9580.508
VCIB(β→∞\beta\to\infty)∞\infty0.1520.8360.9570.816▲
VCIB (2 bits)1.960.165▼0.8340.940▼0.729▲
VCIB (β=24\beta=24)2.390.1500.8370.9580.666▲

The multi-head classifier performs like VCIB(β→∞\beta\to\infty): both keep one readout per environment on an uncompressed code. As such, the gain over pooled methods comes from keeping the environments separate, which is the core idea of the CIB. The rate constraint imposed on VCIB preserves most of these gains. At 2 bits, VCIB is within 0.4−1.10.4-1.1 points of the multi-head accuracy; at β=24\beta=24 (rate 2.3−2.62.3-2.6 bits), its accuracy is not significantly different on any dataset. Furthermore, its credal distortion is not significantly higher on any dataset, and is significantly lower on C-MNIST. VCIB thus matches the multi-head classifier's performance at a rate of 2.3−2.62.3-2.6 bits, while being capable of disentangling uncertainty across model parameters and data generating distributions.

In the final version, we will add the multi-head classifier to Table 1 and Table R11 to Appendix B.

AuthorsOpenReview ↗

Response to PBer (2/6)

Q2/W2. Matched joint rates

Comment: Do the improvements persist when methods are compared at matched joint representation rates? [...] Such comparisons, including the rate-free variant, are needed to assess the compression advantage of the complete predictive representation.

Response: Thank you for the suggestion. VCIB and Ens-VIB transmit five codes, one per encoder hypothesis. RjointR_{\mathrm{joint}} is an upper bound on the information that the five codes together carry about the input, computed with the pairwise-distance bound of [1] (column Rˉjoint\bar R_{\mathrm{joint}} of Table 1; Appendix A.5). For each dataset, we take VCIB at the β\beta of its grid whose RjointR_{\mathrm{joint}} is closest to that of Ens-VIB, the pooled method with the largest joint rate.

Table R12. Comparison at matched joint rate. RjointR_{\mathrm{joint}}: upper bound on the rate of all codes together (bits). VCIB(β→∞\beta\to\infty): rate-free reference (deterministic code, no finite rate), values of Table 1; it is not part of the significance tests. Mean over 1010 seeds. Bold: VCIB, significantly better than every baseline on both metrics (seed-paired Wilcoxon signed-rank test, Holm-Bonferroni over the three baselines, p=0.006p = 0.006 in every case).

DataMethodRjointR_{\mathrm{joint}}D ↓\downarrowWC-Acc_int ↑\uparrow
C-MNISTVIB1.340.2480.810
Ens-VIB2.440.2460.812
VCIB_mod2.020.2430.811
VCIB (β=8\beta=8)2.610.0610.872
VCIB(β→∞\beta\to\infty)∞\infty0.0430.883
C-KMNISTVIB1.750.3150.774
Ens-VIB4.080.3170.778
VCIB_mod3.010.3300.768
VCIB (β=12\beta=12)3.560.1770.821
VCIB(β→∞\beta\to\infty)∞\infty0.1570.830
C-FashionVIB1.510.3140.766
Ens-VIB2.770.3160.769
VCIB_mod2.010.3100.770
VCIB (β=8\beta=8)2.750.1740.836
VCIB(β→∞\beta\to\infty)∞\infty0.1520.836

The results show that, at matched joint rate, VCIB keeps a 1.8−4×1.8-4\times lower distortion and a 4.3−6.64.3-6.6 points higher accuracy after deployment than every pooled method. On C-KMNIST and C-Fashion, it does so at a lower or equal joint rate than Ens-VIB. The distortion of the pooled methods does not move with their joint rate (Ens-VIB on C-MNIST: 0.244/0.242/0.2460.244/0.242/0.246 at 1.76/2.11/2.441.76/2.11/2.44 bits), consistently with Thm. 2(i), to the extent that these encoders are approximately pooled-efficient.

In the final version, we will add Table R12 to Appendix B, and the following sentence to the paragraph "Overcoming the pooling floor" of Section 5: "When all methods are compared at the same joint rate Rjoint, VCIB has a 1.8-4\times lower distortion and a 4.3−6.64.3-6.6 points higher post-deployment accuracy than every pooled method (Appendix B). Raising the joint rate does not lower the distortion of the pooled methods, as predicted by Thm. 2."

[1] A. Kolchinsky and B. D. Tracey, "Estimating Mixture Entropy with Pairwise Distances", Entropy 2017.

Q3/W3. Agreement with Appendix D

Comment: How accurately do the estimated uncertainty components match the analytical quantities in Appendix D?

Response: Thank you for raising this point. To evaluate the uncertainty estimation quality of VCIB against the true uncertainty values, we proceed as follows:

In the colored benchmarks (outlined in Appendix D) labels and colors are sampled with known probabilities. The label equals the class of the shape with probability 0.750.75, while the color equals the label with probability ai(d)a_i(d) which depends only on the environment ii and the class dd.

Hence, the labeling law pi(y∣x)p_i(y\mid x) of each environment is given in closed form (Eq. 67) as a function of the class and color of the image. The true credal set for test image xx is obtained as C∗(x)=convp1(⋅∣x),p2(⋅∣x)C^{*}(x)=\mathrm{conv}\\{p_1(\cdot\mid x),p_2(\cdot\mid x)\\} and its exact aleatoric uncertainty (Def. 4), AU∗(x)=min⁡p∈C∗(x)H(p)\mathrm{AU}^{*}(x)=\min_{p\in C^{*}(x)}H(p), the lowest Shannon entropy (in bits) in the set. Methods without our decomposition are scored with their own AU estimate: the predictive entropy (VIB), the mean entropy of the members (Ens-VIB), and the lowest entropy in the credal set (CBDL, VCIB_mod).

AuthorsOpenReview ↗

Response to PBer (3/6)

Q3/W3. Agreement with Appendix D (continued)

Appendix D defines three input groups: classes 0-4, on which the two environments give the same image different labels (50% of the test set, AU∗=0.22\mathrm{AU}^{*}=0.22 bits); classes 5-9 whose color agrees with the label (34%, AU∗=0.22\mathrm{AU}^{*}=0.22); and classes 5-9 whose color disagrees with it, where the label itself is random (15%, AU∗=0.81\mathrm{AU}^{\ast}=0.81). Only the third group has a high AU∗\mathrm{AU}^{*}; as such, a good AU estimate must rank its images above the images from a low-AU environment. We measure this with the AUROC of each method's AU, with the noisy images as positives and one other group as negatives. Additionally, we compare against a fourth group thinned, from the 22\\% dataset variants of Table 2.

Table R1. We show the AUROC of each method's AU for separating the label-noise images from each low-AU comparison group. Mean over 10 seeds. ▼ marks a baseline significantly worse than VCIB (paired Wilcoxon signed-rank test, Holm-Bonferroni corrected, p<0.05p < 0.05). Bold: best per row.

DataNoisy vs.VIBEns-VIBIB-IRMCBDLVCIB_modVCIB
C-MNISTcolor agrees0.969▼0.976▼0.693▼0.984▼0.975▼0.997
env. disagree0.503▼0.497▼0.525▼0.607▼0.513▼0.990
thinned (22\\%)0.540▼0.510▼0.498▼0.628▼0.544▼0.966
C-KMNISTcolor agrees0.882▼0.924▼0.592▼0.9460.913▼0.960
env. disagree0.476▼0.455▼0.524▼0.570▼0.479▼0.909
thinned (22\\%)0.532▼0.539▼0.563▼0.663▼0.573▼0.829
C-Fashioncolor agrees0.877▼0.882▼0.832▼0.884▼0.886▼0.947
env. disagree0.425▼0.405▼0.520▼0.395▼0.439▼0.860
thinned (22\\%)0.676▼0.677▼0.665▼0.675▼0.688▼0.891

Averaged over the three datasets, the AU estimated by VCIB separates label noise from environment disagreement with an AUROC of 0.920.92, against 0.520.52 for the best baseline (CBDL). Likewise, VCIB separates label noise from missing training data with an AUROC of 0.900.90, against 0.660.66 for CBDL. Thus, VCIB is the only method among baselines that is fundamentally capable of isolating label noise from the other two sources of uncertainty. Pooled models are incapable of separating environment disagreement from label noise (see Cor. 13). In stark contrast, VCIB attributes this disagreement to EU_input instead with an AUROC of 0.93−0.990.93-0.99 (Table 2).

Moreover, we report the mean absolute error of AU against AU∗\mathrm{AU}^{*} over the test set, on the full data and on the 22\\% variant, whose test set and labeling laws, and hence AU∗\mathrm{AU}^{*}, are unchanged.

Table R2. Mean absolute error of AU against AU∗\mathrm{AU}^{*} (bits) over the test set, mean ± std over 10 seeds. ▼ marks a baseline significantly worse than VCIB (same test and correction as Table R1; p=0.01p = 0.01 in every case, as VCIB has the lower error on all 10 seeds). Bold: best per row.

DataVIBEns-VIBIB-IRMCBDLVCIB_modVCIB
C-MNIST0.367 ± 0.010▼0.362 ± 0.007▼0.682 ± 0.005▼0.313 ± 0.004▼0.317 ± 0.007▼0.078 ± 0.007
C-MNIST 2%0.381 ± 0.013▼0.373 ± 0.008▼0.687 ± 0.002▼0.329 ± 0.008▼0.331 ± 0.008▼0.091 ± 0.007
C-KMNIST0.402 ± 0.012▼0.392 ± 0.009▼0.687 ± 0.002▼0.325 ± 0.005▼0.352 ± 0.008▼0.133 ± 0.015
C-KMNIST 2%0.401 ± 0.015▼0.397 ± 0.010▼0.686 ± 0.003▼0.320 ± 0.007▼0.346 ± 0.010▼0.131 ± 0.008
C-Fashion0.406 ± 0.010▼0.406 ± 0.006▼0.660 ± 0.017▼0.351 ± 0.005▼0.363 ± 0.006▼0.151 ± 0.012
C-Fashion 2%0.399 ± 0.011▼0.394 ± 0.011▼0.673 ± 0.016▼0.343 ± 0.008▼0.355 ± 0.005▼0.145 ± 0.005

Averaged over the three datasets, VCIB's AU estimation error is 2.7× lower (0.121 bits against 0.330) than the best baseline (CBDL), and 3.5× lower than the average baseline. Removing 98% of the training images of classes 2 and 7 leaves the error unchanged (0.122 bits); as such, VCIB does not mistake missing training data for label noise, consistently with Table R1.

In the final version, we will add a paragraph on the ground truth of AU to Section 5, after the discussion of Table 2, and Tables R1 and R2 to Appendix B.

Reviewer_F1B5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces the Credal Information Bottleneck (CIB), which replaces point-valued predictive targets with credal sets to preserve uncertainty about both data-generating environments and model processing through a compressed representation. It defines a worst-environment rate–distortion objective and proves that pooled-efficient information bottleneck representations can have a positive credal-distortion floor, while CIB can reduce distortion as the relevance weight increases. The authors then propose a differentiable variational implementation (VCIB), derive an upper-bound relationship to the CIB objective, and decompose predictive uncertainty into aleatoric, environment-induced epistemic, and model-induced epistemic components. Experiments on Colored-MNIST, Colored-KMNIST, and Colored Fashion-MNIST report lower worst-environment distortion, strong environment-conflict detection, and improved worst-case accuracy after unlabeled test-time reweighting. However, the evaluation is limited to three related synthetic/controlled benchmarks, and model-level uncertainty estimation is comparatively weak.

Critical strengths
  1. The paper gives a clear formalization of uncertainty-aware compression using credal sets and provides meaningful guarantees, including a characterization of the pooling distortion floor and conditions under which CIB can overcome it.
  2. Across three benchmarks, VCIB substantially reduces worst-environment distortion and improves post-intervention worst-case accuracy, while achieving high AUROC for detecting environment disagreement.
Critical weaknesses
  1. All experiments use closely related colored-image benchmarks with a two-environment construction. It remains unclear whether the method scales to more realistic distribution shifts, more environments, or higher-complexity tasks.
  2. The reported model-epistemic uncertainty detection is modest and sometimes not better than the baselines; on C-KMNIST, VCIB is worse than the VIB ensemble on this metric. This weakens the broad claim that the method reliably separates all uncertainty sources.
  3. Methods are selected by the grid point closest to a 2-bit rate, but the realized rates and total rates across multiple codes differ. In addition, test-time environment weighting relies on unlabeled batches and environment-specific code priors, whose robustness under batch composition changes is not established.
Questions for authors
  1. Can the authors provide evidence on datasets with more than two environments or on a less controlled distribution shift to establish how well CIB generalizes beyond colored-image benchmarks?
  2. Why is EU_model detection relatively weak at a finite rate budget, and do additional capacity or calibration studies improve it consistently across all three datasets?
  3. How sensitive are the test-time environment weights and reported gains to batch size, batch composition, and mismatch between test environments and the training environment set?
5 comments
AuthorsOpenReview ↗

Response to F1B5 (1/5)

Q1/W1. More environments and a less controlled shift

Comment: Can the authors provide evidence on datasets with more than two environments or on a less controlled distribution shift to establish how well CIB generalizes beyond colored-image benchmarks?

Response: Thank you for the suggestion. We ran all methods on ACS public coverage [1], a real tabular benchmark with nine environments. Each datapoint is a person under 65 sampled from the 2018 US census. The input features consist of 18 census variables (age, education, income, employment, disability, citizenship, etc.), while the target label is whether the person has public health coverage. The environments are the nine Census Bureau divisions (e.g., Pacific, South Atlantic). Considering the same census variables, whether a person is covered depends on the state in which that person resides. Hence, the labeling law pi(y∣x)p_i(y\mid x) changes across environments, as shown in [2]. Since the true labeling laws are unknown, DD replaces Hi(Y∣X)H_i(Y\mid X) with the test log-loss of a reference network trained on the same environment without a bottleneck, as in Thm. 3.

Table R5. ACS public coverage, nine Census divisions, with the 2-bit protocol of Section 5 and a shared MLP encoder. Rate: rate of one code (R in Table 1, bits); RjointR_{\mathrm{joint}}: rate of all codes together (Rˉjoint\bar R_{\mathrm{joint}} in Table 1); -: no rate term. D: worst-division distortion (bits); WC-Acc: lowest accuracy over the nine divisions. Mean over 10 seeds. ▲/▼: significantly better/worse than VCIB (seed-paired Wilcoxon signed-rank test, Holm-Bonferroni over the six other methods, p<0.05p < 0.05). Bold: best method with a rate term.

MethodRateRjointR_{\mathrm{joint}}D ↓WC-Acc ↑
VIB2.152.810.037▼0.725▼
Ens-VIB2.255.870.038▼0.727▼
IB-IRM1.972.670.073▼0.721▼
CBDL--0.034▼0.726▼
VCIB_mod2.174.220.035▼0.725▼
VCIB(β→∞\beta\to\infty)--0.003▲0.731▲
VCIB2.204.290.0110.729

The results show that VCIB's worst-division distortion is 3.1×3.1\times lower than that of the best pooled method (0.0110.011 vs. 0.0340.034), at a lower joint rate than Ens-VIB (4.294.29 vs. 5.875.87 bits). Furthermore, its worst-environment accuracy is significantly higher than that of every pooled method. In the final version, we will add this real-world data benchmark to Section 5 and its details to Appendix A.

[1] F. Ding et al., "Retiring Adult: New Datasets for Fair Machine Learning", NeurIPS 2021.

[2] J. Liu et al., "On the Need for a Language Describing Distribution Shifts: Illustrations on Tabular Datasets", NeurIPS 2023.

AuthorsOpenReview ↗

Response to F1B5 (2/5)

W3. Rate matching

Comment: Methods are selected by the grid point closest to a 2-bit rate, but the realized rates and total rates across multiple codes differ.

Response: We thank the reviewer for raising this point. The pooled baselines with rate constraints never reach 2 bits: their rate saturates at 1.1−1.41.1-1.4 bits over the whole β\beta grid (column R of Table 1; Appendix A.4). To strengthen our evaluation, we compared all methods at a matched joint rate. VCIB and Ens-VIB transmit five codes, one per encoder hypothesis. RjointR_{\mathrm{joint}} is an upper bound on the information that the five codes together carry about the input, computed with the pairwise-distance bound of [1] (column Rˉjoint\bar R_{\mathrm{joint}} of Table 1; Appendix A.5). For each dataset, we take VCIB at the β\beta of its grid whose RjointR_{\mathrm{joint}} is closest to that of Ens-VIB, the pooled method with the largest joint rate. The results are shown in Table R12.

Table R12. Comparison at matched joint rate. RjointR_{\mathrm{joint}}: upper bound on the rate of all codes together (bits). VCIB(β→∞\beta\to\infty): rate-free reference (deterministic code, no finite rate), values of Table 1; it is not part of the significance tests. Mean over 1010 seeds. Bold: VCIB, significantly better than every baseline on both metrics (seed-paired Wilcoxon signed-rank test, Holm-Bonferroni over the three baselines, p=0.006p = 0.006 in every case).

DataMethodRjointR_{\mathrm{joint}}D ↓\downarrowWC-Acc_int ↑\uparrow
C-MNISTVIB1.340.2480.810
Ens-VIB2.440.2460.812
VCIB_mod2.020.2430.811
VCIB (β=8\beta=8)2.610.0610.872
VCIB(β→∞\beta\to\infty)∞\infty0.0430.883
C-KMNISTVIB1.750.3150.774
Ens-VIB4.080.3170.778
VCIB_mod3.010.3300.768
VCIB (β=12\beta=12)3.560.1770.821
VCIB(β→∞\beta\to\infty)∞\infty0.1570.830
C-FashionVIB1.510.3140.766
Ens-VIB2.770.3160.769
VCIB_mod2.010.3100.770
VCIB (β=8\beta=8)2.750.1740.836
VCIB(β→∞\beta\to\infty)∞\infty0.1520.836

The results show that, at matched joint rate, VCIB keeps a 1.8-4× lower distortion and a 4.3-6.6 points higher accuracy after deployment than every pooled method. On C-KMNIST and C-Fashion, it does so at a lower or equal joint rate than Ens-VIB. The distortion of the pooled methods does not move with their joint rate (Ens-VIB on C-MNIST: 0.244/0.242/0.246 at 1.76/2.11/2.44 bits), consistently with Thm. 2(i), to the extent that these encoders are approximately pooled-efficient. In the final version, we will state the saturation of the pooled rates in Section 5, add Table R12 to Appendix B, and add the following sentence to the paragraph "Overcoming the pooling floor" of Section 5: "When all methods are compared at the same joint rate Rˉjoint\bar R_{\mathrm{joint}}, VCIB has a 1.8-4× lower distortion and a 4.3-6.6 points higher post-deployment accuracy than every pooled method (Appendix B). Raising the joint rate does not lower the distortion of the pooled methods, as predicted by Thm. 2."

[1] A. Kolchinsky and B. D. Tracey, "Estimating Mixture Entropy with Pairwise Distances", Entropy 2017.

AuthorsOpenReview ↗

Response to F1B5 (3/5)

Q2/W2. EU_model at a finite rate

Comment: Why is EU_model detection relatively weak at a finite rate budget [...]? [...] on C-KMNIST, VCIB is worse than the VIB ensemble on this metric.

Response: We thank the reviewer for giving us the opportunity to further clarify this point. As in Table 2, AUROC_model is the AUROC of EUmodel\mathrm{EU}_{\mathrm{model}} for separating the test images of the thinned classes 2 and 7 from the others, on the 22\\% variant. EUmodel\mathrm{EU}_{\mathrm{model}} comes from the spread of the model hypotheses. Information about this spread can only reach the decoders through the code. Under a rate budget, the code first allocates the available bits to encode information on labels and environment disagreement, thus reducing the distortion. Consequently, the remaining rate can be allocated to express how uncertain the model is, as discussed in Section 5 (paragraph "Uncertainty disentanglement"). We demonstrated this experimentally by extending VCIB's β\beta grid above the 2-bit budget. The results are reported in Table R6.

Table R6. AUROC_model of VCIB along its β\beta grid on the 22\\% variant, with the test rate RR (bits) in parentheses. No rate: VCIB(β→∞\beta\to\infty), as in Table 2. Mean over 10 seeds.

Dataβ=8\beta=81216243264No rate
C-MNIST0.59 (1.8)0.61 (2.0)0.61 (2.1)0.70 (2.4)0.72 (2.6)0.70 (3.0)0.76
C-KMNIST0.57 (1.9)0.59 (2.1)0.58 (2.3)0.66 (2.6)0.67 (2.8)0.67 (3.6)0.72
C-Fashion0.55 (1.7)0.55 (1.9)0.54 (2.0)0.53 (2.4)0.54 (2.6)0.52 (3.1)0.48

On C-MNIST and C-KMNIST, AUROC_model tends to increase with the rate: it rises from 0.59-0.61 at 2 bits to 0.66−0.720.66-0.72 at 2.4−2.82.4-2.8 bits, and then levels off below the rate-free value. On C-Fashion AUROC_model stays between 0.520.52 and 0.550.55 at every rate, and the rate-free variant is lower (0.4800.480 against 0.5400.540); hence, the rate explanation does not account for that dataset.

Regarding Ens-VIB, at 2 bits it is ahead of VCIB on C-KMNIST (AUROC_model 0.6370.637 vs. 0.5900.590, Table 2), and level with it on C-MNIST and C-Fashion. With a slightly higher rate, the order is reversed: at β=24\beta=24 (rate 2.4−2.62.4-2.6 bits), VCIB reaches 0.6960.696 on C-MNIST and 0.6630.663 on C-KMNIST, significantly above Ens-VIB (0.6420.642 and 0.6370.637; seed-paired Wilcoxon signed-rank test, Holm-Bonferroni over the four VCIB grid points, p≤0.014p \leq 0.014), and stays level with it on C-Fashion.

Reviewer_rriyOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to compress a representation while preserving information needed for prediction across different environments. The proposed CIB retains a set of possible predictive distributions, allowing differences between environments to survive compression. The authors also introduce a trainable version, VCIB, which aims to distinguish label noise, disagreement between environments, and model uncertainty. On three colored-image benchmarks, VCIB reduces distortion and improves worst-case accuracy after adaptation by up to 6.4 percentage points.

Critical strengths

1, The theoretical motivation is clear. Theorem 2 explains when pooling loses environment-specific information. 2, The practical formulation is well connected to the theory. Theorem 3 links the trainable VCIB loss to the original CIB objective through an explicit bound that accounts for reference-model error. 3, Figure 3 and the VCIB_mod ablation support preserving environment structure. Strong conflict detection and improved post-adaptation accuracy demonstrate practical value in the tested settings.

Critical weaknesses
  1. The main-text results do not quantitatively validate AU against ground truth. Table 2 tests whether AU detects out-of-distribution inputs. However, an unfamiliar input can still have a clear label, while familiar inputs can have noisy labels. Good OOD detection therefore does not establish accurate AU estimates.

Figure 2 provides a visual comparison, but does not report the error against the AU target in Eq. (15). This leaves insufficient evidence that the method can distinguish label uncertainty from uncertainty caused by limited training data or unfamiliar inputs.

  1. It is unclear which parts of VCIB explain the gains. VCIB_mod also removes separate modeling of the environments, and the baselines use a different adaptation method (Section 5). These comparisons do not show how much VCIB adds beyond learning separate predictors for each environment and adapting them at test time.

Also, Eq. (16) sets environment weights without directly using the three uncertainty scores in Eq. (15). Higher accuracy after adaptation therefore does not demonstrate the benefit of separating uncertainty sources.

Questions for authors
  1. How closely does estimated AU match the targets in Appendix D? Please report estimation error, such as mean absolute error, for the three input groups described there. In the existing experiment retaining only 2% of the training images for classes 2 and 7, how does AU estimation error change on the unchanged test set?

  2. Do simpler alternatives retain the gains? Keeping the same encoders, environment-specific decoders and priors, and adaptation in Eq. (16), what changes if Eq. (13)'s emphasis on the worst environment is replaced by a fixed weighted average across all environments? Please match actual coding rates, not just beta, and report worst-environment distortion and accuracy after adaptation. Separately, with trained predictions and environment weights fixed, does accuracy change if you replace the paper's decision rule by averaging predictions within each environment, combining them using those weights, and choosing the most likely label?

5 comments
AuthorsOpenReview ↗

Response to rriy (1/4)

Q1/W1. AU against the ground truth

Comment: How closely does estimated AU match the targets in Appendix D? Please report estimation error, such as mean absolute error, for the three input groups described there.

Response: We thank you for giving us the opportunity to clarify this point. We compare estimated AU directly with the exact values as follows.

In the colored benchmarks (outlined in Appendix D) labels and colors are sampled with known probabilities. The label equals the class of the shape with probability 0.75, while the color equals the label with probability ai(d)a_i(d) which depends only on the environment ii and the class dd.

Hence, the labeling law pi(y∣x)p_i(y\mid x) of each environment is given in closed form (Eq. 67) as a function of the class and color of the image. The true credal set for test image xx is obtained as C∗(x)=convp1(⋅∣x),p2(⋅∣x)C^{*}(x)=\mathrm{conv}\\{p_1(\cdot\mid x),p_2(\cdot\mid x)\\} and its exact aleatoric uncertainty (Def. 4) is AU∗(x)=min⁡p∈C∗(x)H(p)\mathrm{AU}^{*}(x)=\min_{p\in C^{\ast}(x)}H(p), the lowest Shannon entropy (in bits) in the set. Methods without our decomposition are scored with their own AU estimate: the predictive entropy (VIB), the mean entropy of the members (Ens-VIB), and the lowest entropy in the credal set (CBDL, VCIB_mod).

Appendix D defines three input groups: classes 0-4, on which the two environments give the same image different labels (50% of the test set, AU∗=0.22\mathrm{AU}^{*}=0.22 bits); classes 5-9 whose color agrees with the label (34%, AU∗=0.22\mathrm{AU}^{\ast}=0.22); and classes 5-9 whose color disagrees with it, where the label itself is random (15%, AU∗=0.81\mathrm{AU}^{*}=0.81). Only the third group has a high AU∗\mathrm{AU}^{*}; as such, a good AU estimate must rank its images above the images from a low-AU environment. We measure this with the AUROC of each method's AU, with the noisy images as positives and one other group as negatives. Additionally, we compare against a fourth group thinned, from the 22\\% dataset variants of Table 2.

Table R1. We show the AUROC of each method's AU for separating the label-noise images from each low-AU comparison group. Mean over 1010 seeds. ▼ marks a baseline significantly worse than VCIB (paired Wilcoxon signed-rank test, Holm-Bonferroni corrected, p<0.05p < 0.05). Bold: best per row.

DataNoisy vs.VIBEns-VIBIB-IRMCBDLVCIB_modVCIB
C-MNISTcolor agrees0.969▼0.976▼0.693▼0.984▼0.975▼0.997
env. disagree0.503▼0.497▼0.525▼0.607▼0.513▼0.990
thinned (22\\%)0.540▼0.510▼0.498▼0.628▼0.544▼0.966
C-KMNISTcolor agrees0.882▼0.924▼0.592▼0.9460.913▼0.960
env. disagree0.476▼0.455▼0.524▼0.570▼0.479▼0.909
thinned (22\\%)0.532▼0.539▼0.563▼0.663▼0.573▼0.829
C-Fashioncolor agrees0.877▼0.882▼0.832▼0.884▼0.886▼0.947
env. disagree0.425▼0.405▼0.520▼0.395▼0.439▼0.860
thinned (22\\%)0.676▼0.677▼0.665▼0.675▼0.688▼0.891

Averaged over the three datasets, the AU estimated by VCIB separates label noise from environment disagreement with an AUROC of 0.92, against 0.52 for the best baseline (CBDL). Likewise, VCIB separates label noise from missing training data with an AUROC of 0.90, against 0.66 for CBDL. Thus, VCIB is the only method among baselines that is fundamentally capable of isolating label noise from the other two sources of uncertainty. Pooled models are incapable of separating environment disagreement from label noise (see Cor. 13). In stark contrast, VCIB attributes this disagreement to EU_input instead with an AUROC of 0.93-0.99 (Table 2).

AuthorsOpenReview ↗

Response to rriy (2/4)

Q1/W1 (continued). Estimation error on the 22\\% variant

Comment: In the existing experiment retaining only 22\\% of the training images for classes 2 and 7, how does AU estimation error change on the unchanged test set?

Response: As requested, we report the mean absolute error of AU against AU∗\mathrm{AU}^{*} over the test set, on the full data and on the 22\\% variant, whose test set and labeling laws, and hence AU∗\mathrm{AU}^{*}, are unchanged.

Table R2. Mean absolute error of AU against AU∗\mathrm{AU}^{*} (bits) over the test set, mean ± std over 10 seeds. ▼ marks a baseline significantly worse than VCIB (same test and correction as Table R1; p=0.010p = 0.010 in every case, as VCIB has the lower error on all 10 seeds). Bold: best per row.

DataVIBEns-VIBIB-IRMCBDLVCIB_modVCIB
C-MNIST0.367 ± 0.010▼0.362 ± 0.007▼0.682 ± 0.005▼0.313 ± 0.004▼0.317 ± 0.007▼0.078 ± 0.007
C-MNIST 2%0.381 ± 0.013▼0.373 ± 0.008▼0.687 ± 0.002▼0.329 ± 0.008▼0.331 ± 0.008▼0.091 ± 0.007
C-KMNIST0.402 ± 0.012▼0.392 ± 0.009▼0.687 ± 0.002▼0.325 ± 0.005▼0.352 ± 0.008▼0.133 ± 0.015
C-KMNIST 2%0.401 ± 0.015▼0.397 ± 0.010▼0.686 ± 0.003▼0.320 ± 0.007▼0.346 ± 0.010▼0.131 ± 0.008
C-Fashion0.406 ± 0.010▼0.406 ± 0.006▼0.660 ± 0.017▼0.351 ± 0.005▼0.363 ± 0.006▼0.151 ± 0.012
C-Fashion 2%0.399 ± 0.011▼0.394 ± 0.011▼0.673 ± 0.016▼0.343 ± 0.008▼0.355 ± 0.005▼0.145 ± 0.005

Averaged over the three datasets, VCIB's AU estimation error is 2.7×2.7\times lower (0.121 bits against 0.330) than the best baseline (CBDL), and 3.5×3.5\times lower than the average baseline. Removing 98% of the training images of classes 2 and 7 leaves the error unchanged (0.122 bits); as such, VCIB does not mistake missing training data for label noise, consistently with Table R1. In the final version, we will add a paragraph on the ground truth of AU to Section 5, after the discussion of Table 2, and Tables R1 and R2 to Appendix B.

Q2a. Fixed average instead of the worst case

Comment: Keeping the same encoders, environment-specific decoders and priors, and adaptation in Eq. (16), what changes if Eq. (13)'s emphasis on the worst environment is replaced by a fixed weighted average across all environments? Please match actual coding rates, not just beta, and report worst-environment distortion and accuracy after adaptation.

Response: Thank you for the suggestion. We trained the variant you propose, VCIB-avg. For each encoder, VCIB takes the maximum over environments of a per-environment term (a bound on that environment's rate plus β\beta times its lost information), while VCIB-avg takes their uniform average. To match the actual coding rates, β\beta is, as for every method, the grid value whose training rate is closest to 2 bits. The realized rates RR (as in Table 1) are 1.98−2.021.98-2.02 bits for VCIB-avg and 1.96−2.071.96-2.07 bits for VCIB.

Table R3. VCIB-avg against VCIB at matched rate. Column D marks the worst-environment distortion (excess log-loss, bits); WC-Acc_int and WC-Reg_int: worst-environment accuracy and regret after deployment, all as in Table 1. Mean over 10 seeds. ▼ marks VCIB-avg significantly worse than VCIB (seed-paired Wilcoxon signed-rank test, Holm-Bonferroni within each dataset and metric, p<0.05p < 0.05). Bold: significantly better value.

DataMethodD ↓\downarrowWC-Acc_int ↑\uparrowWC-Reg_int ↓\downarrow
C-MNISTVCIB-avg0.060▼0.8700.029▼
VCIB0.0550.8730.026
C-KMNISTVCIB-avg0.184▼0.813▼0.085
VCIB0.1770.8210.079
C-FashionVCIB-avg0.174▼0.8330.069
VCIB0.1650.8340.068

At the same rate, VCIB gives a significantly lower credal distortion on all benchmarks (by 4-9%). VCIB-avg loses 0.8 points of accuracy after deployment on C-KMNIST. The worst case in Eq. 13 directly targets the credal DD (Eq. 6), while an average only controls the mean: the code may lose information about one environment as long as it retains more about the others. In the final version, we will add VCIB-avg to the ablations of Section 5 and Table R3 to Appendix B.

AuthorsOpenReview ↗

Response to rriy (3/4)

Q2b. Simpler decision rule

Comment: With trained predictions and environment weights fixed, does accuracy change if you replace the paper's decision rule by averaging predictions within each environment, combining them using those weights, and choosing the most likely label?

Response: Thank you for the suggestion. We compared the credal decision rule of Section 4.3 with the rule you propose, on the same checkpoints and with the same environment weights wi(B)w_i(B) of Eq. 16. For each environment i, VCIB has KJ=15KJ=15 predictions p^iℓ(⋅∣x)\hat p_{i\ell}(\cdot\mid x), one per model hypothesis ℓ\ell. Without deployment, both rules use the per-input environment weights of the default decision (Appendix A.7).

Table R4. Worst-environment accuracy of our credal rule and of the simple rule. int: after deployment on unlabeled batches of 100; default: without deployment. Δ\Delta: simple minus ours (points); (*) marks a significant Δ\Delta (seed-paired Wilcoxon signed-rank test over 1010 seeds, Holm-Bonferroni over the two settings of each dataset, p<0.05p < 0.05). Re-evaluation of the same models with new Monte Carlo draws and batches (within 0.2 points of Table 1).

DataOurs (int)Simple (int)Δ\DeltaOurs (default)Simple (default)Δ\Delta
C-MNIST0.8730.8730.00.8050.809+0.4
C-KMNIST0.8200.821+0.10.7790.777−0.2*
C-Fashion0.8330.8330.00.7720.771−0.1

After deployment, the two rules reach the same accuracy on all three datasets (none of the differences is significant). Without deployment, the differences stay within 0.4 points; the only significant one is on C-KMNIST, where our rule is 0.2 points more accurate. Thus, the gains of VCIB come from the environment-specific credal set and its weights, not from the decision rule. In the final version, we will add Table R4 to Appendix B.

W2. Role of the decomposition

Comment: Eq. (16) sets environment weights without directly using the three uncertainty scores in Eq. (15). Higher accuracy after adaptation therefore does not demonstrate the benefit of separating uncertainty sources.

Response: The reviewer is correct that Eq. 16 does not directly use the three scores as input. However, the uncertainty decomposition of VCIB is what makes the intervention possible in the first place. Eq. 16 intervenes on exactly one source of uncertainty, EUinput\mathrm{EU}_{\mathrm{input}} , by reweighting the environments of the credal set (C^B(x)=∑iwi(B),C^i(x)\widehat{C}_B(x)=\sum_i w_i(B)\\,\widehat{C}_i(x)), and leaves AU and EUmodel\mathrm{EU}_{\mathrm{model}} unaffected. This is only possible because VCIB keeps the environment uncertainty separate. A pooled model merges all sources into one predictive distribution and thus cannot reweight the environments at inference time. Hence, the accuracy of pooled methods changes by at most 0.4 points after intervention (Section 5, Table 1).

The uncertainty signal can also predict where the intervention is most likely to help: as shown in Table 4, high values of EUinput\mathrm{EU}_{\mathrm{input}} correlate with post-intervention accuracy gains. On classes 0-4, where the environments disagree, accuracy rises from 0.7260.726 to the Bayes accuracy of 0.8790.879; on classes 5-9, where EUinput≈0\mathrm{EU}_{\mathrm{input}}\approx 0, it stays unchanged.

In the final version, we will add this reading of Eq. 16 to Section 4.3, and the analysis of Table 4 to Section 5.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response and summary of revisions

Dear Area Chair and Reviewers,

We thank all reviewers for their constructive feedback, which has helped us strengthen the paper.

We appreciate that the reviewers recognized the strengths of our work. In particular, they found the theoretical motivation clear, the result of Thm. 2 meaningful [rriy, F1B5, PBer], and the practical formulation well connected to the theory through Thm. 3 [rriy]. They also considered the preservation of environment-specific information through compression a meaningful problem [PBer]. Finally, they noted the strong conflict detection and the improved post-adaptation accuracy of VCIB [rriy, F1B5].

We summarize our contributions below:

  • The CIB is a new information bottleneck theory in which the information object is a credal set, such that the uncertainty about the data-generating environment and the model survives compression, with a worst-environment credal rate and distortion.
  • Fundamental limits of pooled IB (Thm. 2). Every pooled-efficient IB representation incurs a strictly positive credal-distortion floor at every rate, whereas the credal distortion of the CIB vanishes as β\beta grows. As a consequence, uncertainty estimates computed from pooled-efficient representations have an irreducible error (Cor. 13).
  • Variational credal bound (Thm. 3). The differentiable VCIB objective upper-bounds the CIB objective, so minimizing it trains the CIB end to end. The bound holds for every mixture of the training environments (Prop. 15).

We have uploaded a revised manuscript, in which all changes are marked in red. Following the Reviewers' suggestions, we have performed 14 new experiments. All of them use 10 seeds, with paired Wilcoxon signed-rank tests and Holm-Bonferroni correction. The results are summarized below.

  • Quantitative validation of the decomposition. [rriy W1/Q1, PBer W3/Q3] The reviewers asked us to compare the estimated AU with the ground truth. Since the labeling laws of our benchmarks are known, the exact AU of every test image is computable. Against the ground truth, the AU of VCIB distinguishes label noise from environment disagreement (average AUC 0.920.92, against 0.520.52 for the best baseline) and from missing training data (0.900.90 against 0.660.66), with a 2.7×2.7\times lower error than the best baseline (Tables R1-R2). (Sec. 5, App. B.1.)
  • Evaluation on real data. [F1B5 W1/Q1, PBer W4] On ACS public coverage, VCIB has a 3.1×3.1\times lower worst-division distortion than the best pooled method and a significantly higher worst-division accuracy (Table R5). (Sec. 5, App. A.9.)
  • Source of the gains. [rriy W2/Q2, PBer W1/Q1] Replacing the worst case with an average increases the distortion at the same rate (Table R3). Using a simpler decision rule gives the same accuracy (Table R4). A multi-head classifier performs like VCIB without a rate term, showing that the gain comes from environment separation. VCIB matches its accuracy at a rate of 2.3−2.62.3-2.6 bits, while also estimating model-level uncertainty (Table R11). We show that intervention requires estimation of EUinput\mathrm{EU}_{\mathrm{input}}, which a pooled model cannot disentangle. (Tab. 1, Sec. 4.3, 5, App. B.2.)
  • Rate matching. [F1B5 W3, PBer W2/Q2] At the same joint rate, VCIB has a 1.8−4×1.8-4\times lower distortion and a 4.3−6.64.3-6.6 points higher accuracy after deployment than every pooled method, whose distortion does not decrease as their rate grows (Table R12). (Sec. 5, App. B.3.)
  • Model-level uncertainty. [F1B5 W2/Q2] On C-MNIST and C-KMNIST, the quality of EUmodel\mathrm{EU}_{\mathrm{model}} is rate-dependent. EUmodel\mathrm{EU}_{\mathrm{model}} detection rises from 0.59−0.610.59-0.61 at 22 bits to 0.66−0.720.66-0.72 at 2.4−2.82.4-2.8 bits, where VCIB is significantly better than Ens-VIB, while capacity and calibration leave it unchanged (Tables R6-R8). (Sec. 5, App. B.4.)
  • Deployment conditions. [F1B5 W3/Q3, PBer W5/Q4, minor 2-3] We report the effect of batch size, mixed batches, unseen environments and code priors (Tables R9, R10, R13). We show that the default decision covers every mixture of the training environments (Prop. 15) staying within 1 point of the Bayes rule on unseen mixtures. (Sec. 4.3, 6, App. A.6, B.5.)
  • Distortion, cost and related work. [PBer W6/Q5, minor 1, 4-6] Decoders retrained on frozen encoders reproduce the reported distortion within 0.040.04 bits for every method with a rate term (Table R14). We added costs (Table R15), the pooled-efficiency qualifier throughout, the relation of the training choices to Thm. 3, and a discussion of Group DRO, ARM and disentanglement benchmarks. (Sec. 1-6, App. A.2, A.8.)

We thank you again for your feedback. We believe these new results help clarify the contribution of the paper and improve the quality of our evaluation. We remain available to clarify any further concern.

Kind Regards, The Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.