acceptodds
Under review as a conference paper at ICLR 2027

Task-Aware Certified Block Frames for Exact Latent Recovery

21 views

Abstract

Most block-erasure designs optimize reconstruction stability. With erasure locations known at decoding, every full-column-rank survivor frame recovers the same latent representation in the zero-noise case. Under bounded observation noise, however, survivor geometry determines the amplification of each class-pair score difference. We exploit this remaining design freedom to reduce decision uncertainty while preserving exact recovery of the full latent representation at zero noise. For a linear head and a total observation-noise budget, we derive the worst-case pairwise score uncertainty and the largest radius that preserves the multiclass prediction over all survivor sets; an explicit perturbation reaches its boundary. The resulting certificate identifies the decision directions that the block design must protect. Task-Aware Certified Block Frames (TACBF) combines closed-form task-subspace allocation with learned rank-constrained block geometry. Across held-out test sets from seven datasets, TACBF improves both grid-based and continuous-radius measures of the area under the certified-accuracy curve (AUC) relative to a fixed-factor control paired by training seed. Continuous-radius AUC improves for every paired seed on all seven datasets, and TACBF has the highest mean grid AUC among seven methods on every dataset. The method preserves exact zero-noise recovery, transmitted width, and trace without changing the inference procedure; dataset-level mean clean accuracy differs by at most percentage points.

Primary area:
learning theory
Keywords:
certified robustness, block erasures, frame theory, task-aware communication, least-squares reconstruction, representation learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_U2ZLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the design of redundant block frames for transmitting a latent representation over a block-erasure channel when the downstream task is multi-class classification. The key observation is that, with erasure locations known and least-squares decoding, every full-column-rank survivor frame recovers exactly the same latent at zero noise, so the remaining design freedom can be redirected from reconstruction criteria toward reducing decision uncertainty under bounded observation noise. Based on these, the paper makes three contributions: (1) derives an exact worst-case pairwise score-uncertainty certificate and the exact per-sample all-survivor certified radius, (2) solves the rank-relaxed trace-constrained allocation in closed form, proves minimax optimality of an anisotropic allocation via polar factorization, and proves that the rank constraint makes the relaxation unattainable, (3) realizes TACBF, combining the closed-form allocation with a learned Parseval factor trained by a certificate loss. On seven datasets with five paired seeds at (m, q) = (8, 2), TACBF improves certified-accuracy AUC over a matched fixed-factor control on all datasets (all 35 paired seed differences positive, significant after Holm correction), preserves exact recovery, width, trace, and the LS decoder, and changes clean accuracy by at most 0.25 pp, at the cost of a 3.4–65.2% increase in expected LS reconstruction error.

Critical strengths
  1. The paper proposes a novel formulation built on a full certificate-to-design theory. It innovatively repurposes the design freedom from noiseless exact recovery for optimizing decision geometry, departing from prior frame design focused on reconstruction metrics. Beyond the robustness certificate, it derives closed-form relaxed optimum, minimax allocation and proves the rank constraint gap, forming a complete theoretical storyline supporting this positive evaluation.
  2. The experiments adopt rigorous methodology. It uses seed-paired baselines to isolate the learned factor, enumerates all survivor sets for certification, applies proper statistical testing, and transparently reports negative results. Such strict controls render empirical claims reliable.
  3. Empirical improvements are consistent across diverse settings with tiny deployment overhead. The method works on seven datasets of three modalities, with consistent positive gains over all seed pairs. It introduces marginal training and memory cost while preserving inference speed and clean accuracy.
Critical weaknesses
  1. A restrictive operating model and purely synthetic evaluation limit the demonstrated significance. The certificate relies on known erasure locations, deterministic ℓ2-bounded observation noise, a linear classification head, and unregularized least-squares decoding, which the authors acknowledge in Section 8. The experiments add noise in feature space instead of simulating a real communication channel. In addition, no existing baseline accounts for the exact-recovery constraint; block dropout serves as the closest comparison but does not optimize for the certificate objective. The core mathematical component of Theorem 1, the exact worst-case score perturbation of a linear classifier over an ℓ2 ball under a linear transformation, draws from prior classical work such as Hein & Andriushchenko (2017). The real novelty of this paper comes from combining this formulation with least-squares decoding, optimizing over the worst case across survivor sets, and developing the accompanying design theory. The work is technically solid. Even so, in its current form, the paper reads primarily as a strong contribution to frame theory rather than a practical robust communication solution.
  2. Certified-accuracy AUC is evaluated over author-selected parameter grids using inconsistent metrics and ranges, which deviates from standard community protocols of fixed-radius certified accuracy or full certification curves. Several observed improvements are marginal relative to baseline values, and gains on the Banking77 dataset only appear at small radii, concentrating where the baseline performs poorly. The primary experiments adopt the setting (m,q)=(8,2); tests with m=16 are limited to an ablation search, and direct training for q=3 is only conducted on one dataset with three seeds. Besides, exhaustive certification incurs combinatorial computational cost scaling with (m¦q). 3.The certified-classification gains come at a substantial, quantified reconstruction cost that weakens the “exact latent recovery” value proposition under noise. Expected squared LS reconstruction error rises 3.4–65.2% (e.g., PCam 12.59 vs. 7.63), with task-orthogonal directions de-amplified by design, so any downstream use of the latent beyond the fixed linear head inherits noisier recovery. The tradeoff is characterized exactly (Eqs. 10, 19), but no practical interpolation is studied (e.g., sweeping μ).
Questions for authors

Q1. Does the certificate extend to input-space perturbations or stochastic (e.g., Gaussian) channel noise? If not, please state this scope explicitly, since “decision uncertainty” may be read as input-adversarial robustness. Q2. What happens with unknown or imperfectly estimated erasure locations — is the all-survivor certificate still valid, or does a wrong mask break exact recovery and the radius? Q3. How much certified robustness does the exact-recovery/Parseval/floor machinery cost? A certificate-aware baseline without these constraints, under the same channel and budget, would price the main constraint.

5 comments
AuthorsOpenReview ↗

Response to Reviewer U2ZL: W1 & Q1 & Q2

Thank you for your thoughtful review and for recognizing the theoretical development and experimental controls. Your questions led us to add Gaussian tests and a comparison with freely parameterized frames. We provide detailed responses to each of your concerns below.

W1. Operating model and contribution.

A restrictive operating model and purely synthetic evaluation limit the demonstrated significance. [...] The real novelty of this paper comes from combining this formulation with least-squares decoding, optimizing over the worst case across survivor sets, and developing the accompanying design theory.

We study classification with complete zero-noise latent recovery under known block erasures and LS decoding. A camera could send latent features in identified packets for industrial inspection (e.g., classifying components as defective or intact). The receiver could then make an immediate decision and retain the recovered latent for later diagnosis. All admissible frames recover the same latent at zero noise, but amplify noise differently across directions; design protects directions relevant to the decision. We evaluate on real image, activity and intent benchmarks, with noise and erasures introduced in the latent channel.

The linear support function calculation is classical; Section 2 (p. 2) cites related certification work, including Hein and Andriushchenko (2017). The contribution is a frame design rule: the exact radius over all survivor sets identifies decision directions to protect; relaxed allocation specifies an ideal task/orthogonal energy split; the strict block-rank gap explains why finite blocks cannot realize the corresponding survivor geometry. Allocation and geometry optimization are complementary. Figure 3 (p. 10) illustrates the geometry's effect.

Q1. Input perturbations and Gaussian noise.

Does the certificate extend to input-space perturbations or stochastic (e.g., Gaussian) channel noise? If not, please state this scope explicitly, since “decision uncertainty” may be read as input-adversarial robustness.

The deterministic certificate concerns additive observation noise on surviving received blocks. For ηC∼N(0,σ2I)\eta_C\sim\mathcal N(0,\sigma^2I), GC=AC⊤ACG_C=A_C^\top A_C, v=wy−wkv=w_y-w_k and clean margin myk>0m_{yk}>0, LS gives the exact pairwise crossing probability

Pr⁡(m^yk≤0∣C)=Φ!(−mykσv⊤GC−1v).\displaystyle \Pr(\widehat m_{yk}\le0\mid C)=\Phi\\!\left(-\frac{m_{yk}}{\sigma\sqrt{v^\top G_C^{-1}v}}\right).

Here Φ\Phi is the standard normal CDF. We measure multiclass accuracy directly, accounting for correlated competitors. At matched training mean transmitted energy and -10 dB, TACBF/fixed-factor control accuracies are 93.17/92.80% (CIFAR-10), 83.47/83.44% (PCam) and 60.39/55.13% (Banking77). Banking77 gains 5.26 pp, paired 95% CI [5.01, 5.50]; CIFAR-10 gains .37 pp, and PCam is mixed across settings. Appendix G.3 (pp. 38–40), Tables 20–21 (p. 39) and Figures 10–11 (pp. 39–40) report the Gaussian tests and reconstruction cost.

For an input perturbation ξ\xi, noiseless LS returns f(x+ξ)f(x+\xi). An input certificate therefore requires a bound on the encoder's response, such as a Lipschitz bound for ff; the present channel certificate alone does not supply it. Sections 3 and 8 (pp. 3 and 10) state this scope.

Q2. Erasure-mask identities.

What happens with unknown or imperfectly estimated erasure locations — is the all-survivor certificate still valid, or does a wrong mask break exact recovery and the radius?

A wrong mask can break exact recovery and the radius because LS uses the supplied block identities. If LassumedL_{\rm assumed} is the decoder and AtrueA_{\rm true} the actual survivor operator,

s^−s=(LassumedAtrue−I)s+Lassumedη.\displaystyle \widehat s-s=(L_{\rm assumed}A_{\rm true}-I)s+L_{\rm assumed}\eta.

The first term is a zero-noise bias. A scalar encoded with coefficients 1 and 2 is recovered from either correctly identified packet; treating the second as the first returns 2s2s. The all-survivor guarantee covers every admissible set with its true block identities. For an unknown mask, the block identities must first be recovered before using the present decoder and certificate. Section 3 (p. 3), Eq. (3), gives the decomposition.

AuthorsOpenReview ↗

Response to Reviewer U2ZL: Weakness 2

W2. AUC protocol, radius-dependent gains, redundancy and certification cost.

Certified-accuracy AUC is evaluated over author-selected parameter grids using inconsistent metrics and ranges, which deviates from standard community protocols of fixed-radius certified accuracy or full certification curves. Several observed improvements are marginal relative to baseline values, and gains on the Banking77 dataset only appear at small radii, concentrating where the baseline performs poorly.

We appreciate your request for full curves and fixed-radius results; these show where the gains occur. Table 1 (p. 7) reports fixed-radius accuracy; Figures 2 and 5 (pp. 8 and 31) show full curves alongside grid and exact continuous AUC. The original fixed-radius results are:

DatasetRadiusControl (%)TACBF (%)
CIFAR-10ϵ=1.5\epsilon=1.584.9287.02
STL-10ϵ=1.5\epsilon=1.514.8147.33
EuroSAT RGBϵ=1.5\epsilon=1.580.5290.54
CIFAR-100δ=.42\delta=.4219.6051.40
PCamδ=.42\delta=.4265.2869.48
UCI HARδ=.42\delta=.4261.2669.30
Banking77δ=.42\delta=.420.000.10

For Banking77, the following points show gains where the control still certifies a substantial fraction of examples:

RadiusControl (%)TACBF (%)Gain (pp)
δ=.10\delta=.1075.8579.02+3.17
δ=.20\delta=.2046.3057.32+11.03
δ=.30\delta=.308.4623.87+15.41

The control still certifies 75.85% at δ=.10\delta=.10, and the largest gain among these points is 15.41 pp at δ=.30\delta=.30. The Banking77 curve in Figure 5 (p. 31) shows this range alongside the tail at δ=.42\delta=.42, where both methods are close to zero. Exact continuous AUC improves in all 35 paired runs; Tables 12–13 (p. 32) report the paired intervals and Holm-adjusted tests for the seven datasets:

DatasetΔAUCPaired 95% CIpHolmp_{\rm Holm}
CIFAR-10+0.0078[0.0030, 0.0126]0.0212
STL-10+0.0797[0.0623, 0.0972]0.0011
EuroSAT RGB+0.0153[0.0127, 0.0178]0.0004
CIFAR-100+0.0357[0.0274, 0.0440]0.0011
PCam+0.0562[0.0400, 0.0724]0.0019
UCI HAR+0.0190[0.0074, 0.0307]0.0212
Banking77+0.0308[0.0293, 0.0324]<10−5\lt 10^{-5}

Section 6 (pp. 6–7) defines the axes: absolute ϵ∈[.02,1.5]\epsilon\in[.02,1.5] for CIFAR-10/STL-10/EuroSAT, and δ∈[0,.8]\delta\in[0,.8] for the other datasets, where δ=ϵ/Econtrol\delta=\epsilon/\sqrt{E_{\rm control}}. Both methods use the same fixed control's training energy as reference, and the ranges were fixed using training/development data. Exact integration removes dependence on grid spacing. With ri=0r_i=0 for a clean error, normalized continuous AUC on [a,b][a,b] is

1N(b−a)∑imax⁡0,min⁡(ri,b)−a.\displaystyle \frac{1}{N(b-a)}\sum_i\max\\{0,\min(r_i,b)-a\\}.

Besides, exhaustive certification incurs combinatorial computational cost

In the main setting there are (mq)=28\binom{m}{q}=28 masks, and the worst-survivor coefficient for each class pair is cached across test examples. For larger settings, branch-and-bound supplies valid radius intervals at a search budget and is exact on completion; worst-case growth remains combinatorial. The separate (16,4)(16,4) search diagnostic takes 10.4 s for enumeration; beam width eight visits 295 nodes per pair on average, with empirical mean relative gap 4.0×10−44.0\times10^{-4}. Appendix D (pp. 28–29) and Figure 4 (p. 29) report the search results.

For configuration coverage, frozen models trained at q=2q=2 are evaluated at q=0,1,2,3q=0,1,2,3 on all seven datasets. The common-δ\delta AUC gain rises from 0.0040 at q=0q=0 to 0.0723 at q=3q=3, and the q=3q=3 gain exceeds q=0q=0 in all 35 paired runs. A separate STL-10 study with three paired seeds trains directly at q=3q=3 and gives mean grid AUC 0.3728 versus 0.1781 for the control. The m=16m=16 experiment evaluates search scalability. Appendix F.1 (p. 34) reports transfer and direct training separately, with Figure 7 showing transfer. Appendix G.4 (Table 22 and Figure 12, p. 41) reports sensitivity to training radius and survivor sampling.

AuthorsOpenReview ↗

Response to Reviewer U2ZL: W3 & Q3

W3. Reconstruction tradeoff.

The certified-classification gains come at a substantial, quantified reconstruction cost [...] no practical interpolation is studied (e.g., sweeping μ).

Keeping representation, head, BB, width and trace fixed, we swept the floor and squared allocation (λ=0\lambda=0: isotropic; λ=1\lambda=1: analytic). All nine settings meet their floors; five seeds per dataset.

Working points: AUC / full unit-sphere MSE.

(μ,λ)(\mu,\lambda)CIFAR-10PCamBanking77
(0.01,0)(0.01,0) isotropic0.308211 / 1.26070.070365 / 1.23390.202310 / 1.2059
(0.01,0.5)(0.01,0.5)0.827118 / 2.13350.537281 / 2.23370.231653 / 1.4565
(0.01,1)(0.01,1) default0.920851 / 10.25710.690974 / 12.59400.257676 / 5.9745
(0.0025,1)(0.0025,1)0.922939 / 41.00970.698943 / 50.37560.261342 / 22.5822
(0.04,1)(0.04,1)0.884185 / 2.57220.622815 / 3.14860.242456 / 1.8758

At μ=.02,λ=1\mu=.02,\lambda=1, full MSE falls 49.96%, 50.00%, 46.07% versus default; AUC losses are .00383, .01318, .00497 (dataset order above). Mean per-seed task MSE rises 13.73%, 12.02%, 3.97%, while orthogonal MSE is about halved.

Let TC,OCT_C,O_C be the task and orthogonal traces of (BC⊤BC)−1(B_C^\top B_C)^{-1}, and ρ,τ\rho,\tau the corresponding eigenvalues of D2D^2. For Gaussian noise,

E∣s^−s∣22=σ2(TC/ρ+OC/τ).\displaystyle \mathbb E\\|\widehat s-s\\|_2^2=\sigma^2(T_C/\rho+O_C/\tau).

For unit-sphere noise, divide by the observation dimension. Increasing orthogonal allocation reduces reconstruction error at the expense of task protection. Table 3 (p. 9) and Appendix G.5 (pp. 42–44) report these paths.

Q3. Joint constraint cost and W1's baseline concern.

W1 asks:

[...] block dropout serves as the closest comparison but does not optimize for the certificate objective.

Q3 asks:

How much certified robustness does the exact-recovery/Parseval/floor machinery cost? A certificate-aware baseline without these constraints, under the same channel and budget, would price the main constraint.

The original fixed-factor control retains exact zero-noise recovery, analytic allocation and the certificate objective; block dropout uses survivor cross-entropy. The new full-width refit matches the objective, frozen representation, head, width, trace and schedule, with only a global Frobenius energy constraint. It imposes no Parseval structure, prescribed allocation, survivor floor or full-rank constraint, and uses its actual pseudoinverse.

Full-width comparison: AUC / full unit-sphere MSE; effective paired n.

DatasetTACBFFull widthn
Original trace
CIFAR-100.691807 / 3.72480.010189 / 10599.30832
PCam0.677964 / 11.83920.711418 / 90.02423
Banking770.252630 / 5.65560.253261 / 1.49183
Matched mean energy
CIFAR-100.433178 / 35.02310.029746 / 30203.54562
PCam0.667403 / 1049.27020.695435 / 8761.18093
Banking770.248185 / 9.14810.229367 / 2.83903

One CIFAR-10 TACBF fit fails the floor (n=2). At original trace, PCam full width gains .03345 AUC with 7.60 times TACBF's MSE. At matched energy, Banking77 TACBF gains .01882 AUC and 7.22 pp Gaussian accuracy at -10 dB (59.60% vs. 52.38%; paired 95% CI [6.66, 7.78], n=3), with 3.22 times full width's MSE. CIFAR-10 full-width fits are severely ill-conditioned.

All nine full-width fits were full rank on all 28 masks and recovered exactly at zero noise: observed outcomes, not training guarantees.

Any full-column-rank AA admits A=BDA=BD, B⊤B=IB^\top B=I, so Parseval factorization preserves capacity. This comparison measures the joint effect of allocation, floor and training.

After energy matching, all eight TACBF fits retain full rank but fall below the original .01 floor. Appendix G.6 (pp. 45–46), Tables 27–28, reports rank, conditioning, floors and all five Gaussian SNRs.

We especially appreciate your suggestion to study the reconstruction tradeoff in practice. It helped us show how the floor and allocation control the balance between classification robustness and latent recovery. We hope these additions address your concerns and would welcome your thoughts on the new comparisons.

Reviewer_1pv8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies block-frame design with known erasure locations and bounded total ℓ2\ell_2 observation noise. With full-column-rank survivor frames, least-squares decoding recovers the full latent representation at zero noise. The authors derive the exact pairwise score uncertainty for a linear head and use it to design TACBF, combining the closed-form allocation D∗D^{\ast} with a learned Parseval factor BB. Their experiments across seven datasets compare the method with several baselines and include controls intended to separate allocation and geometry effects.

Critical strengths

The paper provides a connection between certification and encoding design. The equality characterization of the rank-relaxed optimum and the resulting strict rank-constrained gap clarify why block geometry remains relevant after directional allocation. The experiments have several useful controls. The fixed-factor control and shared-D∗D^* comparison help show what learning contributes. I also appreciate the mean-codeword-energy matching and the reporting of reconstruction error alongside certified accuracy.

Critical weaknesses

I am less clear about how much is lost by using only the scalar factor floor β\beta to choose D∗D^{\ast}. Theorem 4 proves optimality for Rβ(D)R_\beta(D), but this does not mean D∗D^{\ast} is optimal for the actual survivor Grams of a fixed BB. This is not an error in the theorem. Still, since the method has access to BB, I would like to understand how conservative this allocation is in practice.

The allocation gain also seems worth examining more directly. The paper explains that the trace budget is different from mean codeword energy, and includes mean-codeword-energy matching against the fixed-factor control. But that control already uses D∗D^{\ast}. I could not find the corresponding allocation-only comparison between DisoD_{\mathrm{iso}} and D∗D^{\ast} under matched mean codeword energy. This would help interpret the large gains from analytic task allocation.

Questions for authors

For a fixed BB, have you compared D∗D^{\ast} with an allocation optimized using the actual survivor Grams? Even a small example or experiment would help show how much the scalar-floor restriction costs.

How does the comparison between DisoD_{\mathrm{iso}} and D∗D^{\ast} change after matching mean codeword energy? Please also report whether both satisfy the required survivor-floor constraint after matching.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 1pv8: W1 & Q1

Thank you for your thoughtful review and for asking how the analytic guarantee translates into practical allocation gains. Your questions prompted direct optimization and a separate comparison at matched mean energy. Below, we first address the conservatism of the analytic allocation; our second reply examines energy matching and feasibility.

W1 & Q1. How conservative is the analytic allocation?

W1: I would like to understand how conservative this allocation is in practice.

Q1: Even a small example or experiment would help show how much the scalar-floor restriction costs.

The analytic rule provides a minimax guarantee; the direct comparison quantifies its practical conservatism. Theorem 4 minimizes a spectral bound based on a scalar description of survivor geometry. We use the actual survivor Grams and class pairs to measure the additional gains available.

Direct optimization gives the largest AUC gain on Banking77; the closed form retains lower reconstruction error on every dataset. The table makes both sides of this tradeoff explicit.

We fix the representation, classifier and block geometry BB, and optimize the symmetric positive definite (SPD) allocation matrix DD directly, with trace 128 and survivor floor .01. Each comparison uses five seeds and all 28 survivor sets. Mean transmitted energy is not matched in this comparison. Here QQ measures the largest squared noise amplification over class pairs and survivor sets.

Direct allocation: common trace and survivor floor.

DatasetAllocationQQ reduction (%)AUCFull MSE
CIFAR-10Closed form—0.92085110.2571
CIFAR-10Direct SPD20.310.92143319.7500
PCamClosed form—0.69097412.5940
PCamDirect SPD5.830.69725321.2423
Banking77Closed form—0.2576765.9745
Banking77Direct SPD58.050.30139813.6963

AUC is the exact continuous integral divided by interval length; MSE uses unit-sphere observation noise. All candidates meet the trace and floor requirements. The Banking77 candidates are feasible, with relative solver gaps 2.81–15.25%; global optimality is unproven.

How to read the uncertainty and AUC results. A reduction in worst class-pair uncertainty need not yield a proportional AUC gain: each example's radius also depends on its margins and competing classes. In our results, CIFAR-10 reduces QQ by 20.31% with .00058 AUC gain. The gains are .00628 on PCam and .04372 on Banking77.

Why we use the closed form. It provides an analytic minimax allocation that can be updated as the classifier changes during training. Theorem 4 minimizes Rβ(D)=β−1λmax⁡(U⊤D−2U)\mathcal R_\beta(D)=\beta^{-1}\lambda_{\max}(U^\top D^{-2}U) under tr⁡(D2)=P\operatorname{tr}(D^2)=P and βD2⪰μI\beta D^2\succeq\mu I, where UU is an orthonormal task basis and β\beta bounds survivor geometry. Intuitively, this protects the weakest task direction when only that scalar bound is available. The direct comparison refines allocations for a fixed classifier and BB.

The additional gain has a reconstruction cost. Direct SPD gives 1.93, 1.69 and 2.29 times the closed form's full MSE on CIFAR-10, PCam and Banking77, respectively. On Banking77 it also uses 2.21 times the mean transmitted energy despite the common trace budget. These measurements show the practical value of the analytic rule alongside the additional AUC gains available from direct refinement.

Section 7.3 (p. 9) and Appendix G.1 (Tables 17–18, pp. 36–37) present these comparisons and the distinction between the minimax guarantee and refinement using actual survivor geometry.

AuthorsOpenReview ↗

Response to Reviewer 1pv8: W2 & Q2

W2 & Q2. Does the allocation gain survive energy matching, and is the floor still satisfied?

W2: The allocation gain also seems worth examining more directly. The paper explains that the trace budget is different from mean codeword energy, and includes mean-codeword-energy matching against the fixed-factor control.

Q2: Please also report whether both satisfy the required survivor-floor constraint after matching.

Thank you for requesting the feasibility check alongside the performance comparison. At matched mean energy, PCam retains an allocation gain; CIFAR-10 and Banking77 are nearly tied. The pattern has a simple explanation: normalization cancels a uniform gain when nearly all signal energy already lies in the task subspace, while energy outside that subspace leaves room for useful reallocation.

For each of five paired seeds, we freeze the representation, classifier and fixed tight frame BB, and compare isotropic allocation with D⋆D^\star. This uses the fixed tight frame, separately from the learned TACBF geometry in our first reply. Both allocations are scaled to training mean codeword energy 1.

Normalized continuous AUC at matched mean energy.

DatasetIsotropicD⋆D^\starGain
CIFAR-100.4959690.495969≈0
PCam0.5240440.578270+0.05423
Banking770.2216290.221632≈0

PCam's paired 95% interval for the gain is [.04980, .05865]. Its rescaled task allocation falls below the original .01 floor, as detailed below. The CIFAR-10 and Banking77 residuals are about 1.74×10−101.74\times10^{-10} and 2.35×10−62.35\times10^{-6}. At the original trace, the respective gains are .647260, .537295 and .044450. The two budgets use different evaluation axes and intervals; gains compare allocations within each budget. AUC is the exact continuous integral divided by interval length.

Why the benefit remains on PCam. When nearly all signal energy is already in the task subspace, amplifying task directions also increases total transmitted energy by almost the same factor. Matching that energy largely cancels the gain. On PCam, 17.9% of signal energy lies outside the task subspace, so allocating less to those directions leaves more of the budget for classification. A 10.35% radius gain remains after normalization. Appendix G.2 derives this relation.

What happens to the survivor floor. All original-trace allocations satisfy .01. Scaling a frame by cc scales both its trace and survivor floor by c2c^2, and its radii by cc. Isotropic allocation still satisfies .01 in all five seeds on each dataset. The last column checks whether the rescaled task allocation still meets the original threshold.

DatasetIsotropic floorD⋆D^\star floorD⋆D^\star meeting .01
CIFAR-100.05213–0.060670.0008666–0.00096580/5
PCam0.05213–0.060670.0001131–0.00011790/5
Banking770.05213–0.060670.01068–0.012135/5

PCam's matched-energy gain is therefore measured below the original .01 floor; Banking77 retains that floor in all five pairs. Rank and zero-noise recovery are preserved by rescaling.

Before normalization, isotropic/D⋆D^\star mean energies are 1/11.91, 1/86.90 and 1/1.565 in dataset order. Appendix G.2 (pp. 37–38), Table 19 and Figure 9 give these results. This comparison evaluates all 28 sets at q=2q=2; original Table 2 used q=3q=3 for PCam and Banking77.

We sincerely appreciate the care and insight behind your suggestions. Addressing them has helped us clarify the theoretical arguments and strengthen the empirical evidence. We hope the new comparisons address your concerns about conservatism and energy matching. We would be happy to discuss any remaining questions.

Further findings on allocation quality, training cost and energy efficiency

Dear Reviewer 1pv8,

After our earlier replies, we continued analyzing what the direct comparisons imply for choosing an allocation during training. Direct optimization showed room for improvement on a fixed model. We wanted to understand how tightly that improvement could be bounded in the actual certificate, and what repeated optimization would cost as the classifier and frame evolve. We also wanted to see whether the gain at equal mean energy could be retained together with the original recovery floor.

We therefore revisited the numerical optimization bounds and training logs, and used the task and orthogonal energy decomposition to choose an allocation directly for the mean energy budget. We would like to share two new findings that help explain both the practical value of the analytic rule and the gains available from further refinement.

1. The analytic rule combines a small certificate gap with inexpensive updates.

With the representation, classifier and learned frame fixed as in Appendix G.1, PCam's single class pair makes every correctly classified example's radius proportional to 1/Q(D)1/\sqrt{Q(D)}. Numerical dual bounds therefore bound the attainable AUC under trace 128 and survivor floor .01.

PCam, mean over 5 seedsNormalized continuous AUC
Analytic allocation.690974
Feasible direct allocation.697253
Numerical upper bound.697296

The analytic rule retains at least 99.09% of the mean AUC allowed by this numerical upper bound. The direct candidates gain .006279 AUC; the numerical upper bound allows at most .006322. The analytic rule also retains at least 96.14% of the best feasible radius in every seed according to these bounds.

The computational advantage makes this rule useful during training. A complete PCam analytic update takes about 0.064 seconds, versus 312 seconds for direct optimization in our recorded runs: approximately 99.98% less time. The analytic timing includes computing the task subspace, scanning survivor floors and forming the allocation.

Direct optimization was run on frozen models. Applying its recorded cost to two solves per epoch gives the approximate training totals below (five-seed means):

DatasetObserved trainingAnalytic shareWith direct solvesDirect share
CIFAR-10, 200 epochs1.11 h.66%1,661 h99.93%
PCam, 35 epochs2.96 h.04%9.03 h67.2%
Banking77, 100 epochs10.8 min4.1%204 h99.92%

Banking77 uses the approximate direct solver reported in Appendix G.1.

On PCam, 70 analytic updates take 4.5 seconds; 70 direct solves at the recorded cost would take about 6.07 hours.

Reviewer_eh34OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies certified classification robustness after block erasures in linear latent coding. It derives an exact class-pair noise bound and uses it to design the coding matrix. The method allocates coding energy to the classifier subspaces and learns the remaining block geometry with a certified margin loss. Experiments on several datasets show improved certified AUC over a matched fixed-factor control, with limited changes in clean accuracy.

Critical strengths
  • The paper studies a well-defined problem. Traditional frame design focuses on reconstruction stability. The proposed objective directly considers classification robustness under observation noise.

  • The paper provides an exact robustness certificate for linear classification after LS decoding. The theoretical analysis seems to establish the optimal energy allocation under a relaxed constraint and identifies the effect of block-rank constraints.

  • The experiments cover multiple datasets from different domains. The results show improvements in certified AUC over the matched fixed-factor control. The paper also provides confidence intervals and statistical tests across multiple seeds.

Critical weaknesses
  • The theoretical allocation optimizes a spectral task bound, while the actual certificate depends on class-pair margins and uncertainty. To my understanding, these are different objectives. It remains unclear why the proposed uniform allocation within the task subspace can be the best choice. A comparison with direct class-pair allocation is missing.

  • The method mainly combines task-subspace scaling, Parseval-constrained matrix optimization, and a certified margin loss. It seems that these operations are closely related to existing optimization methods. The main contribution seems to lie in their combination for the proposed setting.

  • The evaluation mainly relies on the proposed certification metric. The experiments do not report classification accuracy under common stochastic noise models or practical communication channels. Besides, it seems tha the method also increases expected squared reconstruction error. Thus the practical value of this tradeoff remains unclear.

  • Most experiments use a fixed transmission configuration. Yet, systematic evaluations of redundancy and the survivor-floor constraint are limited. There is also little discussion of the sensitivity to the training noise radius and survivor sampling. These choices may affect the reported gains.

Questions for authors
  • Why does the method use uniform energy allocation within the task subspace? Can the authors compare the closed-form allocation with direct optimization of class-pair uncertainty under the same trace and survivor-floor constraints?

  • Can the authors provide noisy classification accuracy and reconstruction error under Gaussian noise and random block erasures? Does the certification gain remain under matched transmission power and SNR?

5 comments
AuthorsOpenReview ↗

Response to Reviewer eh34: W1 & Q1; W2

Thank you for your careful and constructive review, and for recognizing the value of the classification objective and exact certificate. Your suggestion to compare allocation rules helped us clarify what the analytic allocation optimizes and how it compares with direct optimization of class pair uncertainty. Below, we respond in detail to your comments and questions.

W1 & Q1. Why uniform task allocation, and how does direct optimization compare?

W1: The theoretical allocation optimizes a spectral task bound, while the actual certificate depends on class-pair margins and uncertainty. [...] It remains unclear why the proposed uniform allocation within the task subspace can be the best choice. A comparison with direct class-pair allocation is missing.

Q1: Why does the method use uniform energy allocation within the task subspace? Can the authors compare the closed-form allocation with direct optimization of class-pair uncertainty under the same trace and survivor-floor constraints?

Uniform gain solves Theorem 4's minimax problem. At a fixed energy budget, unequal gains leave a weaker task direction that determines the worst case. For SPD DD and orthonormal task basis UU, the bound uses only the scalar factor floor β\beta:

Rβ(D)=β−1λmax⁡(U⊤D−2U),tr⁡(D2)=P,βD2⪰μI.\displaystyle \mathcal R_\beta(D)=\beta^{-1}\lambda_{\max}(U^\top D^{-2}U),\qquad \operatorname{tr}(D^2)=P,\quad \beta D^2\succeq\mu I.

Actual survivor Grams and finite class pairs allow refinement beyond this scalar bound. We directly optimized SPD DD with the representation, head and BB fixed, under trace 128 and survivor floor 0.01. Closed form denotes D⋆D^\star; QQ reduction is relative to it.

DatasetAllocationQQ reduction (%)Normalized AUCFull MSE
CIFAR-10Closed form—0.92085110.2571
CIFAR-10Direct SPD20.310.92143319.7500
PCamClosed form—0.69097412.5940
PCamDirect SPD5.830.69725321.2423
Banking77Closed form—0.2576765.9745
Banking77Direct SPD58.050.30139813.6963

Three datasets, five seeds each. All candidates pass trace and floor checks over all 28 masks. Banking77's five candidates are feasible but not proven globally optimal, with available relative gaps of 2.81–15.25% (Appendix G.1, Tables 17–18, pp. 36–37). AUC is the exact continuous integral divided by interval length; MSE uses unit-sphere observation noise.

With vab=wa−wbv_{ab}=w_a-w_b and FC=BC⊤BCF_C=B_C^\top B_C, the direct objective is Q(D)=max⁡a<bqab(D)Q(D)=\max_{a<b}q_{ab}(D), where

qab(D)=max⁡Cvab⊤D−1FC−1D−1vab.\displaystyle q_{ab}(D)=\max_C v_{ab}^\top D^{-1}F_C^{-1}D^{-1}v_{ab}.

For correctly classified samples, ri=min⁡k≠yimik/qyik(D)r_i=\min_{k\ne y_i}m_{ik}/\sqrt{q_{y_i k}(D)}, with clean margins mikm_{ik} fixed here. Reducing worst raw uncertainty QQ need not improve these margin-dependent radii proportionally: CIFAR-10 reduces QQ by 20.31% with little AUC change.

Banking77 gains .04372 AUC. Direct allocation increases full MSE on all three datasets and uses 2.21 times the analytic rule's mean transmitted energy on Banking77. At a common trace budget, without matching mean transmitted power, Banking77 certified accuracy at δ=.42\delta=.42 rises from 0.0974% (D⋆D^\star) to 9.2338% (direct SPD). Appendix G.2 (Table 19 and Figure 9, pp. 37–38) separately compares isotropic allocation and D⋆D^\star at matched mean codeword energy. The closed form supplies a minimax reference and can be updated with the head and factor during training; the explanation after Theorem 4 (p. 5) distinguishes this objective from fixed-BB refinement.

W2. What is the contribution beyond combining familiar operations?

The method mainly combines task-subspace scaling, Parseval-constrained matrix optimization, and a certified margin loss. [...] The main contribution seems to lie in their combination for the proposed setting.

The substantive contribution is deriving frame design from exact decision uncertainty. Exact zero-noise recovery leaves freedom in noise amplification across latent directions. Reconstruction stability alone does not identify which directions control a class decision. Our certificate identifies which class differences the frame must protect and gives the exact radius over all survivor sets, with a perturbation attaining its boundary. The allocation theorem gives a closed-form minimax rule from the task subspace, with equality conditions. The strict gap for block rank r<dr<d shows why finite blocks cannot attain the relaxed ideal; TACBF learns the survivor geometry left unresolved by allocation. These results determine the objective, allocation and role of geometry learning, while preserving LS decoding and full zero-noise latent recovery.

AuthorsOpenReview ↗

Response to Reviewer eh34: W3 & Q2

W3 & Q2. Matched energy, Gaussian noise and reconstruction.

W3: The experiments do not report classification accuracy under common stochastic noise models or practical communication channels. [...] Thus the practical value of this tradeoff remains unclear.

Q2: Can the authors provide noisy classification accuracy and reconstruction error under Gaussian noise and random block erasures? Does the certification gain remain under matched transmission power and SNR?

Thank you for suggesting stochastic channel tests and asking how power matching affects the gains. Against the original fixed-factor control, certification gains remain positive on all seven datasets after matching mean codeword energy. Comparisons pair the original five seeds. Gaussian results average uniformly over all 28 known block-erasure masks, shared by both arms. Energy matching uses training mean codeword energy, not test-set power; both Gaussian arms have training mean energy one and share the noise variance at each SNR.

Matched-energy certification. Differences are TACBF minus control; grid AUC is the original unnormalized integral on each dataset's common radius axis. CIs are paired 95% intervals over seeds.

DatasetΔ grid AUC95% CI
CIFAR-10+0.0059[0.0008, 0.0110]
STL-10+0.1066[0.0838, 0.1294]
EuroSAT RGB+0.0255[0.0107, 0.0404]
CIFAR-100+0.0657[0.0527, 0.0788]
PCam+0.0412[0.0301, 0.0524]
UCI HAR+0.0175[0.0055, 0.0296]
Banking77+0.0347[0.0325, 0.0368]

Gaussian classification. Accuracy is in percent; Δ is in percentage points (pp).

DatasetSNR (dB)ControlTACBFΔ (pp)
CIFAR-10-1092.795393.1695+0.3742
CIFAR-10094.719894.7838+0.0640
CIFAR-101094.783694.8028+0.0193
CIFAR-102094.794594.8076+0.0131
CIFAR-103094.794694.8034+0.0088
PCam-1083.435283.4710+0.0358
PCam084.490084.2725-0.2175
PCam1084.621284.3662-0.2550
PCam2084.622484.3734-0.2490
PCam3084.621484.3753-0.2461
Banking77-1055.129960.3857+5.2557
Banking77089.215689.5571+0.3415
Banking771091.528291.5490+0.0208
Banking772091.796791.7830-0.0137
Banking773091.823191.8059-0.0172

These results show that protecting decision directions can improve Gaussian accuracy, most clearly on Banking77 at -10 dB. Banking77 improves in all five seeds there, with paired 95% CI [5.01, 5.50] pp; CIFAR-10's CI is [0.06, 0.69] pp. PCam is mixed: its CIs cross zero, and higher-SNR differences are close to its clean-accuracy difference.

Energy calibration preserves full rank and exact zero-noise recovery but rescales the absolute survivor floor. For the separately trained TACBF/control models, 0/5 pairs per dataset have both methods satisfying the original μ=.01\mu=.01 floor (G.3). G.2's Banking77 result of 5/5 floor-valid pairs instead compares isotropic and D⋆D^\star allocations on a fixed host.

Gaussian reconstruction. Relative MSE changes (%) versus control:

DatasetFullTaskOrthogonal
CIFAR-10+50.98-4.33+51.04
PCam+80.09-16.85+80.09
Banking77+69.49-10.85+82.38

For GC=AC⊤ACG_C=A_C^\top A_C, Gaussian LS error has covariance σ2GC−1\sigma^2G_C^{-1}. For a class difference v=wy−wkv=w_y-w_k, score variance is σ2v⊤GC−1v\sigma^2v^\top G_C^{-1}v, while full MSE is σ2tr⁡(GC−1)\sigma^2\operatorname{tr}(G_C^{-1}). Protecting decision directions can therefore improve noisy accuracy while increasing total MSE; the decomposition locates that cost in the orthogonal component.

Holding the representation, head, BB and trace budget (128) fixed, raising μ\mu from .01 to .02 reduces full unit-sphere MSE by 49.96%, 50.00% and 46.07%, at normalized AUC costs of .00383, .01318 and .00497 for CIFAR-10, PCam and Banking77, respectively. This offers an operating choice when classification is immediate and the latent is retained for later analysis.

Table 3 (p. 9) summarizes the tradeoff; Appendices G.3 (pp. 38–40) and G.5 (pp. 42–44) give Gaussian results and reconstruction paths. Evaluation on physical communication hardware is a valuable next step toward deployment.

AuthorsOpenReview ↗

Response to Reviewer eh34: Weakness 4

W4. Redundancy, the survivor floor, training radius and mask sampling.

Most experiments use a fixed transmission configuration. Yet, systematic evaluations of redundancy and the survivor-floor constraint are limited. There is also little discussion of the sensitivity to the training noise radius and survivor sampling. These choices may affect the reported gains.

Thank you for highlighting these configuration choices. We examined them in two ways: retraining TACBF at radii .01 and .04 around the default .02, and using all 28 masks instead of two sampled masks per batch. All four settings use the same first three original seeds per dataset (n=3n=3).

Training sensitivity (AUC values and changes ×100; mean ± sample SD).

DatasetDefault (100×AUC)ΔAUC, radius .01ΔAUC, radius .04ΔAUC, all 28 masks
CIFAR-1092.0166 ± 0.1224+0.1300 ± 0.3069+0.2118 ± 0.1654+0.2167 ± 0.0678
PCam68.9378 ± 0.5326-0.5355 ± 1.5181+0.8942 ± 0.9312+0.2289 ± 0.7332
Banking7725.6864 ± 0.1536-0.6893 ± 0.2274+0.8410 ± 0.2723+0.4696 ± 0.1296

The default setting leaves room for further gains: radius .04 and training with all 28 masks improve mean AUC on all three datasets.

Changes are relative to the same seed's default model. Radius .04 improves all nine pairs; training with all 28 masks improves seven of nine. These are TACBF sensitivity results, while the original paired comparison retains its common default settings. The results appear in Appendix G.4 (Table 22 and Figure 12, p. 41).

The nine allocation and survivor floor settings per dataset each use their specified floor and five seeds (Appendix G.5, Tables 23–25 and Figures 13–14, pp. 42–44). These paths quantify how the survivor floor trades classification protection against reconstruction quality for the full latent.

The original erasure study evaluates frozen models at q=0,1,2,3q=0,1,2,3 on all seven datasets; direct training at q=3q=3 is available for STL-10 with three pairs, while the m=16m=16 experiment studies survivor search. Appendix F.1 (Figure 7, p. 34) distinguishes transfer from direct training, and Appendix D.1 (Figure 4, pp. 28–29) reports search results. The main training configuration remains (m,q)=(8,2)(m,q)=(8,2).

Thank you again for your careful review and constructive suggestions. We hope these explanations and new results address your concerns about the allocation, contribution and empirical robustness. We would be happy to discuss any remaining questions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Author update: Additional experiments and manuscript revisions

We sincerely thank all reviewers and the Area Chair for the time and care devoted to our paper. We have learned a great deal from your questions and suggestions, which have helped us make our arguments clearer and strengthen the empirical evidence.

Following your feedback, we have added six groups of experiments covering direct allocation, energy matching, Gaussian channel noise, sensitivity to training settings, the tradeoff between classification robustness and latent reconstruction, and comparisons with freely parameterized frames. The revised main text includes the new tradeoff results in Table 3 (p. 9), while Appendix G.1–G.6 presents the detailed settings, results and analyses. All revisions are marked in green for ease of reference.

We hope these additions, together with our responses in the individual review threads, address your concerns. We would welcome any further questions and discussion.

AuthorsOpenReview ↗

Rebuttal and Discussion Summary

Dear AC, SACs, and PCs,

We sincerely thank you for your time and effort in managing the review process for our paper. We provide a summary of our paper and the progress of our rebuttal.


Strengths

All three reviewers recognized the breadth of our evaluation and the consistent gains over matched controls. Reviewer U2ZL described the work as “a novel formulation,” and Reviewer 1pv8 found the gains from learning the frame “convincing.”

In particular, the reviewers highlighted three strengths.

  • Frame design for robust classification. We use the freedom left by exact latent recovery at zero noise to protect predictions when received blocks are noisy.

    (Reviewers eh34 and U2ZL)

  • Theory that guides the design. We derive the exact noise tolerance of each prediction across all allowed erasure patterns. The theory gives a closed-form energy allocation and shows why block geometry matters.

    (Reviewers eh34, 1pv8, and U2ZL)

  • Broad evaluation with matched controls. We test on seven datasets spanning images, sensor data, and text, with matched controls that isolate the effect of learning the frame.

    (Reviewers eh34, 1pv8, and U2ZL)


Weaknesses and Responses

All three reviewers followed up and confirmed that the rebuttal resolved their main concerns. Reviewer 1pv8 raised their rating from 2 to 3; Reviewers U2ZL and eh34 maintained theirs. All three rate the paper 3 (Weak Acceptance).

  • Direct optimization and the analytic rule (Reviewers eh34 and 1pv8)

    We compared the allocation from Theorem 4, which minimizes a spectral bound, with allocations optimized for worst-case class-pair uncertainty on fixed models at the same trace and recovery floor. Direct optimization gave its largest gain in normalized area under the certified accuracy curve (AUC) on Banking77 (+0.0437) and increased reconstruction error on all three datasets.

    On PCam, the analytic rule reaches at least 99.09% of the numerical upper bound on mean AUC. An update takes about 0.064 s versus 312 s for a direct solve on a frozen model. Analytic updates account for 0.04% of training time.

  • Gains at equal transmission energy (Reviewer 1pv8)

    We compared the new analytic allocation with an isotropic one on the same PCam frames, matching mean transmission energy on the training set. The new rule raises normalized AUC from 0.5240 to 0.5693 and increases the certified radius by 8.63% on average. Both allocations meet the original 0.01 recovery floor for all five seeds.

    With the representation, classifier, and allocation fixed, learning the frame improves grid AUC in all 35 paired runs at identical transmission energy for each example.

  • Gaussian noise at matched energy (Reviewers eh34 and U2ZL)

    We matched mean transmission energy on the training set. TACBF retains higher mean grid AUC than the fixed-factor control on all seven datasets. At −10 dB SNR, Banking77 accuracy rises from 55.13% to 60.39%. CIFAR-10 gains are smaller, and PCam results are mixed.

  • Reconstruction tradeoff and unconstrained frames (Reviewers eh34 and U2ZL)

    Raising the recovery floor from 0.01 to 0.02 cuts mean squared reconstruction error by 46–50% across three datasets and lowers normalized AUC by 0.0038–0.0132. On PCam, a freely parameterized frame trained with the same objective and trace improves AUC by 0.0335 at the cost of 7.6 times TACBF’s reconstruction error.

  • Certificate scope and fixed-radius evaluation (Reviewer U2ZL)

    We clarified why the certificate requires known erasure locations. In the fixed-radius evaluation on Banking77, TACBF raises certified accuracy from 75.85% to 79.02% at a normalized radius of 0.10.

  • Sensitivity to training settings (Reviewer eh34)

    We varied the training noise radius and the number of erasure patterns per batch. Raising the radius from the default 0.02 to 0.04, the largest value tested, improves AUC in all nine paired runs. At the default radius, using all 28 patterns rather than sampling two per batch improves mean AUC on all three datasets.


Revision Summary

  • [Sections 7.2–7.3, pp. 9–10] We summarize the reconstruction tradeoff in Table 3. At Reviewer U2ZL’s suggestion, we note that the energy-calibrated TACBF fits in Appendix G.3 fall below the original 0.01 recovery floor.
  • [Table 15, p. 33] We separate the gains from allocation and frame learning with the representation and classifier fixed.
  • [Appendix G, pp. 36–47] We report six groups of new experiments and analyses. Table 19 and Figure 9 (pp. 38–39) present the new PCam allocation.

Initial revisions are in green and follow-up changes in light blue.


Final Remarks

All major reviewer concerns were resolved during the rebuttal, and all three reviewers now support the paper. We deeply appreciate the reviewers' and the Area Chair's efforts and the opportunity to improve our work.

Best regards,

Authors of Submission 8670

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.