- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
-the authors analyze how the grouping of asymmetric error constraints changes the amount of information a compressed representation must retain. -for a structured class of classification problems, it proves that the optimal frontier error for an M-message representation scales as Theta(M^(-2/m)) when severe errors are grouped into m risk groups. -paper also connects frontier preservation to downstream choice of risk preferences and provides synthetic experiments plus experiments on frozen calibrated WiCE verifier outputs.
Critical strengths
-technically substantive paper with a clear conceptual contribution. The main theorem is crisp and the setup is well motivated -experimental section is appropriate for the paper’s goals. The synthetic studies are well aligned with the theory, and the WiCE experiment is not a full real-world benchmark but does provide a meaningful bridge
Critical weaknesses
The theory is derived in a fairly specialized setting with a common default action and one competing risk-incurring action. The paper is honest about this, but the practical scope of the result is narrower than the framing may initially suggest. Also, the empirical section is mainly illustrative rather than validating a broad applied claim. That is acceptable for a theory paper, but readers looking for evidence in learned end-to-end systems may find the practical takeaways somewhat indirect.
Questions for authors
How much of the rate phenomenon do you expect to survive once there are several non-default risk-incurring actions with interacting decision boundaries?
In the WiCE experiment, how sensitive are the qualitative conclusions to the specific verifier and calibration recipe used to produce the frozen posteriors?
Thank you for your careful review and helpful suggestions. We respectfully clarify the scope of our results and summarize the additional evidence below.
W1/Q1: What survives with several risk-incurring actions and interacting decision boundaries?
Response: For a fixed encoder, finer control still cannot reduce the worst optimal penalized-risk loss from compression. Merged control ties penalty weights within groups, whereas finer control enlarges the penalty family. This ordering holds with multiple competing actions.
The sharp rate additionally depends on the decision geometry. Theorem 4.2 obtains 2/m through the common gain Δ and the m-dimensional state Bw/Δ. Beyond this structure, we expect rate separation when refinement exposes additional independent distinctions that affect optimal decisions. With multiple actions, some boundaries may be hidden by a better competing action, so identifying the jointly active distinctions is essential. This gives a concrete basis for expecting the mechanism to persist, while the general sharp exponent is not established by our current analysis.
W2: How do the results connect to practical risk control?
Response: In an evidence-verification service, an operator may tighten the allowed rates of labeling unsupported or partially supported claims as “supported.” For a chosen grouping, the verifier and codebook stay fixed; a downstream policy maps the messages to decisions under the new budgets. The practical objective is to preserve these later choices when designing the representation.
We tested this workflow using policies selected on development labels, with budget margins chosen by source-group cross-validation. On the official WiCE test set, all 12 configurations—full or M=32 outputs, two groupings and three budgets—met their empirical caps while retaining nonzero supported predictions. This provides label-based evidence that budget adaptation can work with unchanged codebooks, rather than requiring a new representation for every operating point.
Q2: How sensitive are the conclusions to the verifier and calibration?
Response: The qualitative grouping effect was consistent across the tested settings. We compared uncalibrated probabilities with temperatures T^, 0.8T^ and 1.2T^, where T^ is the fitted temperature. Codebooks were refitted for each setting, and fixed messages were decoded using their development-set mean posteriors. Split control had higher worst posterior-based regret in all 48 comparisons across the two original verifiers, four temperature settings and M∈{4,8,16,32,64,128}.
Two additional, separately calibrated verifier pipelines gave these split-to-merged regret ratios:
The consistent ordering supports a grouping effect beyond the original verifier and calibration choice. The varying ratios show that its magnitude remains dependent on the verifier pipeline.