acceptodds
Under review as a conference paper at ICLR 2027

Decision-Preserving Diagnostic Compression: When Knowing the Mismatch Is Not Enough

Abstract

**A correct diagnosis need not be a sufficient decision representation:** it may identify what changed while discarding distinctions needed to decide how to intervene. We study this gap between diagnostic recovery and intervention sufficiency by treating diagnosis as a compression of legal pre-decision information . For a fixed action set, we characterize when this compression preserves action-selectable value and decompose the headroom over the best fixed intervention into representation loss, rule defect, and delivered value. In the binary case, representation loss admits an exact mechanism: opposing action advantages cancel within diagnosis cells. This population distinction also creates an identification problem. We introduce the Counterfactual Adaptation Audit (CAA), which ties each empirical claim to its estimand, information regime, identifying contrast, and status. In a controlled adaptation study, the diagnostic target is recovered near-affinely, intervention preference changes sharply with mismatch source, and a prespecified source-matched rule achieves privileged opportunity value (95% CI ) over the best fixed intervention. Yet the estimator-derived diagnostic becomes available only after one candidate action has been executed, leaving legal pre-decision and unidentified. The resulting boundary is structural: information can be recoverable and decision-relevant without being preserved in a form, and available at a time, that can change the decision.

Primary area:
learning theory
Keywords:
decision representations, diagnostic compression, value of information, decision-time admissibility, adaptive systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yLLiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies why an accurate diagnosis of a system mismatch may still be insufficient for choosing an effective intervention. It treats diagnosis as a compression of information available before the decision and characterizes when that compression preserves attainable intervention value. The analysis decomposes improvement over the best fixed action into representation loss, decision-rule deficiency, and delivered value. For binary actions, it identifies cancellation between opposing action advantages within diagnostic categories as the mechanism of compression loss. The proposed Counterfactual Adaptation Audit links empirical claims to their information requirements, identifying contrasts, and evidential status. A controlled adaptation study demonstrates accurate diagnostic recovery and source-dependent intervention efficacy, with a source-matched rule improving on the best fixed intervention. However, the measured diagnostic becomes available only after a candidate action begins, leaving its pre-decision value unidentified. The study therefore distinguishes recoverable information, decision-relevant information, and information available in time to guide action.

Critical strengths
  • The finding of the paper is overall interesting, which addresses decomposition that separates representation failure from rule failure. Sections 2.2 to 2.3 distinguish value lost through compression from value still available but unused by a particular rule. The binary formula explains how a diagnosis can merge states that favor different interventions. This distinction is useful because poor decisions alone do not show that the diagnostic representation lacks useful information.

  • The paper is careful about what its evidence identifies. CAA separates privileged source information, post-action measurements, and admissible pre-decision information. Appendix E gives two models with identical observed logs but different required repairs. This is a persuasive illustration of why recovery accuracy and intervention outcomes cannot identify which part of a system needs to change by themselves.

Critical weaknesses
  • The experiment does not establish the compression loss that motivates the paper. No legal pre-decision pair ZZ and M=h(Z)M = h(Z) is evaluated through VZ−VMV_Z − V_M. The residual-arm scalar overlap in Section 5.4 establishes only that a single threshold cannot separate the recorded sources, as the authors acknowledge. The main positive result instead concerns privileged source information, while the recovered diagnostic is acquired after action selection. Thus the empirical study does not demonstrate a case in which diagnostic compression destroys otherwise available decision value, leaving the central theory-to-experiment connection incomplete.

  • The technical advance beyond existing decision-value results is modest. Section 6 and Appendix E explicitly connect the results to feature-gap regret, common optimal actions, and conditional sign-change criteria. Equation (6) is an additive decomposition rather than a new identification result. The main contribution is consequently the CAA synthesis and its reporting discipline. However, CAA does not supply an estimation procedure that identifies its loss terms or a repair from the partially observed record. Further clarification is recommended on the methodological advance beyond organizing established distinctions.

Questions for authors
  • In Section 4, what observations constitute ZZ, and at what exact point is the initial value chosen? Is any concrete pre-decision compression hh evaluated?

  • What update equations, parameter bounds, and fitting budgets define A1A_1 and A2A_2 in the primary decision record? Are their adaptation and evaluation episodes separate?

  • What substantive conclusion does Proposition 3 add to the prior conditional sign-change criterion discussed in Appendix E, beyond accommodating ties and almost-sure statements?

  • For Equation (17), what is the interval under resampling of the five recurring seed blocks with nominal-reference uncertainty propagated? Does the corresponding raw-utility interaction support the same conclusion?

2 comments
AuthorsOpenReview ↗

Response to Reviewer yLLi

Thank you for the detailed questions, especially about when the measurements become available and how the interventions are updated. Those details are central to making the theory–experiment comparison legitimate. We have made them explicit and added F3 as a separate post-review experiment; the original K0 results remain outside its confirmation.

Q1 — Information available at the decision. F3 pays for one common A0 transition and selects a complete program at t=0.05t=0.05 s. The legal packet ZZ contains measured gyro and encoder yaw rates, left and right wheel commands, estimated heading, measured wheel-distance difference, interaction index and time. Geometry, controller constants and remaining budget are shared public configuration. The selector does not receive injected source or severity, encoder calibration, seed, true pose, reward or future outcomes.

The fixed diagnosis MM is observation-side when g−e>0.025g-e>0.025 rad/s and dynamics-side otherwise. The frozen witness additionally uses re=e−(R/L)(wRcmd−wLcmd)r_e=e-(R/L)(w_R^{\mathrm{cmd}}-w_L^{\mathrm{cmd}}), with R/L=0.25R/L=0.25 and cutpoints −0.025, 0 and +0.025 rad/s. In ascending residual bins, its choices are (A0, replay-A1, encoder-only, encoder-only) for observation-side and (A2, A2, A2, A0) for dynamics-side (§3; Figure 1, p. 4; Appendix G).

Q2 — Updates, bounds and scoring. A1 updates its bias estimate by

b^t+1=b^t+0.08(gt−b^t−et),b^0=0.\widehat b_{t+1}=\widehat b_t+0.08(g_t-\widehat b_t-e_t),\qquad\widehat b_0=0.

Heading integrates 0.85(gt−b^t)+0.15et0.85(g_t-\widehat b_t)+0.15e_t, with the new bias estimate affecting later steps. There is no explicit hard bound on b^\widehat b or retrospective heading correction. Replay-A1 applies this ordinary update to the acquired packet once, without another sensor step.

For A2, ht=clip⁡(ψ^t,deg/45,−1,1)h_t=\operatorname{clip}(\widehat\psi_{t,\mathrm{deg}}/45,-1,1), ϕt=(1,ht)\phi_t=(1,h_t) and at=tanh⁡(θt⊤ϕt)a_t=\tanh(\theta_t^\top\phi_t). Its update is

θt+1=0.999θt+0.05×0.3ht(1−at2)ϕt.\theta_{t+1}=0.999\\{\theta_t+0.05\times0.3h_t(1-a_t^2)\phi_t\\}.

The two parameters start with normal SD 0.01 and have no hard box or norm bound; the executed noisy action is clipped to [−1,1] and total commanded yaw to ±1.5 rad/s. This is a heuristic update, not a reward-gradient learner with a convergence guarantee. F3 scores all 2,000 transitions, including the paid prefix, subsequent adaptation, resets and censored final episode. The original endpoint study instead used 2,000 adaptation steps followed by separate locked-parameter evaluation episodes (Appendix E, p. 16; Appendix C, pp. 13–14).

Q3 — Novelty and the missing empirical link. We agree that the additive decomposition and binary sign-change condition draw on established information-value and conditional-effect results; the treatment of ties is a clarification, not a new theorem. CAA organizes which comparisons support which claims, but does not estimate missing counterfactuals (Appendix F, pp. 17–18). The new evidence is an explicit lawful witness compared with all 25 diagnosis-only initial mappings over the same five programs. Its empirical class gap is +0.568464474 and the frozen one-sided lower bound is +0.319293 on the population contrast, under the specified computational protocol. As VZ≥V(πZ)V_Z\geq V(\pi_Z), this lower-bounds loss from the fixed diagnosis without estimating the optimal full-packet policy (Figure 3, p. 6; Table 1, p. 7; Appendix G).

Q4 — Original five-seed uncertainty. The normalized interaction is 2.010958, with descriptive joint-bootstrap interval [1.907529, 2.105970]. The raw-utility interaction is 18.237706, with interval [18.032027, 18.429594], and has the same sign. We enumerate all 55=3,1255^5=3,125 ordered resamples of the five recurring seed blocks, deduplicate source-labelled nominal records once, and recompute the nominal reference within each draw while keeping seed multiplicities. The 90 source/severity records are not independent units. These historical descriptive intervals are distinct from the F3 confirmation (Appendix C, pp. 13–14).

The F3 finding remains limited to this imperfect diagnosis and the declared simulator mixture. It does not establish insufficiency of a perfectly accurate diagnosis, broad mixture robustness, or hardware safety (§5; Appendix H).

Thank you for the detailed response and additional experiment. My main reservation is that the diagnosis is imperfect, with source-label-disagreeing cases supplying the net gain. This leaves the motivating claim that a correct diagnosis can still discard essential decision information empirically unestablished. The concentration of the gain at the −3% calibration level also limits evidence of robustness beyond the specified mixture.

I also retain concerns about the methodological contribution. As acknowledged in the response, the decomposition and sign-change criterion largely follow established results, while CAA organizes evidential requirements without identifying missing counterfactuals or providing a repair procedure. The revisions strengthen the empirical example, but the broader technical advance remains modest.

Reviewer_stWyOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies whether accurate diagnostic representations preserve the information needed for effective intervention selection. It introduces a decision-value decomposition that separates representation loss, rule defect, and delivered value, and proposes the Counterfactual Adaptation Audit (CAA) framework to assess empirical claims. Through controlled adaptation experiments, the paper demonstrates source-dependent intervention preferences and the decision value of privileged source information, highlighting the importance of information availability in adaptive decision-making.

Critical strengths
  1. This paper addresses a meaningful distinction between diagnostic accuracy and decision sufficiency, which is relevant to adaptive systems and decision-focused representation learning.

  2. The paper distinguishes privileged retrospective information from decision-time admissible information, and carefully identifies which empirical claims are supported by the available evidence.

Critical weaknesses

W1: Weak connection between theory and experiments. After reading this paper, if I understand it correctly, Section 2.1 defines decision-preserving representation, Section 2.3 derives the within-cell cancellation mechanism, and Figure 1 devotes substantial space to illustrating the compression-loss mechanism. However, the experimental section does not directly verify or quantify this compression loss. I think one of the biggest problems with this paper is that the experiments do not sufficiently address the central theoretical problem that the paper aims to highlight.

W2: Lack of a clear narrative. I find the writing logic of this paper a little unnatural. For example, the abstract is divided into multiple paragraphs, and Sections 2 and 3 feel more like a collection of related concepts and frameworks than a naturally developed argument. In particular, the transition from the theoretical analysis in Section 2 to the CAA framework in Section 3 is not sufficiently motivated. At least for me, I am outside this specific area, the paper is difficult to follow, and I think the overall organization and presentation would benefit from more careful revision.

Questions for authors

Could the authors provide an experiment that directly compares VZV_Z and VMV_M to demonstrate the proposed compression-loss mechanism? If this is not feasible in the current experimental setting, could the authors explain why?

2 comments
AuthorsOpenReview ↗

Response to Reviewer stWy

Thank you for pressing us on the link between theory and experiment. We agree with your W1: our original fault-source results showed that different repairs could be useful, but the measured diagnosis arrived after the original K0 choice. Those results could not establish compression loss at that decision. Rather than reinterpret them, we conducted F3 as a separate post-review experiment.

W1 — Direct evidence for the stated compression. F3 fixes a diagnosis M=h(Z)M=h(Z) available after one shared, charged observation step at t=0.05t=0.05 s. A residual-based selector πZ\pi_Z, frozen before confirmation, is compared with the complete diagnosis-only class over the same five feedback programs. The contrast is

G=V(πZ)−max⁡r:M→AV(r(M)),Labs=VZ−VM≥G.G=V(\pi_Z)-\max_{r:M\to\mathcal A}V(r(M)), \qquad L_{\mathrm{abs}}=V_Z-V_M\geq G.

This is why an optimal ZZ-based policy need not be learned: a positive lower bound for a lawful witness against every MM-only rule already implies positive loss from the specified compression. Unlike a comparison with one prescribed router, it rules out mere failure to choose the best mapping on that diagnosis (§2, p. 2).

In F3, both gyro–encoder discrepancy and encoder–command residual are measured before selection. Injected source, calibration, true pose, seed and future outcomes are unavailable to the selector. The five programs—A0, replay-A1, A2, encoder-only and joint-sync—share the same copied state, 2,000-transition budget and subsequent feedback (Figure 2, p. 5). The eight-cell witness was selected on eight development blocks, checked on eight different blocks without refitting, and evaluated in one confirmation: 24 equally weighted conditions with 64 independent pre-abort seed draws each. The five outcomes within each of the 1,536 world units are paired, not independent replicates.

The witness's sample utility is −4.876870513; the highest sample value among all 25 diagnosis-only mappings is −5.445334987. The empirical gap is +0.568464474. The frozen procedure yields a one-sided lower bound of +0.319293 on GG, with unconditional whole-program error at most 0.05 under the stated seed law and draw/collision-abort/test protocol (Figure 3, p. 6; Table 1, p. 7; Appendix G, pp. 19–21). We do not interpret the sample-best mapping as the population maximizer or the bound as the exact VZ−VMV_Z-V_M.

W2 — A clearer path through the argument. We have organized the revision around the same corridor robot: first the information distinction (§2), then the legal packet and matched comparison (§3), then development motivation and independent confirmation (§4). The figures follow that order (pp. 4–6). The longer value ledger and CAA vocabulary are retained as supporting material rather than separate headline contributions.

The result applies to this imperfect diagnosis and specified calibration mixture. Its fixed-reference gain is concentrated at the −3% calibration level, and removing that level leaves an empirical full-class gap of 0.011927 without a new certificate (§5; Appendix H). The original K0 limitation and negative K1/F2 development results have not been recast as confirmation.

Thank you for your detailed responses. While I appreciate the authors' efforts, my main concerns remain unresolved, so I will maintain my original score. I hope the authors can further address these issues to strengthen the paper in future revisions.

Reviewer_fiaUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper asks whether a diagnosis that correctly identifies what changed also keeps enough information to decide how to intervene. It treats the diagnosis as a compression M = h(Z) of the pre-decision information and splits the headroom over the best fixed action into representation loss, rule defect, and delivered value. It also proposes the Counterfactual Adaptation Audit (CAA), which links each empirical claim to the estimand and contrast needed to identify it.

Critical strengths
  • The paper provides a clean separation between representation loss and rule failure. The headroom decomposition separates value lost because the compression throws information away from value that is still in M but not used by the rule.

  • The experiments show a clear interaction between mismatch source and which intervention works, supported by paired comparisons and sensitivity checks.

Critical weaknesses
  1. The paper is very hard to follow. Even after careful reading, I found it difficult to extract the main argument. A revision built around one worked example connecting the theory and experiments would be much easier to follow. A few examples of the presentation issues include:
  • The text introduces a lot of notation for quantities that are never estimated (V_Z, V_M, L_abs, O(M), D(r; M)).
  • The paper repeats caveats about what is and isn't identified in almost every section, and introduces many specialized terms (privileged, admissible, identified, I/C/D/R coordinates) without a clear, unified example to tie them together.
  • Figures 1-3 are very dense and hard to read.
  1. The central theory is about value lost when a diagnostic compression merges states that need different actions. However, the experiments do not estimate V_Z - V_M, demonstrate positive representation loss, or empirically compare optimal policies using full versus compressed pre-decision information. Instead, the experiments establish the value of privileged source information for intervention selection.
Questions for authors

See the weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer fiaU

Thank you for reading the paper so carefully. Your suggestion to organize it around one worked example was especially useful. Revisiting the argument also made clear that the difficulty was not just presentation: the original fault-source comparison did not test information lost by a diagnosis available before choosing a repair. We have kept that limitation explicit and added a separate experiment, F3, to address it.

The revised paper follows a robot drifting along a straight corridor. At 0.05 s, after one paid observation step, the robot has both a gyro–encoder discrepancy and an encoder–command residual. The inherited binary diagnosis uses the first and omits the second. In development checking, the same observation-side label contains two residual regions with different program preferences: the mean A0-minus-replay-A1 difference is +2.07 below −0.025 rad/s and −10.89 between −0.025 and 0. These are development means, not certified optimal actions (Figure 3b, p. 6; Table 13, p. 22).

We then tested the decision consequence, rather than relying on that illustration. A frozen eight-cell selector retaining the residual bin was compared with all 25 mappings from the two diagnosis labels to the same five complete feedback programs. The programs share the initial observation, the 2,000-transition budget and subsequent feedback. In 1,536 paired world units, the empirical gap over the best sample diagnosis-only mapping is +0.568464. The frozen one-sided procedure gives a lower bound of +0.319293 on the corresponding population contrast at whole-program error level 0.05, under the stated computational sampling and abort assumptions (Figure 3, p. 6; Table 1, p. 7; Appendix G, pp. 19–21). Because a selector using the full lawful packet can reproduce this frozen selector, the bound also lower-bounds loss from this diagnosis.

The main text now carries this example from the information available at choice time to the complete-rule comparison (§§1–4). The finding is deliberately specific to an imperfect diagnosis and the stated simulator mixture; the earlier negative and inconclusive studies remain separate (§5).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.