acceptodds
Under review as a conference paper at ICLR 2027

When Sufficient Representations Fail under Deployment

Abstract

Exact sufficiency on a current support does not guarantee sufficiency for the same target after actions selected through the representation. We call the missing property deployment closure. For every K ≥ 2, two minimum-cardinality K-state representations can agree on the certified support and induce the same deployment, while only one remains sufficient afterward. Repair changes the quantifier: one target-distinct collision refutes closure, whereas interface repair must resolve every such collision under the information actually available to the decision maker. In ABIDES, identical public books hide ownership distinctions that become consequential after a public-book-selected trade; a prospective 128-pair study yields a graded evidence–risk frontier, while reanalysis without pair identity shows that repairing designated pairs need not repair the retained interface. In a controlled learned-world-model assay, recurrent state retains a payload distinction that the native transition fails to use; a transition repair fixed before testing reduces real regret to 0.0044 of a q-only baseline on 128 fresh matched pairs, with history-erasure and carrier-transplant controls. Together, the results separate current certification, reached-support sufficiency, and access to decision-relevant repair information.

Primary area:
reinforcement learning
Keywords:
representation learning, state abstraction, sufficiency, closed-loop deployment, decision making, information structure, representation evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NhfnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates whether a state representation that is sufficient for a given task remains sufficient after the system takes actions based on that representation. The authors formalize this property as deployment closure and show that initially sufficient representations can become insufficient when deployment makes previously irrelevant hidden information consequential. They further analyze the information required to repair such representations and distinguish between repairing designated state pairs and the entire deployed interface. Experiments in the ABIDES market simulator demonstrate how hidden ownership information can affect post-deployment outcomes and how additional observations reduce decision loss. A separate learned world model experiment shows that relevant historical information may already be retained in recurrent states but remain underutilized by the transition predictor. Overall, the paper examines how deployment can expose limitations in state representations and how different information-related failures can be identified and repaired.

Critical strengths

The problem studied is new. Both theoretical and experimental results are provided.

Critical weaknesses
  1. The paper should connect itself better with the literature and spend more effort on demonstrating the applicability. The connection between the theory and numerical results seems weak.

  2. The theorems are all based on standard math techniques. For instance, Theorem 1 uses a simple construction. Proposition 2 reduces to a standard set cover formulation, and Lemma 3 follows from classical conditional expectation properties. Although the paper formalizes an interesting problem, it is unclear how much new theoretical insight these results provide beyond existing work on state abstraction and deployment induced distribution changes.

  3. The paper introduces a cost aware formulation in Proposition 2, but this formulation is only weakly connected to the subsequent decision analysis.

Questions for authors
  1. Can the connection between the theory and experiments be made more explicit or stronger?

  2. Besides technical contributions, can the applicability of the proposed framework be better demonstrated, via for example, downstream decision analysis?

Reviewer_zeNeOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I was not able to fully understand this paper, so this summary reflects only my limited understanding. As far as I can tell, the paper argues that a representation that works well on the current states may stop working on the new states that the system reaches when actions are chosen using that representation. The authors call the property of still working on the new states "deployment closure", give a theoretical example, and discuss what extra information is needed to fix such a failure ("repair"). The experiments use a simulated stock market (ABIDES), a learned world model on a synthetic task, and a Cartpole study in the appendix.

Critical strengths

The paper states its limitations explicitly (Sec. 7), separates pre-specified from retrospective analyses, and provides proofs and supplementary material for reproduction. This is commendable, although it does not make up for the presentation problems below.

Critical weaknesses

The paper is very difficult to follow, and the problem and contribution are not clearly stated. After reading the abstract and introduction, I could not identify the problem, the study protocol or the main result. Specifically:

  • The abstract uses many new terms without defining them or relating them to standard terminology, e.g. "current support", "certified support", "repair changes the quantifier", "target-distinct collision", "interface repair", "graded evidence–risk frontier", "designated pairs", "q-only baseline", "history-erasure" and "carrier-transplant controls". Where these terms are explained at all, it is only much later than where they are first used: for example, "target-distinct collision" is used in the abstract and introduction but defined only in Section 3.1 (Prop. 2); "q-only baseline" and "carrier transplant" are used in the abstract but explained only in Section 5; "readout" is used in the introduction but defined only in Section 2.1; and "market cohort" is used in the introduction but described only in App. C.1. A reader therefore has to read most of the paper before the abstract and introduction can be understood.
  • Notation is used before it is introduced: Eq. (1) uses TT and π\pi, and the introduction uses S0S_0, without saying what they are. TT and π\pi are never explicitly introduced as general objects anywhere in the paper; their roles (transition and policy) have to be inferred from the statement and construction of Theorem 1.
  • Some details needed to read Section 4 appear only in the appendices: the content of the level-two book and the meaning of the rows of Table 1 are explained in App. B.1, and the "cohort" in App. C.1. Table 1 is not linked to App. B.1.
  • The introduction opens with a trading example (worlds, public order book, controller, future-response program, queue ownership) without any background for an ML audience. Section 4 likewise uses "level-two book", "dealer ownership", "displayed ask depth" and "owner-balanced prefix" without explanation in the main text, which makes Table 1 hard to interpret.
  • Figures 1, 2 and 4 (Figure 4 is in Appendix A) are never referenced in the text, and Figure 3 is referenced only once (panel 3c). The reader is not told when to look at them or what to take from them.
  • The contributions paragraph is written in the same unexplained vocabulary and mostly describes what each section contains, so it is hard to tell what is new.
  • Much of Section 2 explains what the results are not ("not frozen-readout validity", "not an unchanged prediction head", "not an impossibility under known dynamics") rather than what they are.

This matters because an ICLR reader cannot assess or use the contribution if they cannot work out what it is.

Unclear significance and limited evaluation. As far as I could understand, the experiments are in specially constructed settings (a simulated market and a synthetic task), and the paper does not show the problem in a realistic ML or robotics setting. In the world-model study, the fix appears to have been chosen after a preliminary analysis showed the failure (App. E.2). There is no experimental comparison with any of the related methods the paper cites. In addition, much of the positioning in Section 6 relies on five very recent, single-author, non-peer-reviewed preprints, all posted within about four months of the submission deadline: Sevetlidis (2026a, arXiv, Jun 2026), Sevetlidis (2026b, arXiv, May 2026), Wang (2026, arXiv, Aug 2026), Takahashi (2026, Zenodo/SSRN, Aug 2026) and Murray (2026, SSRN). I checked that these works exist, but their claims have not been peer reviewed, so they cannot establish the paper's novelty or its relation to prior work. This matters because it is unclear what an ML practitioner should take away from the paper.

Questions for authors

All of the following are clarification requests that could change my assessment.

  1. Can you state the problem and main result in two or three plain sentences, using a standard ML or robotics example (e.g. a learned encoder in model-based RL), and relate "deployment closure" to familiar concepts such as policy-induced distribution shift?
  2. In which realistic ML or robotics setting does this failure occur? Can you show it outside the constructed market and synthetic settings?
  3. How does the proposed approach compare experimentally with the related methods discussed in Section 6?
  4. In the world-model study, how was the fix chosen, and were other fixes tried?
Reviewer_AsxhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether a representation that is sufficient for a fixed target on an initial support remains sufficient after actions selected through that representation move the system to a new support. It calls this property deployment closure, allowing the decoder to change after deployment. This paper constructs two representations that agree on the initial support, have the same minimum cardinality, and induce identical deployment, yet differ in their sufficiency on the reached support. The empirical evidence consists of a controlled ABIDES market example, a randomized study of ownership-based observations, and a synthetic recurrent world-model experiment.

Critical strengths
  1. The definition of deployment closure concerns recoverability of the target on the reached support, while permitting a new decoder. This avoids conflating a loss of information with failure of an old prediction head. This conceptual organization is valuable for interpreting representation failures.
  2. This paper establishes its stated conjunction of properties in theorem 1, which makes the equal-budget point particularly clear: the required partition changes even though the number of target classes does not.
  3. The experiments include relevant controls and exposes an important evaluation issue. The paper shows why evidence that resolves every designated experimental pair can still leave substantial ambiguity when pair identity is unavailable.
Critical weaknesses
  1. Theorem 1 offers a useful equal-budget separation, but its mechanism remains unrestricted off-support non-identifiability. Most supporting results apply established partition and decision-theoretic identities. The additional consequence for representation learning or certification remains unclear.
  2. The native model and repaired readout are evaluated on different cohorts (64 versus 128 pairs), while the headline mean-regret ratio uses a baseline optimized for worst-case regret. A comparison on the same held-out cohort, using a mean-risk-optimal shared-action baseline and additional training seeds, would strengthen the empirical evidence.
  3. The proposed criterion relies on exact representation collisions, which may be uninformative for continuous learned representations: on a finite sample, injective encodings are trivially sufficient regardless of decoder learnability.
  4. A target-distinct collision does not necessarily imply different optimal actions or increased decision regret. Beyond the specific quadratic-loss setting, the paper provides no general characterization connecting closure violations to decision-relevant failures.
Questions for authors
  1. The paper defines deployment-closure failure through target-distinct representation collisions. However, such collisions do not necessarily lead to different optimal actions or increased decision regret, as also suggested by Appendix F. Under what conditions does a closure violation imply a meaningful degradation in decision quality? Can the authors establish a more general connection between deployment closure and decision regret beyond the specific quadratic-loss setting?
  2. The theoretical framework relies on exact sufficiency over known supports. How can deployment closure be meaningfully evaluated for high-dimensional continuous learned representations, where exact collisions may be rare and only finite samples are available? Is there a principled extension to approximate sufficiency or statistical certification?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.