acceptodds
Withdrawn from ICLR 2027

Competent Agents Reveal Exactly What a Random Walk Sees: The Complete World-Model Information in Behavior

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Which properties of a partially observed environment must a competent agent’s choices reveal? Cifuentes (2026, Section 4.2) pose this identification question for goals that name only observations and actions, and leave it open. We answer it for finite communicating models that admit a policy optimal for every such goal and every hidden start. The answer is the random-walk law, the stationary distribution of observation–action words under independent uniform actions. This law determines every history-dependent controlled prediction, and it is exactly the information guaranteed by competence. An information converse shows that no quantity outside it is determined by every competent policy. A constructive recovery obtains it from sampled actions at one supplied observation, under competence averaged over the queried goals. Both directions extend to a fixed relative loss below one half on the full finite class without a common state cap, where the converse connects equal-law models through a finite common predictive realization. A known state cap changes the answer at positive loss: three deterministic-emission states preserve the characterization, while four can force one additional bit.

Primary area:
learning theory
Keywords:
world models, identifiability, partial observability, predictive state representations, reinforcement learning theory

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jV8kOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper focuses on an important problem: what information is important for a competent decision-maker? The authors formulate the notion of a competent agent and provide a complete characterization, a random-walk analysis, and an analysis of a state cap.

Critical strengths
  1. The paper focuses on a really important question: what information of the environment is very important for an effective decision-maker?

  2. The paper studies this problem from a theoretical view. The claimed results are interesting and novel.

Critical weaknesses
  1. The most critical weakness of this paper is the poor writing. As a paper submitted to the learning theory track, it is expected to define the important quantities and provide mathematically rigorous statements of the main results. The following quantities are not mathematically defined:
  • The communication of the environment.

  • The native goal.

  • Depth and width

  • Private randomness.

  1. The papers contain too many new concepts without definition. Thus, it is very hard to check the correctness of the proof.

  2. The following results should be stated mathematically rigorously:

  • Theorem 3.6

  • Theorem 5.1

Questions for authors

Please provide the precise definition of all the important quantities and results.

Reviewer_P8kUOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies which properties of a finite, partially observed controlled Markov model can be identified from the actions of a competent agent. The main result is a characterization: a model quantity is recoverable from every competent agent if and only if it is determined by the random-walk law, the stationary distribution of observation–action sequences under uniformly random actions. The paper also gives a constructive procedure recovering J, and illustrates it on small synthetic models.

Critical strengths

S1. The paper addresses an identifiability question left open in prior work (Cifuentes, 2026): which properties of a partially observed environment are revealed by the actions of a competent goal-conditioned agent. It gives a constructive recovery procedure together with an information-theoretic converse.

Critical weaknesses

W1. The main text relies heavily on a 46-page appendix. Key definitions (the goal semantics, the monitor of Lemma B.2) and the central bridge construction appear only there, so the main results cannot be understood from the main body alone. It makes it impossible for a reviewer to even understand if the hypothesis are coherent. I believe that a human reviewer can not correctly evaluate this contribution without reading the extensively long appendix. I recommend the author to submit their contribution to an other venue.

W2. The paper’s main body, and especially Section 2, strongly lacks formal definitions. For example, the paragraph at line 107 is crucial for establishing the setup of the paper, but it relies solely on definitions from another paper (Cifuentes, 2026, an arXiv preprint). This makes the main results particularly hard to follow. Moreover, most of the work builds on this same recent preprint.

W3. For realistic agents, this competence is neither guaranteed nor verifiable. Moreover, answering these queries correctly already requires predicting the target cylinder probabilities.

Questions for authors

Q1: Could the authors give a self-contained formal definition of the setting in Section 2?

Q2: Is there any setting in which the competence assumption on the threshold queries (Equations 9 and 10) could be verified for a learned agent rather than for a planner with access to the true model?

1 comment

We strongly disagree with W3. The observation that “answering these queries correctly already requires predicting the target cylinder probabilities” misses the point of the paper. This is a theoretical paper about the information identifiable from competent behavior. Its central question is precisely which environment quantities a behavioral competence condition forces an agent’s choices to reveal. The contribution is a sharp: on the full admissible class, we establish recovery of the random-walk law and a matching converse ruling out any universally recoverable information beyond it. Extending this characterization to fixed positive loss below one half is the central technical result. W3 does not engage with that result.

Reviewer_X8j2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper answers the question of which properties of a partially observed environment must a competent agent’s choices reveal. The authors' answer is the random-walk law—the stationary distribution of observation–action words under independent uniform actions.

Critical strengths
  1. The paper presents an extensive theoretical analysis, which reflects substantial technical effort.

  2. Under some assumptions, the paper characterizes which environment properties are determined by every competent policy.

Critical weaknesses
  1. The exposition needs substantial improvement. Many key concepts lack clear, self-contained definitions, and the central research problem is not well formulated and stated. As a result, the paper is difficult to follow, and its contributions are difficult to assess.

  2. The paper contains several theoretical results, but the exposition does not clearly explain how they fit together to answer the central research question. In particular, the role of each result, its dependence on earlier results, and the motivation for introducing additional constructions are often left for the reader to reconstruct. This makes it difficult to follow the overall argument and assess the contribution, even when individual statements are formally specified.

  3. The admissibility assumption appears restrictive, leaving the theory’s practical relevance unclear. Concrete, practically motivated examples satisfying this assumption would strengthen the paper.

  4. The experiments illustrate the recovery procedure on small synthetic models but provide limited evidence of practical utility. A downstream planning or control task would help establish the value of the recovered predictions.

Questions for authors
  1. Could you provide a practically relevant example and demonstrate that it satisfies the admissibility assumption?

  2. Lemmas G.17 and G.18 give sufficient conditions for admissibility. Am I correct that these conditions cannot be satisfied when δ=0\delta=0?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.