- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
- The paper quantifies how much directly observing the context improves next-state prediction beyond only conditioning on the current state, the entire history, or a subset between the two. It formalises predictive sufficiency, which simply asks the model to preserve transition information rather than the context itself.
- The main point is that headroom (the potential improvement in next-state prediction from observing the true context under the agent’s observed trajectories) can collapse simply because the agent’s visitation reveals the true context. If headroom remains, this can either be due to finite model deficit, or because the headroom cannot be closed by history, and so requires the true context.
- Unlike prior work, which asks how to infer and adapt to hidden context gained from experience, this paper asks whether the information supplied to the inference mechanism actually contains useful information that is not already available to the predictor.
Critical strengths
- Clean conceptual framework: unlike prior work, which asks how to infer and adapt to hidden context gained from experience, this paper asks whether the information supplied to the inference mechanism actually contains useful information that is not already available to the predictor.
- Experimental design: the experimental design is good and interesting; I particularly like Table 2 which clearly shows the connection to related work.
- Implication for downstream context-inference benchmarks: simply introducing a hidden dynamics parameter does not automatically create a meaningful context-inference benchmark.
Critical weaknesses
- Presentation and Clarity: I think that both of these could be improved. While the theoretical framework is very interesting, the paper could be made much easier to follow. Firstly, I think that the introduction could be made less abstract and verbose. The theoretical framework section could be restructured so that the paragraph headings are consistent (e.g., organising the paragraphs around the hierarchy of conditioning information), rather than some introducing assumptions, others new quantities or ideas. Figure 1 is not very clear (I understood it once reading the paper, but I think that it could either be improved or removed).
- Practical utility: the theoretical framework uses Bayes-optimal risks, but the practitioner only has finite fitted probes. This means that if, for example, a history probe does not improve prediction to the same level as the oracle (e.g., as required for P2), we do not know whether the history contains no useful additional information, or whether the probe has simply failed to extract that useful information. The oracle itself is also fitted, which can cause
Questions for authors
- The abstract describes the main contribution as a practical criterion for matching contextual mechanisms to the information available to them, but in practice (please correct me if I am wrong) this criterion is implemented through differences in held-out losses of finite probes conditioned on different information sets. How should a practitioner distinguish absence of predictive information in a conditioning set from failure of the corresponding finite probe to extract that information (i.e. if how do we know if (history genuinely lacks information), or (the information is present, but the finite probe failed to extract it)?
- Since estimating headroom requires ground-truth context labels, does this mean that the criterion is only applicable in simulation? Is this only intended as a benchmark-design diagnostic?
3 comments
Thank you very much for your response. I wonder whether the authors might consider framing the paper more explicitly as a theoretical contribution, with the practical diagnostic presented as an illustration of the framework and its limitations.
Whether I increase my score depends on seeing the revisions to the manuscript regarding scope and clarity. I am concerned that, without either a convincing practical contribution or a clearly communicated theoretical framework, the paper's impact may be limited.
W1 Please could you update the manuscript so that I can see the changes to Figure 1 and the reorganised Section 3?
Q1 I do understand the value of the idea, but I still feel that the proposed solution of enlarging the history probe family does not fundamentally resolve the ambiguity raised in my original question: if the gap remains open, it is still unclear whether history lacks predictive information or whether the finite probe fails to extract it.
Q2 I understand that some components of the diagnostic can be applied without context labels, but my concern remains: fully determining whether a contextual mechanism has additional predictive information to exploit still requires access to ground-truth context. I think it would be beneficial to clarify this limitation and provide concrete examples of real-world applications in which the full criterion could be used.
Q3 Reading through the paper again, I have an additional question regarding the claimed upper bound on the residual. Writing the fitted risks as and , the observed gap satisfies , and therefore only upper-bounds the residual when , which does not appear to be guaranteed. What justifies this upper-bound interpretation? More generally, how can we distinguish differences in the predictive information available to the probes from differences arising from their respective function approximation, estimation, or optimisation errors?
We thank the reviewer for engaging with our rebuttal and for the follow-up questions. The uploaded PDF incorporates the agreed restructuring, addresses the scope and clarity points, as well as positions our work as an analytics contribution with practical illustration. On the remaining points:
Q1
We agree with the point that a larger history probe family doesn't fully solve the raised ambiguity, because decisions follow from closed gaps and an open gap is a bound, both up to , whereas enlarging the family only changes . However, we note that since , a history probe whose held-out risk separates above the memoryless probe's indicates without . The table below illustrates this in the identifiable cRSSM-D row. In revised Sec 5 we state the decision rule linking these measurements to the criterion's branches.
Q2
Indeed, full determination requires labels in general, since headroom is a functional of the mixing distribution over kernels, which unlabeled trajectories need not identify (finite-state mixtures under strong assumptions are a special case outside our setting). For instance, low headroom follows from weak kernel separation in our Resistance and from visitation-mediated identification in Wind, and Sec 4 separates those two with . Also, in the case of single-step episodes, one kernel (zero headroom) is indistinguishable from a mixing distribution over distinct kernels (headroom > 0). However, for Bayes-optimal predictors the label-free history gap still lower-bounds headroom and is tight when the episode's past determines the kernel at .
As to concrete examples, the full criterion applies in settings where world modeling is typically used and the context is available per episode. For instance, in real-life physical systems such as robots with measured payload and in simulated physical systems such as games, where is known by design. Furthermore, gaming setups have applications for non-physical systems, where the transition kernel captures the underlying game mechanics and parameters, such as level difficulty, are logged in telemetry. Other randomized simulators are applicable as well.
Q3
Sec 5 states . Our earlier replies clarify both error terms separately: the history-to-Oracle match holds up to the Oracle probe's own fitting error , and the history probe's error can only widen the gap. The bound doesn't require the assumption: with the fitted probes' population risks and , , and a closed population gap gives . The table below reports retained held-out minima, which is distinct from the validation-selected history-only gap. In the calibration rows the fitted residual exceeds the exact filter residual by in that environment's units. The equal-information pairs show how much fitting error alone contributes, but they do not pin down by itself. The calibration rows give a direct comparison of the fitted gap with the closed-form reference, which shows how much the probes attenuate it.
| Cell | Method | Headroom | Null ( permuted across episodes) | Min over window, learned state, learned state + window, recurrent probes | |
|---|---|---|---|---|---|
| Identifiable | cRSSM-D | ||||
| Hard | DALI-S | ||||
| Calibration | exact | ||||
| Calibration | fitted probes |
We thank the reviewer for the time dedicated to reviewing our paper and for recognizing our conceptual framework, along with its experimental support and downstream implications. We address the remaining raised points below:
W1 - On improving paper's presentation
We are thankful for those suggestions. In the revised manuscript, we will re-organize Sec 3 to preview the hierarchy presented in Table 2, and improve Fig 1 so its panels state the two mechanisms directly. We believe that an illustrative cartoon in Fig 1 has its value in communicating the motivating idea before diving into theory, so we will commit to improving rather than removing it.
W2 - On using practical finite probes for assessing Bayes-optimal risks
We agree with the observation that the inference from probes is one-sided, and we will make this notion more clear in the revision. In essence, a history probe that matches the oracle probe justifies that history carries the information, up to the oracle probe's own fitting error, that our calibration study characterizes. Moreover, the probes are used comparatively rather than as point estimates of Bayes risks. The quantity that bounds the residual from above with the probes already fitted is the gap between the best finite-family history probe and the oracle probe. In revision we will report it together with history recovery and the permutation null, so that a reader can see where this one-sidedness binds.
Q1 - On separating absence of the predictive information from failure of the finite probes
Our criterion is indeed implemented through held-out losses of finite probes. We'd like to highlight however that this is the direction in which finite probes are informative, because no difference shows that history carries the information, while an open gap is an upper bound on the deficit, since the history probe's error can only increase it. As such, the criterion's decisions follow from gaps being small, so they depend on the bound rather than on the exact value. While an open gap can't distinguish between the cases, a practitioner can then enlarge the history probe's family (i.e., the finite-family reference of Sec 5) to see whether the gap closes, which changes only the history probe's error. In revision, we will make the bound interpretation explicit and also exemplify it by reporting the corresponding quantities as per the W2 answer.
Q2 - On estimating headroom without ground-truth labels
We note that logged context is needed only for the oracle quantities, such as the probe headroom, residual, decodability, and the null. At the same time, history recovery, the finite-model deficit, and the drift metrics are computed from the agent's own data without labels. As such, they answer whether history carries predictive information under the trained agent's visitation, which is the part of what our criterion asks. The diagnostic is therefore applicable wherever a context label can be logged, which includes not only benchmarks, but also simulators and similar systems. At the same time, its label-free part applies everywhere. We appreciate the opportunity to clarify this distinction and we will make those aspects explicit in Sec 5 in the revision.
We thank the reviewer again for their careful reading and the opportunity to improve our paper, and remain happy to answer any follow-up questions.