- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper claims that a self-improving learning rule must have 5 properties, inspired by the Era of Experience paper. Namely, 1) the ability to learn over long horizons, 2) be robust to varying reward scales, 3) the ability to learn from continuing non-episodic streams of interaction, 4) adapt to changes in the learning environment, and 5) explore in sparse reward settings.
The authors study how the recurrent hidden state in Disco-103's meta-network influences the inner agent's learning properties with respect to the criteria above by using causal interventions, for example, setting the hidden state to 0.
Critical strengths
None
Critical weaknesses
The paper is very difficult to read, and each individual line is borderline incomprehensible. This makes it very difficult to flag technical points, simply because I’m unable to follow what is being said.
Below, I provide examples of such text in the paper, and also raise some major technical concerns. Hence, I recommend a Reject rating.
-
How does a study on the learning history of Disco-103 relate to recursive self-improvement? What insights does this paper provide for recursive self-improvement?
-
The paper does not sufficiently describe background knowledge (Section 2)
-
the Disco 103 architecture
-
In Eq. (1), what is u_t?
-
What are h_t and c_t in the recurrent state?
-
What constitutes “experiences, predictions” in \tau?
-
-
Experiments:
-
They miss a lot of key details, with some details scattered all across the paper
- Ex: In Section 4.2, what exactly is inherited, and what are the donors? In Section 4.3, what is Tracking?
-
It is very difficult to follow the experiments. Further, the authors do not clearly state the intended takeaway/insight.
-
-
The paper contains many terms that are not defined:
- Ex: replay turnover
-
Finally, many statements are difficult to interpret, for example:
-
[Line 26] Decoupling historical content from its maintenance reveals that the penalty of mismatched history stems from perpetual clamping; allowing imported state to evolve naturally attenuates
this burden -
[Line 203] What scalar information can the categorical value head preserve, and can similar behavior conceal different uses of that representation?
-
[Line 388] A self-discovered rule must not treat past states as permanent physical constraints
-
[Line 406] Establish capability before attributing changes in task competence to memory.
-
[Line 412] OPEN (Goldie et al., 2024) demonstrates protocol portability with condition-dependent effects: one of six endpoint live-minus-zero contrasts has an adjusted interval above zero
-
[Line 452] Separating algorithmic demands from perceptual complexity makes individual mechanisms experimentally accessible.
-
Questions for authors
I believe the paper requires substantial revision to improve clarity and make its technical contributions assessable. Can the authors clearly state the technical contributions of this paper, how they relate to recursive self-improvement, and what novel insights they produce compared to Oh et al. 2025?