- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies how a learning system can identify relations that its current representation cannot express and use those relations to modify its subsequent learning process. It introduces “causal separation” to connect unrepresented conditions affecting the system with their observable consequences. In finite settings, the paper develops partition-refinement and response-feature constructions that remove error floors imposed by a fixed representation. For a two-layer linear network, it characterizes function-restoring parameter preparations executable through the original gradient flow using internally generated targets, and analyzes how experience can guide their selection to improve subsequent adaptation. A synthetic experiment examines the roles of source correspondence, relation formation, and deployment.
Critical strengths
-
The paper usefully distinguishes limitations in information availability, representational capacity, and deployment. These are different reasons why additional experience may fail to improve a system. The formal analyses and targeted controls help make this distinction concrete.
-
The constructive reachability result provides a concrete technical contribution. Theorem 19 characterizes the attainable preparations in the two-layer linear setting and constructs paths through the original gradient update rule.
Critical weaknesses
-
The additional contribution of the unifying framework over established theory remains insufficiently clear. Several components are closely connected to existing ideas. The partition-refinement construction relates to state equivalence and model minimization in MDPs, as studied by Givan et al. [1]. Changing subsequent optimization behavior through parameter transformations also relates to the parameter-symmetry and teleportation literature, including Zhao et al. [2]. The authors acknowledge these connections, which is helpful. Nevertheless, I found it difficult to distinguish applications of established tools from the principal new theoretical advances, and to identify what additional conclusions specifically follow from the causal-separation framework. Theorem 19 appears particularly important in this respect: the paper would benefit from a more explicit comparison explaining what its reachability characterization establishes beyond prior parameter-transformation results.
-
The connection between the concrete results and recursive self-improvement remains limited and highly conditional. The central neural construction concerns a particular two-layer linear family, internally controllable training targets, and a fixed subsequent adaptation period. The recursive analysis in Appendix F.7 assumes adequate candidate coverage, accurate evaluation, effective selection, and reliable execution at each generation. It remains unclear which concrete constructions preserve these conditions through successive self-modifications, and what the recursive result adds beyond a general accumulation of per-round approximation errors. The experiment in Appendix E.7 supports the proposed mechanism, but uses a tightly specified setting with a known observation alphabet, noise rate, future query, and gradient dynamics for candidate evaluation. These constrain how strongly the paper supports its broader contribution concerning recursive self-improvement. I do not consider large-scale application benchmarks necessary for this theoretical work, but a sharper account of the recursive guarantee and its significance would be more consequential.
[1] Equivalence Notions and Model Minimization in Markov Decision Processes.
[2] Improving Convergence and Generalization Using Parameter Symmetries.
Questions for authors
See weaknesses.
2 comments
Thank you for your thoughtful response and additional clarifications. My first concern has been largely addressed. Regarding my second concern, I remain somewhat uncertain about how the required conditions are maintained across successive self-modifications.
I must also acknowledge that my expertise in the relevant theoretical areas is limited, and I have already informed the AC of this. After considering your response and the other reviewers’ assessments, I have decided to maintain my current rating for now. Thank you again for your efforts.
Thank you for your careful review. We appreciate your recognition of the information, representation and deployment distinctions, and of Theorem 19. Your comments help us clarify the learning contribution and its recursive realization.
The contribution is a common prototype of learning as the reproduction of causal separation. A judgment becomes independent of particular training circumstances, while feedback continues to change the connections supporting it. Comparing their different consequences produces new representations through the use of existing ones.
This viewpoint is analogous to renormalization. Its four sectors are causal functions: a uses a judgment; b receives consequences; c compares them; d reconstructs the distinctions used next. Let Ki be their transition laws, with retained memory included. For coarse maps qi preserving the next-interface law,
\quad\Longrightarrow\quad \bar T=\bar K_d\bar K_c\bar K_b\bar K_a.$$ Successive conditioning proves the implication. Different implementations can therefore share an effective learning cycle. Feedback failure, insufficient comparison capacity and failed renewed use remain distinct causes of similar plateaus. This is a supplementary formalization of the renormalization-like view. Givan et al. [1] supply the state-equivalence foundation. Our construction learns and rebuilds the grouping from encounters. Different consequences force distinctions; these can expose further differences in paths reaching a condition. Theorem 3 and Corollary 4 prove formation and recovery under specified sampling conditions. Zhao et al. [2] study parameter transformations and optimization benefits. We ask whether the learner can execute the change through its original gradient rule. Theorem 19 gives exactly $(QD,0)$, $Q\in SO(n)$, as the returned endpoints from $(D,0)$, with $D$ positive diagonal. The invariant $W^\top W-VV^\top=D^2$ excludes some equivalent-output points, including $(2D,0)$; the construction reaches every permitted rotation. The learning conclusion is that restoring the present prediction need not restore the old learning response. Experience can change how the same learner adapts next. Theorem 20 analyzes attainable successor selection under its information and control conditions. > W2: *The connection between the concrete results and recursive self-improvement remains limited and highly conditional. [...]* For sustained recursive learning, two requirements matter: the concrete return must remain executable, and its effective cycle must allow new distinctions to change later interaction. We now implement both in one continuous learning program. Neural parameters, source-bound records and reference directions persist. Returned observations guide gradient preparation; the actual trained output proposes a new environmental query; response comparisons validate it; execution generates the next data. The environmental operator is hidden, and later queries are generated internally. The existing neural construction supports repeated execution. With $r=Wv$, the invariant keeps $K=WW^\top+\\|v\\|^2I$ positive definite. From output $r_0$, internal error $-K^{-1}r_0$ gives $\dot r=-r_0$, returning to a state where preparation applies again. Thus this program carries the execution condition into the next round. The test completes 48 runs of 16 rounds. In 8D, response-law MSE is 0.00010, versus 0.75000 when new queries cannot change the environment. A paired same-encounter replay also measures a learning benefit from preparation. Regarding the information assumptions, the new program keeps a finite interface and the known internal update rule. It estimates variability from samples, generates its own queries, and prepares before the fresh signed target is revealed. F.7 identifies the coverage, evaluation, selection and execution errors that must remain controlled. Our finite-candidate experiment does not establish vanishing errors over the full rotation orbit. It tests a concrete continuing implementation. Progress additionally requires informative opportunities; external shutdown ends the return. These conditions explain what sustained self-improvement must preserve, rather than treating repeated execution alone as sufficient. Thank you again for encouraging this connection to a concrete recursive program. **References** [1] Givan, R., Dean, T., and Greig, M. Equivalence notions and model minimization in Markov decision processes. *Artificial Intelligence*, 147(1–2):163–223, 2003. [2] Zhao, B., Gower, R. M., Walters, R., and Yu, R. Improving Convergence and Generalization Using Parameter Symmetries. *International Conference on Learning Representations (ICLR)*, 2024.