- Rating
- 4(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes to use epiplexity, an information-theoric metric recently proposed by Finzi et al., and turn it into a differentiable objective that one can optimize for different applications.
Authors have 3 main contributions:
- They turn epiplexity, originally used “post-hoc” to quantify interestingness of data, into a an optimization objective for AI to maximize learnable novelty.
- They propose a differentiable and tractable way to estimate learnable novelty using a random frozen reservoir and linear readout.
- They show applications of optimizing this metrics across 3 domains: complexity generation in cellular automata systems, data compression in MNIST resulting in unsupervised classification, and exploration bonus in reinforcement learning resulting in higher task achievement.
I think epiplexity is indeed an interesting metric and that this work is original and will be of interest to the ICLR audience which is why I am supporting acceptance for this work. However, while the applications in 1D NCA, MNIST image compression and intrinsic reward bonus to PPO are interesting proof-of-concepts that authors use to support a unified view of "intelligence" as the pursuit of learnable novelty, it is unclear to me how cherry-picked these tasks are and whether such metric can be truly useful for exploration and/or open-ended learning (see next comments).
Critical strengths
Defining observer-dependent metrics of learnable novelty (where the observer has bounded compute) is an important area of research with a broad range of potential applications for open-ended learning and discovery. This paper proposes to use epiplexity as such metric, and proposes a differentiable, closed-form estimator that can be applied as an active objective to drive discovery and learning across different domains. Turning epiplexity into a practical, active discovery objective is a timely and interesting contribution that nicely complements the original work by Finzi et al.
Moreoever, the paper is well written and combines both a theoretical explanation and a suite of experiments to support author's argument that the pursuit of learnable novelty is an important quest for AI across various domains.
Critical weaknesses
I am more skeptical about claims of epiplexity as THE new metric of “intelligence”. First, the pursue of learnable novelty is not new, and is in fact a common definition and pursue of open-ended learning often casted as the pursue of interessingness/creativity=novel+learnable from an observer perspective, see e.g. [1] and learning progress ideas [2], [3]. While this epiplexity metric is a novel way to try and estimate interestingness/learnable novelty, it comes with limits in practice (here the choice of the learner model class, or even the choice of the reservoir width) and as with other metrics it remains an open question to what degree it can really drive the automated generation of continuously interesting, open-ended problems. It would be useful to add a Related Work section to better position this metric and situate this work in the light of open-ended learning. It would also be interesting to highlight the limitations of that metric, e.g. study the influence of the model class and/or reservoir width on the obtained discoveries.
Moreover, the proposed experiments are interesting proofs of concept, but none compare to established baselines/benchmarks. For instance, the use of epiplexity as RL bonus is shown on Gym tasks where simple heuristics often suffice, it would be interesting to test on more challenging tasks with sparse reward and compare with previous work such as ICM [4] or RND [5].
[1] OMNI: Open-endedness via Models of human Notions of Interestingness, Zhang et al., 2024 [2] Intrinsic Motivation Systems for Autonomous Mental Development, Oudeyer et al., 2007 [3] MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces, Gaven et al. 2025 [4] Curiosity-driven Exploration by Self-supervised Prediction, Pathak et al., 2017 [5] Exploration by Random Network Distillation, Burda et al., 2018
Questions for authors
-
The claims that optimizing NCA rules against epiplexity lead to the emergence of solitons are questionable: how do you quantify a soliton here? What kind of “interactions” are you referring to? Do the solitons persist when running the system for longer? These are important properties of solitons that are hard to claim just by looking at resulting dynamics in Figure 2. It would also be more interesting to test that into a 2D or 3D NCAs, where the emergence of solitons (localized moving patterns that persist) is more interesting.
-
Suggestion for future work: It could also be interesting to extent the ranking in elementary CA via epiplexity (Figure 1) to higher-dimensional complex systems. For instance to rank patterns in Lenia, and see whether patterns with high epiplexity overlap with solitons (following e.g. the visualization and classification of [1]). Or to rank discoveries made by an open-ended process, e.g. humans exploring the Game of Life as in [2], and see whether epiplexity of human-discovered patterns correlates with time of discovery.
[1] Visualizing the Structure of Lenia Parameter Space, Hudcova et al., 2026 [2] AssemblyCA: A Benchmark of Open-Endedness for Discrete Cellular Automata, Patarroyo, Keith Yuan, et al. 2023.
8 comments
2. Limitations of the Metric: Observer Model Class, Width, and Fixed Capacity (Weakness 1)
It comes with limits in practice (here the choice of the learner model class, or even the choice of the reservoir width) [...]. It would also be interesting to highlight the limitations of that metric, e.g. study the influence of the model class and/or reservoir width on the obtained discoveries.
The reviewer highlights a central practical constraint: because is an observer-dependent metric, the influence of the model class (inductive bias) and width (capacity) is a direct manifestation of that dependence. In practice, the observer's architecture must match the geometry of the task data. Beyond the ablations in Appendices B and J.2, we ran a cross-architecture control experiment to illustrate this boundary concretely:
First, regarding the observer's model class: in the cellular-automaton experiments, lattice dynamics are local and translation-invariant, so the main text uses a 1D convolutional observer. We replaced the convolutional observer in the ECA ranking with a random MLP observer of the same width (256, matching the channel count of the convolutional observer) that has neither locality nor translation invariance, holding all other settings fixed:
| Evaluation dimension | Convolutional observer (main-text default) | MLP observer (no locality or translation invariance) |
|---|---|---|
| Highest score relative to uniform noise | (rule 110) | All rules within |
| Rules scoring above uniform noise (out of 88) | ||
| Ranks of the four Class IV complex rules | 1, 5, 6, 19 | 3, 4, 7, 9 |
| Rank of chaotic rule 30 | 23 | 11 |
While the coarse outline of the ranking remains (Spearman correlation between the two rankings), discrimination between rules is lost; when this MLP observer is used as the objective to train the NCA, epiplexity rises only slightly above random space-time noise ( to bits, vs. bits for noise), producing chaotic-looking textures without solitons. When the architecture does not match the local nature of the system, the observer learns poorly on this data and the scores of different dynamics compress together. The hyperparameter ablations in Appendix B (Figure 6) show the continuous version of the same effect: the observer's depth and kernel size control its spatial learning capacity; reducing depth to 1 (seeing only a single cell) drops rule 110 to rank 40, whereas increasing depth and kernel size allows the observer to learn more short-range information from the chaotic rule 30, moving its rank closer to the complex rule 54.
Second, regarding the observer's width: in offline ECA evaluation where samples are abundant, rule 110 ranks first across the entire tested range of channel counts from 64 to 512 (Appendix B, Figure 6a). The constraint of width is much more pronounced in reinforcement learning: a single trajectory contains relatively few samples, and if the reservoir is too wide, the readout can fit even random or chaotic trajectories, causing the bonus to reward chaos; we therefore use a narrow reservoir of width paired with strong regularization (; Appendix J.2) so that only regular, coherent trajectories are rewarded.
Finally, regarding the fixed observer (Section 5, Discussion): freezing the observer yields the closed-form solution, but for particularly complex structure, shallow random features may be unable to learn it or may have an unsuitable inductive bias, and once the system exhausts what the observer can read linearly, the score saturates. Letting the observer co-evolve with the observed system is a natural direction for sustained open-ended learning; the core challenge is how to let the observer grow without making it so strong that it violates epiplexity's requirement of bounded compute. This lies beyond the scope of the present paper and is a key question for future work.
3. Comparisons with Established Baselines (Weakness 2)
None compare to established baselines/benchmarks. [...] it would be interesting to test on more challenging tasks with sparse reward and compare with previous work such as ICM or RND.
It is unclear to me how cherry-picked these tasks are.
Following the reviewer's suggestion, we added RND and ICM baselines, together with a total-surprise () control under the same observer, to the reinforcement-learning experiments (Table 1 in the main text; Appendix J.4):
| Environment | Task | + epiplexity | + magnitude | + RND | + ICM | + surprise | Epiplexity |
|---|---|---|---|---|---|---|---|
| Acrobot | |||||||
| MountainCarCont. | |||||||
| Hopper | |||||||
| BipedalWalker | |||||||
| HalfCheetah | |||||||
| Walker2d | |||||||
| Swimmer | |||||||
| PointMaze | |||||||
| LunarLander | |||||||
| Pendulum |
(Mean std. over 10 seeds at 600,000 steps. Bold: improves on the task-reward baseline. "+ surprise L" uses the same observer, replacing epiplexity with the total surprise maximized by novelty search; "Epiplexity" uses no task reward.)
On the standard environments, our method performs on par with RND and ICM: across the ten tasks, + epiplexity improves on the task reward alone on eight tasks, tying RND and beating ICM by one task. Its advantage is most pronounced on sparse-reward tasks: on MountainCarContinuous, it solves the task across all ten seeds (), whereas RND and ICM still have failing seeds; on the sparse navigation task PointMaze, it achieves the highest mean return (). (Testing on more challenging large-scale sparse-reward benchmarks is a valuable direction, which could not be included within the timeframe of the discussion period.) As for the reviewer's observation that "simple heuristics often suffice" on Gym tasks, the state-magnitude control (+ magnitude) in Table 1 is precisely such a heuristic: while it beats the baseline on seven tasks, it collapses severely on Hopper and LunarLander (falling far below the task reward alone); total surprise under the same observer (+ surprise L) behaves similarly, beating the baseline on seven tasks but dropping below the task reward on LunarLander. By contrast, + epiplexity never collapses on any of the ten tasks.
We thank the reviewer for supporting our paper and for offering insightful comments and literature suggestions from the perspectives of open-ended learning and complex systems. Following your suggestions, the revised manuscript adds a dedicated Related Work section (Appendix K, with pointers in the Introduction and Discussion), examines the influence of the observer's model class and width through control experiments, expands the reinforcement-learning experiments with RND and ICM baselines as well as an action-controlled noisy-television experiment (Tables 1 and 2 in the main text; Appendices J.4 and J.5), and clarifies our task-selection principles, soliton characterization, and directions for future work below.
1. Positioning in Open-Ended Learning and Related Work (Weakness 1)
We fully agree that our work belongs to this tradition in curiosity and open-ended learning. The revised manuscript adds a dedicated Related Work section (Appendix K) and incorporates this lineage into the Introduction (Section 1) and Discussion (Section 5).
As the reviewer points out, learning progress (Oudeyer et al., 2007), OMNI (Zhang et al., 2024), MAGELLAN (Gaven et al., 2025), compression progress (Schmidhuber, 2010), and our work ask the same question: what makes an experience "interesting," and what kind of novelty is worth pursuing? They also share the same core answer: only learnable novelty is worth pursuing, which naturally excludes both unlearnable noise and unchanging stasis. All of these accounts likewise introduce an observer, and they differ in what prior must be supplied to the observer in advance. Learning progress takes the learner itself as the observer, but the partition into regions of experience or tasks must still be given from outside; OMNI uses a foundation model as a proxy for a human observer's prior to judge which tasks are interesting; MAGELLAN uses the semantic space of a language model as a prior to organize the goal space. Intrinsically motivated goal exploration processes (IMGEP; Forestier et al., 2022) require no hand-made partition and can learn a goal space without supervision using a VAE (Reinke et al., 2020), but they treat diversity as the outcome of sampling in that goal space rather than assigning a scalar novelty value to an individual system, so there is no differentiable novelty objective to optimize directly (in Hamon et al., 2025, for instance, gradient descent is directed toward each sampled target point).
Our work replaces these external priors with compression (program description length): novelty is measured directly by the description length of the program a bounded learner extracts from the data, with no task partition, behavior descriptor, or pretrained model of interest entering the definition; with a random reservoir as the observer, this measure has a closed-form solution and is differentiable with respect to the system generating the data, enabling direct gradient ascent. At the same time, we inherit from epiplexity the core concept of a computationally bounded observer, from which the connection to dynamical complexity and immunity to the noisy television follow naturally.