acceptodds
Under review as a conference paper at ICLR 2027

Intelligence from Learnable Novelty

Abstract

Intelligence appears under different names in different fields: as data compression in statistics and machine learning, as universal computation in dynamical systems, and as adaptive behavior in agents. Each field has its own foundational theory, and the two most influential theories fail in mirror image: novelty search, seeking surprise, is transfixed by a noisy TV; the free-energy principle is content in a dark room. Both failures have a single cause: each objective treats as one quantity the information a learner can convert into knowledge and the information it never can. Here we show that the learnable part, which we call learnable novelty, yields the seemingly disparate projections of intelligence, and we give a fast, differentiable, closed-form estimator built on a reservoir computer. Used as a measure, without supervision, the estimator recovers decades of complexity classification, ranking the Turing-complete rule 110 highest among the elementary cellular automata. Used as an objective, its gradient carries a neural cellular automaton from simple dynamics into solitons, the traveling, colliding structures by which rule 110 computes, and organizes an image encoder around the ten digit classes of MNIST; no label ever enters training. Handed to a reinforcement-learning agent as an intrinsic reward, it improves on the task baseline in nine of ten environments and collapses in none. Complexity generation, abstraction, and exploration, ordinarily pursued in separate fields, thus emerge from ascent on one differentiable quantity, and the projections of intelligence gain a common footing.

Primary area:
learning on time series and dynamical systems
Keywords:
intelligence theory, Novelty Search, Algorithmic Information Theory, Epiplexity, Reservoir Computer, Complexity Theory

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_gf94OpenReview ↗
Rating
4(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to use epiplexity, an information-theoric metric recently proposed by Finzi et al., and turn it into a differentiable objective that one can optimize for different applications.

Authors have 3 main contributions:

  • They turn epiplexity, originally used “post-hoc” to quantify interestingness of data, into a an optimization objective for AI to maximize learnable novelty.
  • They propose a differentiable and tractable way to estimate learnable novelty using a random frozen reservoir and linear readout.
  • They show applications of optimizing this metrics across 3 domains: complexity generation in cellular automata systems, data compression in MNIST resulting in unsupervised classification, and exploration bonus in reinforcement learning resulting in higher task achievement.

I think epiplexity is indeed an interesting metric and that this work is original and will be of interest to the ICLR audience which is why I am supporting acceptance for this work. However, while the applications in 1D NCA, MNIST image compression and intrinsic reward bonus to PPO are interesting proof-of-concepts that authors use to support a unified view of "intelligence" as the pursuit of learnable novelty, it is unclear to me how cherry-picked these tasks are and whether such metric can be truly useful for exploration and/or open-ended learning (see next comments).

Critical strengths

Defining observer-dependent metrics of learnable novelty (where the observer has bounded compute) is an important area of research with a broad range of potential applications for open-ended learning and discovery. This paper proposes to use epiplexity as such metric, and proposes a differentiable, closed-form estimator that can be applied as an active objective to drive discovery and learning across different domains. Turning epiplexity into a practical, active discovery objective is a timely and interesting contribution that nicely complements the original work by Finzi et al.

Moreoever, the paper is well written and combines both a theoretical explanation and a suite of experiments to support author's argument that the pursuit of learnable novelty is an important quest for AI across various domains.

Critical weaknesses

I am more skeptical about claims of epiplexity as THE new metric of “intelligence”. First, the pursue of learnable novelty is not new, and is in fact a common definition and pursue of open-ended learning often casted as the pursue of interessingness/creativity=novel+learnable from an observer perspective, see e.g. [1] and learning progress ideas [2], [3]. While this epiplexity metric is a novel way to try and estimate interestingness/learnable novelty, it comes with limits in practice (here the choice of the learner model class, or even the choice of the reservoir width) and as with other metrics it remains an open question to what degree it can really drive the automated generation of continuously interesting, open-ended problems. It would be useful to add a Related Work section to better position this metric and situate this work in the light of open-ended learning. It would also be interesting to highlight the limitations of that metric, e.g. study the influence of the model class and/or reservoir width on the obtained discoveries.

Moreover, the proposed experiments are interesting proofs of concept, but none compare to established baselines/benchmarks. For instance, the use of epiplexity as RL bonus is shown on Gym tasks where simple heuristics often suffice, it would be interesting to test on more challenging tasks with sparse reward and compare with previous work such as ICM [4] or RND [5].

[1] OMNI: Open-endedness via Models of human Notions of Interestingness, Zhang et al., 2024 [2] Intrinsic Motivation Systems for Autonomous Mental Development, Oudeyer et al., 2007 [3] MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces, Gaven et al. 2025 [4] Curiosity-driven Exploration by Self-supervised Prediction, Pathak et al., 2017 [5] Exploration by Random Network Distillation, Burda et al., 2018

Questions for authors
  • The claims that optimizing NCA rules against epiplexity lead to the emergence of solitons are questionable: how do you quantify a soliton here? What kind of “interactions” are you referring to? Do the solitons persist when running the system for longer? These are important properties of solitons that are hard to claim just by looking at resulting dynamics in Figure 2. It would also be more interesting to test that into a 2D or 3D NCAs, where the emergence of solitons (localized moving patterns that persist) is more interesting.

  • Suggestion for future work: It could also be interesting to extent the ranking in elementary CA via epiplexity (Figure 1) to higher-dimensional complex systems. For instance to rank patterns in Lenia, and see whether patterns with high epiplexity overlap with solitons (following e.g. the visualization and classification of [1]). Or to rank discoveries made by an open-ended process, e.g. humans exploring the Game of Life as in [2], and see whether epiplexity of human-discovered patterns correlates with time of discovery.

[1] Visualizing the Structure of Lenia Parameter Space, Hudcova et al., 2026 [2] AssemblyCA: A Benchmark of Open-Endedness for Discrete Cellular Automata, Patarroyo, Keith Yuan, et al. 2023.

8 comments

We thank the reviewer for supporting our paper and for offering insightful comments and literature suggestions from the perspectives of open-ended learning and complex systems. Following your suggestions, the revised manuscript adds a dedicated Related Work section (Appendix K, with pointers in the Introduction and Discussion), examines the influence of the observer's model class and width through control experiments, expands the reinforcement-learning experiments with RND and ICM baselines as well as an action-controlled noisy-television experiment (Tables 1 and 2 in the main text; Appendices J.4 and J.5), and clarifies our task-selection principles, soliton characterization, and directions for future work below.

1. Positioning in Open-Ended Learning and Related Work (Weakness 1)

The pursue of learnable novelty is not new, and is in fact a common definition and pursue of open-ended learning [...]. It would be useful to add a Related Work section to better position this metric and situate this work in the light of open-ended learning.

We fully agree that our work belongs to this tradition in curiosity and open-ended learning. The revised manuscript adds a dedicated Related Work section (Appendix K) and incorporates this lineage into the Introduction (Section 1) and Discussion (Section 5).

As the reviewer points out, learning progress (Oudeyer et al., 2007), OMNI (Zhang et al., 2024), MAGELLAN (Gaven et al., 2025), compression progress (Schmidhuber, 2010), and our work ask the same question: what makes an experience "interesting," and what kind of novelty is worth pursuing? They also share the same core answer: only learnable novelty is worth pursuing, which naturally excludes both unlearnable noise and unchanging stasis. All of these accounts likewise introduce an observer, and they differ in what prior must be supplied to the observer in advance. Learning progress takes the learner itself as the observer, but the partition into regions of experience or tasks must still be given from outside; OMNI uses a foundation model as a proxy for a human observer's prior to judge which tasks are interesting; MAGELLAN uses the semantic space of a language model as a prior to organize the goal space. Intrinsically motivated goal exploration processes (IMGEP; Forestier et al., 2022) require no hand-made partition and can learn a goal space without supervision using a VAE (Reinke et al., 2020), but they treat diversity as the outcome of sampling in that goal space rather than assigning a scalar novelty value to an individual system, so there is no differentiable novelty objective to optimize directly (in Hamon et al., 2025, for instance, gradient descent is directed toward each sampled target point).

Our work replaces these external priors with compression (program description length): novelty is measured directly by the description length of the program a bounded learner extracts from the data, with no task partition, behavior descriptor, or pretrained model of interest entering the definition; with a random reservoir as the observer, this measure has a closed-form solution and is differentiable with respect to the system generating the data, enabling direct gradient ascent. At the same time, we inherit from epiplexity the core concept of a computationally bounded observer, from which the connection to dynamical complexity and immunity to the noisy television follow naturally.

2. Limitations of the Metric: Observer Model Class, Width, and Fixed Capacity (Weakness 1)

It comes with limits in practice (here the choice of the learner model class, or even the choice of the reservoir width) [...]. It would also be interesting to highlight the limitations of that metric, e.g. study the influence of the model class and/or reservoir width on the obtained discoveries.

The reviewer highlights a central practical constraint: because SϕS^\phi is an observer-dependent metric, the influence of the model class (inductive bias) and width (capacity) is a direct manifestation of that dependence. In practice, the observer's architecture must match the geometry of the task data. Beyond the ablations in Appendices B and J.2, we ran a cross-architecture control experiment to illustrate this boundary concretely:

First, regarding the observer's model class: in the cellular-automaton experiments, lattice dynamics are local and translation-invariant, so the main text uses a 1D convolutional observer. We replaced the convolutional observer in the ECA ranking with a random MLP observer of the same width (256, matching the channel count of the convolutional observer) that has neither locality nor translation invariance, holding all other settings fixed:

Evaluation dimensionConvolutional observer (main-text default)MLP observer (no locality or translation invariance)
Highest score relative to uniform noise9.8×9.8\times (rule 110)All rules within 1.12×1.12\times
Rules scoring above uniform noise (out of 88)39391313
Ranks of the four Class IV complex rules1, 5, 6, 193, 4, 7, 9
Rank of chaotic rule 302311

While the coarse outline of the ranking remains (Spearman correlation 0.930.93 between the two rankings), discrimination between rules is lost; when this MLP observer is used as the objective to train the NCA, epiplexity rises only slightly above random space-time noise (11561156 to 11641164 bits, vs. 1116±21116 \pm 2 bits for noise), producing chaotic-looking textures without solitons. When the architecture does not match the local nature of the system, the observer learns poorly on this data and the scores of different dynamics compress together. The hyperparameter ablations in Appendix B (Figure 6) show the continuous version of the same effect: the observer's depth and kernel size control its spatial learning capacity; reducing depth to 1 (seeing only a single cell) drops rule 110 to rank 40, whereas increasing depth and kernel size allows the observer to learn more short-range information from the chaotic rule 30, moving its rank closer to the complex rule 54.

Second, regarding the observer's width: in offline ECA evaluation where samples are abundant, rule 110 ranks first across the entire tested range of channel counts from 64 to 512 (Appendix B, Figure 6a). The constraint of width is much more pronounced in reinforcement learning: a single trajectory contains relatively few samples, and if the reservoir is too wide, the readout can fit even random or chaotic trajectories, causing the bonus to reward chaos; we therefore use a narrow reservoir of width 3232 paired with strong regularization (λ=100\lambda = 100; Appendix J.2) so that only regular, coherent trajectories are rewarded.

Finally, regarding the fixed observer (Section 5, Discussion): freezing the observer yields the closed-form solution, but for particularly complex structure, shallow random features may be unable to learn it or may have an unsuitable inductive bias, and once the system exhausts what the observer can read linearly, the score saturates. Letting the observer co-evolve with the observed system is a natural direction for sustained open-ended learning; the core challenge is how to let the observer grow without making it so strong that it violates epiplexity's requirement of bounded compute. This lies beyond the scope of the present paper and is a key question for future work.

3. Comparisons with Established Baselines (Weakness 2)

None compare to established baselines/benchmarks. [...] it would be interesting to test on more challenging tasks with sparse reward and compare with previous work such as ICM or RND.

It is unclear to me how cherry-picked these tasks are.

Following the reviewer's suggestion, we added RND and ICM baselines, together with a total-surprise (LL) control under the same observer, to the reinforcement-learning experiments (Table 1 in the main text; Appendix J.4):

EnvironmentTask+ epiplexity+ magnitude+ RND+ ICM+ surprise LLEpiplexity
Acrobot−167±166-167 \pm 166−83±2\mathbf{-83 \pm 2}−89±4\mathbf{-89 \pm 4}−83±3\mathbf{-83 \pm 3}−84±3\mathbf{-84 \pm 3}−95±9\mathbf{-95 \pm 9}−500±0-500 \pm 0
MountainCarCont.28±4328 \pm 4393±1\mathbf{93 \pm 1}93±1\mathbf{93 \pm 1}47±47\mathbf{47 \pm 47}83±28\mathbf{83 \pm 28}93±1\mathbf{93 \pm 1}−47±6-47 \pm 6
Hopper1879±3251879 \pm 3252073±248\mathbf{2073 \pm 248}516±128516 \pm 1282037±195\mathbf{2037 \pm 195}1992±170\mathbf{1992 \pm 170}1799±5901799 \pm 5901282±881282 \pm 88
BipedalWalker125±74125 \pm 74133±57\mathbf{133 \pm 57}146±51\mathbf{146 \pm 51}90±9090 \pm 90130±58\mathbf{130 \pm 58}163±58\mathbf{163 \pm 58}−62±41-62 \pm 41
HalfCheetah362±201362 \pm 201352±219352 \pm 219623±212\mathbf{623 \pm 212}478±246\mathbf{478 \pm 246}625±117\mathbf{625 \pm 117}505±243\mathbf{505 \pm 243}−563±176-563 \pm 176
Walker2d296±45296 \pm 45281±22281 \pm 22294±32294 \pm 32291±26291 \pm 26271±13271 \pm 13276±22276 \pm 22580±164\mathbf{580 \pm 164}
Swimmer181±76181 \pm 76224±92\mathbf{224 \pm 92}194±76\mathbf{194 \pm 76}213±85\mathbf{213 \pm 85}205±87\mathbf{205 \pm 87}229±96\mathbf{229 \pm 96}19±1419 \pm 14
PointMaze229±77229 \pm 77275±31\mathbf{275 \pm 31}242±79\mathbf{242 \pm 79}267±28\mathbf{267 \pm 28}207±88207 \pm 88244±44\mathbf{244 \pm 44}7±37 \pm 3
LunarLander169±74169 \pm 74176±56\mathbf{176 \pm 56}−171±35-171 \pm 35225±8\mathbf{225 \pm 8}128±29128 \pm 2967±5567 \pm 55−2014±555-2014 \pm 555
Pendulum−1044±42-1044 \pm 42−1018±23\mathbf{-1018 \pm 23}−1016±21\mathbf{-1016 \pm 21}−1041±29\mathbf{-1041 \pm 29}−1043±24\mathbf{-1043 \pm 24}−1006±23\mathbf{-1006 \pm 23}−1131±40-1131 \pm 40

(Mean ±\pm std. over 10 seeds at 600,000 steps. Bold: improves on the task-reward baseline. "+ surprise L" uses the same observer, replacing epiplexity with the total surprise LL maximized by novelty search; "Epiplexity" uses no task reward.)

On the standard environments, our method performs on par with RND and ICM: across the ten tasks, + epiplexity improves on the task reward alone on eight tasks, tying RND and beating ICM by one task. Its advantage is most pronounced on sparse-reward tasks: on MountainCarContinuous, it solves the task across all ten seeds (93±193 \pm 1), whereas RND and ICM still have failing seeds; on the sparse navigation task PointMaze, it achieves the highest mean return (275±31275 \pm 31). (Testing on more challenging large-scale sparse-reward benchmarks is a valuable direction, which could not be included within the timeframe of the discussion period.) As for the reviewer's observation that "simple heuristics often suffice" on Gym tasks, the state-magnitude control (+ magnitude) in Table 1 is precisely such a heuristic: while it beats the baseline on seven tasks, it collapses severely on Hopper and LunarLander (falling far below the task reward alone); total surprise under the same observer (+ surprise L) behaves similarly, beating the baseline on seven tasks but dropping 60%60\% below the task reward on LunarLander. By contrast, + epiplexity never collapses on any of the ten tasks.

Reviewer_NszVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates learnable novelty as a potential unifying principle underlying intelligence across dynamical systems, representation learning, and reinforcement learning. Building on the concept of epiplexity, the authors distinguish learnable structure from irreducible randomness and propose a differentiable, closed-form estimator based on reservoir computing.

The authors evaluate the proposed objective in three settings. First, it identifies complex dynamics in cellular automata and produces structured behavior when optimized. Second, maximizing the objective in an unsupervised MNIST encoder leads to representations that distinguish digit classes. Third, using learnable novelty as an intrinsic reward improves reinforcement learning performance across several environments.

The authors argue that these results support learnable novelty as a common principle connecting complexity, abstraction, and exploration.

Critical strengths
  1. Ambitious and interesting theoretical framing. The paper explores the distinction between learnable structure and irreducible randomness and investigates whether maximizing learnable novelty can explain several aspects of intelligent behavior. The attempt to connect complexity, abstraction, and exploration through a common information-theoretic objective is conceptually interesting.

  2. Computationally tractable formulation. The proposed differentiable, closed-form estimator provides a practical way to optimize a quantity related to epiplexity. This is a meaningful contribution because directly optimizing the original definition can be computationally expensive.

  3. Evaluation across multiple settings. The authors test their approach in dynamical systems, unsupervised representation learning, and reinforcement learning. The results provide encouraging preliminary evidence that the same objective can be useful across different learning problems.

Critical weaknesses
  1. Experimental evidence is limited relative to the broad theoretical claims.

The paper proposes learnable novelty as a unifying principle underlying intelligence. However, while the experiments demonstrate interesting properties of the proposed objective, they do not fully establish the broader theoretical interpretation.

For example, the emergence of soliton-like dynamics in cellular automata does not necessarily establish the emergence of universal computation. Similarly, the formation of digit-related clusters in MNIST demonstrates useful representation learning, but does not necessarily establish that the proposed objective generally leads to abstraction or intelligence.

The authors could more clearly distinguish their demonstrated empirical findings from the broader theoretical hypotheses.

  1. Limited comparisons with existing approaches.

Although the paper discusses related concepts such as prediction-error curiosity and compression progress, the experiments provide limited direct comparisons with established methods pursuing similar objectives.

In particular, the reinforcement learning experiments primarily compare against task-reward-only PPO and a state-magnitude control. Comparisons against established intrinsic motivation or curiosity-based methods would help clarify whether learnable novelty offers a substantial advantage.

Similarly, comparisons against existing unsupervised representation learning objectives would strengthen the interpretation of the MNIST results.

  1. Limited empirical validation of the claimed robustness to irreducible randomness.

A central motivation of the proposed approach is to distinguish learnable novelty from irreducible randomness, thereby avoiding the noisy-TV problem associated with prediction-error-based curiosity.

However, the experiments do not directly isolate this property through controlled environments containing unpredictable stochastic observations. Such experiments would help establish whether the proposed estimator provides the claimed robustness in practice.

Questions for authors
  1. Given the paper’s broad claim that learnable novelty unifies multiple aspects of intelligence, what evidence supports interpreting the observed results as manifestations of a common underlying principle rather than separate consequences of the chosen estimator?

  2. Have the authors considered experiments involving irreducible observation noise to directly evaluate the claimed avoidance of the noisy-TV problem?

5 comments

We thank the reviewer for the positive assessment of our theoretical framing, closed-form estimator, and cross-domain experiments, as well as for the constructive criticisms. In response, the revised manuscript clarifies the boundary between our demonstrated empirical findings and our broader theoretical hypotheses, adds RND, ICM, and total-surprise (LL) baselines to the reinforcement-learning experiments (Table 1), introduces an action-controlled noisy-television experiment to directly test immunity to irreducible randomness (Table 2; Appendix J.5), and examines the respective roles of the common principle and the observer's structural bias through cross-architecture control experiments.

1. Scope of Empirical Evidence and Boundary of Theoretical Claims (Weakness 1)

Experimental evidence is limited relative to the broad theoretical claims. [...] The authors could more clearly distinguish their demonstrated empirical findings from the broader theoretical hypotheses.

We agree with the reviewer that the empirical findings demonstrated in individual experiments should be clearly distinguished from the broader theoretical hypothesis they point toward.

First, on the two specific examples raised by the reviewer, our claims for each individual experiment are bounded. In the cellular-automaton experiment, we do not claim that observing solitons proves that the system has achieved universal computation; rather, our logic is that known universality proofs in cellular automata (such as rule 110 and the Game of Life) rely on solitons as foundational building blocks, because solitons are localized yet able to transmit and combine information as they move and collide. Showing that maximizing SϕS^\phi drives a simple rule to spontaneously develop solitons therefore demonstrates that the objective pushes dynamics into the regime needed to support complex information processing. Similarly, in the MNIST representation-learning experiment, our goal is not to compete on classification accuracy against specialized unsupervised representation-learning algorithms (which is why we do not benchmark accuracy against contrastive or self-supervised methods on MNIST), but to test a pure theoretical prediction: whether maximizing the learnable novelty of a code alone, without any class labels or domain-specific losses, forces the encoder to shed redundancy and spontaneously form clusters separated by digit class.

Second, the evidence supporting learnable novelty as a cross-domain principle lies not in any single experiment in isolation, but in all three experiments taken together. Soliton dynamics, class clusters on MNIST, and exploration in reinforcement learning belong to dynamical systems, representation learning, and agent behavior, which are ordinarily driven by unrelated, domain-specific objectives. Here, all three emerge from the same closed-form formula SϕS^\phi: there are no task-specific loss terms, the observer uses only the most standard default architecture (a basic 1D CNN when the data have spatial lattice structure, and a basic MLP otherwise), and no hyperparameters are tuned for any individual task. On MNIST, for instance, our hyperparameter sweep can raise linear-probe accuracy to 97%97\%, but the main text reports the untuned default setting (86%86\%) because there is no principled a priori reason to select that higher-scoring hyperparameter set. Any single experiment alone would indeed be insufficient to support a broad claim about intelligence; what provides empirical support for learnable novelty as a common underlying principle is that one untuned mathematical quantity fulfills the predictions derived in advance at the end of Section 2 across three very different domains.

2. Common Underlying Principle vs. Consequences of the Chosen Estimator (Question 1)

Given the paper’s broad claim that learnable novelty unifies multiple aspects of intelligence, what evidence supports interpreting the observed results as manifestations of a common underlying principle rather than separate consequences of the chosen estimator?

Building on Section 1 above, we view learnable novelty under a bounded observer as the unifying explanation for these cross-domain results, while at the same time the specific observer chosen does directly affect the empirical outcome; both aspects follow directly from epiplexity theory itself.

On the positive side, as noted in Section 1, all three experiments share the same closed-form formula and default network architectures rather than relying on domain-tailored estimators or hyperparameter tuning. Moreover, across our hyperparameter ablations (Figures 6 and 11, and Appendix J.2), the hyperparameters with the largest impact on the results are consistently the theoretical variables that govern the observer's learning capacity (such as observer depth, convolutional receptive-field size, prediction horizon τ\tau, or the regularization strength λ\lambda in reinforcement learning), in direct agreement with the theoretical definition of epiplexity.

On the negative side, the structural bias of the observer does determine what counts as learnable, as expected for an observer-relative quantity SϕS^\phi. To test this directly, we replaced the 1D convolutional observer in the ECA ranking with a random MLP observer of the same width (256, matching the channel count of the convolutional observer) that lacks spatial locality and translation invariance, holding all other settings fixed:

Evaluation dimensionConvolutional observer (main-text default)MLP observer (no locality or translation invariance)
Highest score relative to uniform noise9.8×9.8\times (rule 110)All rules within 1.12×1.12\times
Rules scoring above uniform noise (out of 88)39391313
Ranks of the four Class IV complex rules1, 5, 6, 193, 4, 7, 9
Rank of chaotic rule 302311

While the coarse outline of the ranking is preserved (Spearman correlation 0.930.93 between the two rankings), discrimination among rules is almost entirely lost. When we further train the NCA using this MLP observer as the objective, the epiplexity rises only slightly above random space-time noise (11561156 to 11641164 bits, vs. 1116±21116 \pm 2 bits for noise), producing chaotic-looking textures rather than solitons.

The reason is that cellular-automaton lattice dynamics are local and translation-invariant; when the observer's inductive bias is mismatched to the geometry of the system, it learns poorly on this data and the scores of all rules compress together. What is learnable therefore depends on the computationally bounded observer: when the observer's basic structure matches the data geometry, the same quantity gives rise to complex dynamics, class structure, and effective exploration across domains; when the observer is mismatched to the system, the results degrade accordingly. This also marks a distinction from Finzi et al. (2026): whereas they leave the role of the model class to future work, we write the observer's structural bias explicitly into the definition (SϕS^\phi).

3. Comparisons with Curiosity Baselines and Irreducible-Noise Experiments (Weaknesses 2 & 3, Question 2)

Comparisons against established intrinsic motivation or curiosity-based methods would help clarify whether learnable novelty offers a substantial advantage.

Have the authors considered experiments involving irreducible observation noise to directly evaluate the claimed avoidance of the noisy-TV problem?

Following the reviewer's suggestion, we added the established curiosity baselines RND and ICM, together with a total-surprise (LL) control under the same observer (Table 1; Appendix J.4), and designed a controlled remote-controlled noisy-television experiment introducing irreducible observation noise (Table 2; Appendix J.5). (To prevent the online estimator from overfitting noise coordinates within the finite sample count of an episode, the revised manuscript sets a fixed ridge parameter λ=100\lambda = 100 for online RL and re-runs all conditions.)

First, on the standard tasks without added noise, our method performs on par with RND and ICM while showing greater stability across tasks:

EnvironmentTask+ epiplexity+ magnitude+ RND+ ICM+ surprise LLEpiplexity
Acrobot−167±166-167 \pm 166−83±2\mathbf{-83 \pm 2}−89±4\mathbf{-89 \pm 4}−83±3\mathbf{-83 \pm 3}−84±3\mathbf{-84 \pm 3}−95±9\mathbf{-95 \pm 9}−500±0-500 \pm 0
MountainCarCont.28±4328 \pm 4393±1\mathbf{93 \pm 1}93±1\mathbf{93 \pm 1}47±47\mathbf{47 \pm 47}83±28\mathbf{83 \pm 28}93±1\mathbf{93 \pm 1}−47±6-47 \pm 6
Hopper1879±3251879 \pm 3252073±248\mathbf{2073 \pm 248}516±128516 \pm 1282037±195\mathbf{2037 \pm 195}1992±170\mathbf{1992 \pm 170}1799±5901799 \pm 5901282±881282 \pm 88
BipedalWalker125±74125 \pm 74133±57\mathbf{133 \pm 57}146±51\mathbf{146 \pm 51}90±9090 \pm 90130±58\mathbf{130 \pm 58}163±58\mathbf{163 \pm 58}−62±41-62 \pm 41
HalfCheetah362±201362 \pm 201352±219352 \pm 219623±212\mathbf{623 \pm 212}478±246\mathbf{478 \pm 246}625±117\mathbf{625 \pm 117}505±243\mathbf{505 \pm 243}−563±176-563 \pm 176
Walker2d296±45296 \pm 45281±22281 \pm 22294±32294 \pm 32291±26291 \pm 26271±13271 \pm 13276±22276 \pm 22580±164\mathbf{580 \pm 164}
Swimmer181±76181 \pm 76224±92\mathbf{224 \pm 92}194±76\mathbf{194 \pm 76}213±85\mathbf{213 \pm 85}205±87\mathbf{205 \pm 87}229±96\mathbf{229 \pm 96}19±1419 \pm 14
PointMaze229±77229 \pm 77275±31\mathbf{275 \pm 31}242±79\mathbf{242 \pm 79}267±28\mathbf{267 \pm 28}207±88207 \pm 88244±44\mathbf{244 \pm 44}7±37 \pm 3
LunarLander169±74169 \pm 74176±56\mathbf{176 \pm 56}−171±35-171 \pm 35225±8\mathbf{225 \pm 8}128±29128 \pm 2967±5567 \pm 55−2014±555-2014 \pm 555
Pendulum−1044±42-1044 \pm 42−1018±23\mathbf{-1018 \pm 23}−1016±21\mathbf{-1016 \pm 21}−1041±29\mathbf{-1041 \pm 29}−1043±24\mathbf{-1043 \pm 24}−1006±23\mathbf{-1006 \pm 23}−1131±40-1131 \pm 40

(Mean ±\pm std. over 10 seeds at 600,000 steps. Bold: improves on the task-reward baseline. "+ surprise L" uses the same observer, replacing epiplexity with the total surprise LL maximized by novelty search; "Epiplexity" uses no task reward.)

Across the ten tasks, + epiplexity beats the task reward alone on eight tasks, matching RND and outperforming ICM by one task; on the sparse-reward task MountainCarContinuous, it solves the task on all ten seeds whereas RND and ICM still have failing seeds; and on sparse navigation in PointMaze, it achieves the highest mean return. Total surprise under the same observer (+ surprise L), while competitive without added noise, falls 60%60\% below the task reward on LunarLander, whereas + epiplexity remains stable across all ten tasks.

Reviewer_LF6hOpenReview ↗
Rating
3(+2)
Confidence
2
Informativeness
2(+1)
Presentation
1
Soundness
2
Read review
Paper summary

The paper asks which information-theoretic quantity a learning system should pursue. The authors argue that novelty search maximizes total code length and is drawn to a noisy television, while the free-energy principle minimizes it and settles in a dark room. Splitting code length into a model part and a residual under minimum description length, they propose to pursue only the model part, the epiplexity of Finzi et al. (2026) relative to a bounded observer, which they call learnable novelty. Their estimator takes a frozen random reservoir with pre-activation normalization as the observer, fits a closed-form ridge readout from reservoir features to the target, and scores the spectral description length of that readout, which makes the score differentiable in whatever produced the data. They use the score as a measure on elementary cellular automata and continuous flows, as a training objective for a neural cellular automaton and an unsupervised MNIST encoder, and as an intrinsic reward for PPO on classic control, locomotion and navigation tasks, compared with the task reward alone, an epiplexity-only reward and a state-magnitude control. They conclude that the score ranks rule 110 highest without supervision, drives the cellular automaton toward traveling and colliding structures, organizes the encoder by digit class and improves on the task reward on most tasks without collapsing on any, so that complexity generation, abstraction and exploration follow from one quantity: learnable novelty.

Critical strengths

The only novelty I can find in this manuscript is using Epiplexity ( Finzi et al. (2026) ) as an intrinsic reward for reinforcement learning. However, there are no supporting experiments demonstrating that Epiplexity is an effective intrinsic reward (see below).

Critical weaknesses

1. The cellular-automaton result just restate prior work without citing it

  • Finzi et al. (2026) already measure epiplexity on elementary cellular automata and find rule 54 above rule 30 (Sec. 5.1, Fig. 3).
  • Finzi et al. (2026, Sec. 5.3.2, Fig. 6) also argue that a compute-limited observer of rule 54 must learn gliders and their collision rules, and that learning them raises its epiplexity. The manuscript explains its solitons in nearly the same terms on p.6. Structures that travel at fixed velocity are cheap for the readout, and their collisions keep producing new configurations.
  • The manuscript cites Finzi et al. as the source of epiplexity but never mentions these experiments.

2. The manuscript equates its score with epiplexity without checking it, and it uses two names for the same quantity.

  • Eq. 9 replaces the description-length minimizer with a ridge solution and fixes α=1/2\alpha = 1/2 (Sec. 3), and the exact minimizer extracts up to about 2.3 times more bits from the same data (App. D). The manuscript calls the result "precisely the epiplexity of Finzi et al. (2026)" (p.3) but never scores any system with both Eq. 9 and the estimators of Finzi et al. Whether Eq. 9 tracks epiplexity is therefore untested.
  • The manuscript defines learnable novelty as epiplexity (p.2, Eq. 4) and then uses both names for the same score. The headings of Sec. 2, Sec. 3 and App. G, the captions of Fig. 2 and Fig. 3, and Table 1 say epiplexity, while the title, abstract, Sec. 4.2 and the Discussion say learnable novelty, sometimes in one sentence ("The epiplexity-only column shows what learnable novelty alone can do", p.8). Two names for one quantity confuse the reader. If the two are the same, the manuscript should call it epiplexity throughout.

3. The RL experiments do not test the two claims that motivate the method.

  • Table 1 has no intrinsic-motivation baseline. Its only control, a state-magnitude bonus, tests whether the agent chases large states (p.8, p.24). No condition uses the same reservoir to reward total surprise instead of the model part, which is the comparison the paper's argument calls for. ICM and RND, which the manuscript calls the closest prior work (p.8), are also missing.
  • The manuscript claims that learnable novelty is a better drive than surprise and that the reward is immune to noise (p.1, p.3, p.8). Table 1 compares only the task reward, an epiplexity-only reward and the magnitude control, and no experiment tests the score on pure noise, so neither claim has experimental support.
Questions for authors

See Weaknesses

8 comments

We thank the reviewer for a careful reading of our paper and for raising specific, substantive criticisms. Following your suggestions, the revised manuscript adds a quantitative comparison with the original estimator of Finzi et al. (2026) across all 88 elementary cellular automata (ECA; Appendix C), proves that the ridge approximation is a lower bound on the exact minimum-description-length (MDL) solution (Appendix E, Proposition 1), and expands the reinforcement-learning experiments with a total-surprise (LL) control under the same observer, RND and ICM curiosity baselines, an action-controlled noisy-television experiment, and direct tests on pure noise (Tables 1 and 2 in the main text; Appendices J.4 and J.5).

1. Core Contribution and Positioning

The only novelty I can find in this manuscript is using Epiplexity (Finzi et al. (2026)) as an intrinsic reward for reinforcement learning.

We would like to clarify that the intrinsic reward in reinforcement learning is only one of three experimental settings in our paper. Rather than merely porting an existing metric into reinforcement learning, our work addresses a broader theoretical and methodological question: what should a learning system pursue when no external task objective is given in advance?

Existing theories each capture one facet of intelligence, yet each faces structural difficulties when turned directly into an optimization objective. Novelty search pursues previously unseen behavior (maximizing cumulative surprise LL), but because it does not separate learnable regularity from irreducible randomness, it is drawn to a noisy television. The free-energy principle (active inference) minimizes surprise; without additional task priors, the safest refuge for the system is an unchanging dark room. The "compression as intelligence" tradition identifies better compression with deeper understanding, and compression progress, learning progress, and recent open-ended learning methods such as OMNI and MAGELLAN go further by requiring that novelty be learnable. Yet these methods either measure a rate of improvement (requiring derivatives along a learning curve) or still rely on priors supplied from outside the learner (hand-partitioned task spaces, foundation models acting as proxies for human interest, or language spaces).

Our paper offers a new theoretical and operational framework within this lineage. We decompose cumulative surprise LL into the description length of the program a bounded observer can extract (epiplexity) and the incompressible random residual, and propose that an intelligent system should maximize the first part alone (which we call learnable novelty when treated as an optimization objective). This avoids both the noisy television and the dark room in principle, without requiring external task partitions or human-interest priors. Crucially, the original estimation procedure of Finzi et al. (2026) trains neural networks from scratch for every system scored, which is computationally expensive and non-differentiable, restricting epiplexity to post-hoc measurement of given data. By introducing a fixed random reservoir with a linear ridge readout, we obtain a closed-form estimator (Eq. 9) that is deterministic, inexpensive, and differentiable with respect to the system generating the data. This differentiability turns epiplexity for the first time from a passive measurement into an objective that can be maximized directly by gradient ascent: beyond providing a noise-immune exploration bonus in reinforcement learning, its gradient drives a neural cellular automaton (NCA) from simple dynamics into traveling, colliding solitons, and organizes an MNIST encoder into digit-class clusters without labels. To our knowledge, this is the first work to link the emergence of complex dynamics, unsupervised representation learning, and active exploration in reinforcement learning under a single, simple closed-form objective.

2. Cellular-Automaton Experiments and Relation to Finzi et al. (2026) (Weakness 1)

The cellular-automaton result just restate prior work without citing it.

We thank the reviewer for pointing out that Section 4.1 of the original manuscript did not adequately describe the specific cellular-automaton experiments of Finzi et al. (2026). Finzi et al. indeed measured epiplexity on elementary cellular automata (showing rule 54 above rule 30 and rule 15 in their Section 5.1, Figure 3) and analyzed gliders and their collision rules in rule 54 from the perspective of a bounded observer (their Section 5.3.2, Figure 6). Whereas the original submission cited their work only in general terms, Section 4.1 of the revised manuscript now explicitly cites their ECA measurements and states the relationship between the two works in the NCA discussion.

At the same time, we wish to clarify that our two cellular-automaton experiments serve roles distinct from theirs. First, the purpose of our ECA ranking experiment (Figure 1) is not to claim a new empirical finding about cellular automata, but to validate our closed-form estimator: does a closed-form estimator that trains no neural networks recover the complexity ordering predicted by epiplexity theory? Finzi et al. provide mainly qualitative analyses on cellular automata and report measurements on very few rules (rules 15, 30, and 54 in Figure 3 of their main text, plus 10 rules in their appendix comparing two coding schemes without per-rule numerical values). On the rules they measured, our closed-form estimator agrees qualitatively with theirs, placing the complex rule 54 (34.234.2 bits) well above the chaotic rule 30 (14.614.6 bits) and the periodic rule 15 (8.28.2 bits); across the full space of 88 independent rules, our estimator further ranks the Turing-complete rule 110 first (63.363.3 bits) and scores rules with constant attractor dynamics at exactly 00 bits, matching theoretical expectations.

Second, our NCA soliton experiment (Figure 2) solves the inverse problem that Finzi et al. did not address. Finzi et al. discuss gliders in rule 54 from a measurement perspective: given a rule known to support gliders, they explain why a bounded observer extracts high epiplexity from it. We proceed in the inverse, generative direction: using the differentiability of our closed-form estimator, we train a randomly initialized continuous neural cellular automaton directly by gradient ascent on SϕS^\phi, showing that maximizing this objective drives simple initial dynamics to spontaneously develop localized structures that travel and collide. This inverse design requires a differentiable estimator and cannot be carried out with the original training-curve method.

3. Estimator Comparison, the Lower-Bound Property of the Ridge Approximation, and Naming Convention (Weakness 2)

The manuscript equates its score with epiplexity without checking it, and it uses two names for the same quantity.

The reviewer raises three key questions here: whether Eq. (9) tracks the estimator of Finzi et al., why exact minimization extracts up to about 2.3 times more bits than the ridge solution, and why both epiplexity and learnable novelty are used.

Regarding the direct comparison between Eq. (9) and the original estimator, we agree with the reviewer that calling our score "precisely the epiplexity of Finzi et al. (2026)" on p. 3 of the original manuscript was imprecise, and we have revised this phrasing to "the epiplexity relative to the observer ϕ\phi." To compare the two estimators rule by rule across the full space of 88 ECA rules, we ran the original prequential Transformer code of Finzi et al. unmodified under their cross-rule comparison protocol (their Appendix C.7) on all 88 rules, and added the results as Appendix C (Figure 7):

Evaluation dimensionEpiplexity theory predictionOriginal estimator (Finzi et al. code)Closed-form estimator (ours, SϕS^\phi)
Rules with constant dynamics (10 rules)Nothing learnable; should be 0≈2.9×106\approx 2.9\times 10^6 bits each (above chaotic rules 30, 45 & complex 106)Exactly 0
Class IV complex rules (110, 54, 41, 106)Highest rankedRanks 7, 8, 66, 88Ranks 1, 5, 6, 19
Class III chaotic rulesBelow complex rulesRules 122, 126, 146, 18 rank 1st to 4thRanks 7 to 23
Rule 54 vs. Rule 30Rule 54 far above Rule 301.4×1071.4\times 10^7 vs. 423423 bits34.234.2 vs. 14.614.6 bits

Across all 88 rules, the Spearman rank correlation between the two estimators is 0.290.29. As the table shows, this discrepancy stems from the estimation procedure of the original paper rather than from a shortcoming of our estimator or of epiplexity theory. The original method trains a Transformer by gradient descent and estimates model description length from the area under the training loss curve above its final value, making the estimate heavily dependent on the optimization trajectory: even on a completely constant rule, the Transformer still takes roughly 160 steps to learn to output a constant, and the loss area accumulated during this initial convergence is priced at about 2.9×1062.9\times 10^6 bits, outscoring several chaotic and complex rules. Our closed-form estimator solves for the optimal linear readout directly without a training trajectory, scoring constant rules at exactly zero and placing the four Class IV complex rules near the top. For this reason, we do not set out to numerically replicate the trajectory artifacts of the original estimator; across the full rule space, the closed-form estimator aligns more closely with the predictions of epiplexity theory itself.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General rebuttal

We thank all three reviewers for their feedback. Our new analyses and experiments resolve every concern raised (in blue):

1. Connecting Epiplexity to Intelligence (LF6h, gf94). Finzi et al. (2026) proposed epiplexity as a passive complexity measure for given data, without treating it as a foundation for intelligence. Our core theoretical contribution is to connect epiplexity to intelligence, asking what a learning system should pursue without external goals. Maximizing total surprise LL (novelty search) chases noise, whereas minimizing free energy settles in a dark room; learning progress, OMNI, and MAGELLAN require learnable novelty, but need external task partitions or foundation-model priors (App. K). Splitting LL into the program length a bounded observer can learn (epiplexity, or learnable novelty as an objective; Eq. 4) and a random residual, we propose maximizing only the first. Whereas Finzi et al. train a network for each system scored and can only measure, our estimator (Eq. 9) is closed-form and differentiable in the data-generating system: one untuned formula drives an NCA into colliding solitons, organizes an MNIST encoder into label-free digit clusters, and gives a noise-resistant RL bonus.

2. Estimator Validation & MDL Lower Bound (LF6h). Running the original code of Finzi et al. (2026) on all 88 ECA rules (App. C) gives a rank correlation of 0.29 with our score: because it integrates the training curve, it assigns ≈2.9×106\approx 2.9\times 10^6 bits to constant rules (ours: 0) and ranks the four Class IV rules 7, 8, 66, 88 (ours: 1, 5, 6, 19). Proposition 1 (App. E) proves Sϕ≤SMDLϕS^\phi \le S^\phi_{\mathrm{MDL}} for any data, so our experiments maximize a lower bound on the exact MDL solution, as with the ELBO. The exact minimizer exceeds ours by 0.7 bits on average, with matching rankings (Spearman 0.992).

3. Baselines, Total-Surprise Control, and Noise Immunity (All Reviewers). We added RND, ICM, and total surprise LL (+ L) under the same observer across all 10 RL tasks (Table 1, App. J.4; mean ± std over 10 seeds at 600k steps; bold: above task reward; last column: no task reward):

TaskTask reward+ epiplexity+ magnitude+ RND+ ICM+ LEpiplexity
Acrobot-167±166-83±2-89±4-83±3-84±3-95±9-500±0
MountainCarCont.28±4393±193±147±4783±2893±1-47±6
Hopper1879±3252073±248516±1282037±1951992±1701799±5901282±88
BipedalWalker125±74133±57146±5190±90130±58163±58-62±41
HalfCheetah362±201352±219623±212478±246625±117505±243-563±176
Walker2d296±45281±22294±32291±26271±13276±22580±164
Swimmer181±76224±92194±76213±85205±87229±9619±14
PointMaze229±77275±31242±79267±28207±88244±447±3
LunarLander169±74176±56-171±35225±8128±2967±55-2014±555
Pendulum-1044±42-1018±23-1016±21-1041±29-1043±24-1006±23-1131±40

+ epiplexity improves on 8 of 10 tasks, tying RND (8), beating ICM (7) and + L (7) with no inner loop, and solving MountainCarContinuous on all 10 seeds; unlike + L (60% below task reward on LunarLander), it never collapses.

In our action-controlled noisy-TV test (Table 2, App. J.5), pressing a remote shows fresh Gaussian noise on extra coordinates and forfeits control for that step. Steps spent watching (%, bold: least among bonuses):

Task+ epiplexity+ RND+ ICM+ LTask reward
MountainCarCont.100.0100.0100.0100.0100.0
BipedalWalker82.197.296.197.496.9
HalfCheetah17.824.726.624.831.7
LunarLander5.37.443.423.23.8
Pendulum7.113.131.518.811.3
Acrobot0.811.131.74.57.8
PointMaze1.01.41.61.22.0
Swimmer0.02.11.60.00.0
Walker2d0.30.50.70.00.6
Hopper0.00.90.00.10.6

Epiplexity watches least on 8 of 10 tasks. As Section 2 predicts, + L is drawn to the TV (LunarLander return drops from 203 to 42), and + ICM watches 31%–43% of steps on three tasks. Scored directly, pure noise stays low: 6.4 bits for binary noise in ECA (vs. 63.3 for Rule 110), 37.1 for random fields in NCA (vs. 87–89 trained), and 3.8 for random codes in MNIST (vs. 61.7 trained).

4. Scope and Observer Dependence (NszV, gf94). Each experiment has a bounded scope (NCA shows persistent solitons, App. I.1, rather than proving universality; MNIST tests label-free clustering at default 86% rather than tuned 97%), while one untuned formula driving all three domains supports a shared principle. Because SϕS^\phi is observer-relative, the observer must match the data geometry (App. B, I): replacing the CNN observer on ECA with an equal-width MLP keeps the coarse ranking (Spearman 0.93) but compresses all rules within 1.12× of noise (vs. 9.8× with CNN), and an NCA trained against it forms no solitons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.