acceptodds
Under review as a conference paper at ICLR 2027

Open-world AI: a Tale of three Requirements

Abstract

The closed world asks AI for specialized competence and lets it manufacture the iid data its theory assumes: a fixed criterion licenses collecting examples until the task is covered. The open world asks for versatility — taking up tasks a machine was not built for, from few examples and with little prior from a human designer — and offers data that is grouped, not shuffled. Its tasks are not known in advance, so a machine must do *inference-time learning* (1st requirement). Modeling tasks as sharing latent factors and decomposing the training objective, we get two more: a *rich representation*, retaining more than iid generalization requires, and a *predictive disentanglement* that aligns inputs and outputs into matched low-dimensional groups. The analysis further predicts the need for a consistent inner learner, and the benefit of long contexts spanning related tasks. We test these claims on language models up to 8B, asking whether a model has a property (qualitative) rather than how much it wins (quantitative). Models that meet the requirements improve as data from related tasks accumulates; the rest degrade, and neither scale nor layer-type interleaving repairs it.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Open-world AI, rich representation, disentanglement, inference-time learning.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_e9azOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose open-world AI should be able to learn new tasks from a few examples. The 3 things needed to achieve this are: 1. It continually learns; 2. It learns features that are useful outside the current task; 3. It splits its learnings into groups predicted separately. The authors claim the part of the model that learns from examples improves further upon seeing more. Furthermore, the authors claim the model should handle similar tasks separately from the current task and then combine them. They test this on language models (1.5B and 8B parameters) trained on groups of similar documents. Memory Mosaics, a model that the authors claim uses this way of learning, gets better when trained on longer texts. They also show that standard Transformers and other popular models do not get better when trained on longer texts. The authors also build "two-stage" models with memories for current documents separate from memories for other similar documents and keep improving as similar documents are added. Models with a single memory stop improving around their training length.

Critical strengths
  • As the authors postulate, Memory Mosaics improves with longer training contexts, while the other models mostly get worse. This is good evidence to support the paper's claims.
  • Two-stage Memory Mosaics+ keeps improving with related context up to 8 times its training length, at both 1.5B and 8B. The one-stage models peak around their training length. This can be useful for long-context models.
Critical weaknesses
  • Fig. 5 doesn't show that separate memories for the current and related documents are the contributing reasons for gains. The two-stage models also use the "+" design. No one-stage "+" model is tested in the Fig. 5 setup.

  • The paper claims Memory Mosaics improves because it's consistent, but this is not tested. Memory Mosaics also differs from the other models in position encoding, key and value construction, and bandwidth. There aren't any experiments which change consistency and keep all else equal. The paper also does not check which models are actually consistent.

Questions for authors
  • In the Figure 5 setting, do one-stage Transformer+ and Memory Mosaics+ also avoid the decline?
  • Do the models trained at l = 64k peak near 64k when evaluated out to 128k?
  • Do two-stage models degrade less than one-stage models when the related context comes from a mismatched cluster?
  • Can you vary consistency within a single model family, for example Memory Mosaics with a constant bandwidth?
  • Do the learned bandwidth parameters make Memory Mosaics consistent?
  • Given your definition of consistent, is NoPE full attention consistent?
  • How is top-1 accuracy at evaluation length l′ computed?
  • At what model scale were Tables 1-3 run?
  • How much do the results vary across random seeds?
  • What causes Hybrid DeltaNet's drop between l = 16k and 32k?
  • What is already covered in Zhang et al. (2025) and Zhang & Bottou (2026) vs what is novel in this paper?
Reviewer_ZVvoOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes three requirements for open-world AI: (1) inference-time learning to adapt to unseen tasks, (2) rich representations that retain information useful across tasks, and (3) predictive disentanglement that decomposes prediction into smaller, sample-efficient problems.

The authors motivate these requirements through a decomposition of prediction error into representation error and learner error. They argue that a consistent inner learner enables rich representation learning with large support sets, while predictive disentanglement facilitates adaptation with few examples. They further propose separating local and long-range processing, as well as target-task and related-task memories.

Experiments on language models up to 8B parameters compare Transformers, DeltaNet, Memory Mosaics, and modified architectures. The results show that longer-context training benefits Memory Mosaics but generally not the other architectures, increasing the number of heads modestly improves Memory Mosaics, and the proposed two-stage architectures benefit from increasingly long contexts containing related documents.

Critical strengths

Interesting theoretical perspective. The decomposition of prediction error into representation and learner errors provides an interesting perspective connecting representation learning, sample complexity, and inference-time adaptation. The distinction between learning rich representations and organizing them for efficient adaptation is conceptually interesting and motivates concrete architectural hypotheses.

Critical weaknesses

1. Insufficient evidence connecting consistency to richer representations (Experiments 1 and 2)

The authors argue that a consistent inner learner has vanishing learner error as the support size grows, allowing training with large support sets to reduce representation error and produce richer representations. However, I have several concerns about both the theoretical argument and its empirical validation.

i. Consistency is not necessary. The argument only requires learner error to become sufficiently small, not necessarily vanish asymptotically. Even an inconsistent learner with small but nonzero asymptotic error could learn rich representations. Therefore, consistency appears to be a sufficient condition rather than a fundamental requirement.

ii. The experiments do not establish richer representations. In Table 1, increasing training context length from 4K to 64K improves Memory Mosaics accuracy by only 0.64 percentage points (48.55% to 49.19%) at 4K evaluation length. Meanwhile, Table 2 shows that the same increase actually degrades standard Transformer performance (48.40% to 47.51%). The authors interpret these architecture-dependent trends through consistency and representation richness, but neither property is directly measured or established experimentally.

iii. Alternative explanations are not ruled out. Longer-context training might lead to easier or more effective optimization, or improve the inference-time learner itself through better memory retrieval and aggregation, thereby reducing learner error rather than representation error. Since the experiments only measure overall prediction accuracy, they cannot distinguish these possibilities. Moreover, the theory concerns increasing the number of support examples from the same task, whereas the experiments increase training context length by concatenating semantically related documents. The equivalence between these settings is not justified.

Overall, Experiments 1 and 2 demonstrate an architecture-dependent effect of longer-context training, but do not establish that consistency is responsible or that the observed improvements arise from richer representations. The authors should provide more direct evidence linking the experimental observations to the proposed theoretical mechanism.

2. Insufficient evidence for predictive disentanglement (Experiment 3)

The authors argue that predictive disentanglement reduces sample complexity by decomposing prediction into smaller, independent input-output groups. However, Experiment 3 does not convincingly validate this claim.

In Table 3, the authors increase the number of heads and observe a modest improvement for Memory Mosaics (48.43% to 49.07% at 32 heads), while Transformer performance deteriorates. They interpret this as evidence supporting predictive disentanglement. However, increasing the number of heads does not necessarily imply that the learned representations become more disentangled. It also changes the per-head dimensionality and potentially the optimization dynamics, which could explain the observed improvements.

More importantly, the authors do not directly measure predictive disentanglement or demonstrate its claimed sample-complexity benefits. In particular, they do not verify whether the learned representations satisfy the group-wise factorization assumption in Eq. (4), nor do they show that increasing the number of groups reduces the number of support examples required to learn a new task.

Therefore, the experiment establishes an architectural effect of the number of heads, but does not demonstrate that predictive disentanglement is responsible for the improvement. A direct measure of group-wise predictive dependence and an evaluation of few-shot learning curves would provide stronger evidence for the proposed mechanism.

3. Overall:

I find a significant gap between the paper's theoretical claims and what is actually demonstrated experimentally. They do not convincingly validate the proposed mechanisms of richer representations, predictive disentanglement, or more effective inference-time learning. This gap substantially weakens the paper's central contribution.

Questions for authors

Please see the Weaknesses.

Reviewer_WUCeOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a set of requirements necessary to train artificially intelligent systems that succeed on "Open-world AI" problems requiring flexibility and online adaptation. The authors claim that these three qualitative requirements (inference-time learning, rich representation, and predictive disentanglement) must be exhibited by an intelligent system that succeeds in such "open-world" settings, and present a set of experiments use to argue for the limitations of particular architectural (e.g., Transformer, DeltaNet, Memory Mosaic) and training decisions (IID sampling of training data).

Critical strengths

I am sympathetic to the view that the dominant training and architectural paradigms may not be sufficient to train systems possessing the flexibility necessary for embodied behavior in rich, temporally-extended environments -- on this dimension at least I appreciate the problem that the paper sets out to address. However, the paper suffers from a number of critical limitations that I expand upon below.

Critical weaknesses

In presenting such a high-level and qualitative framework, the paper suffers from several critical weaknesses, the most serious being that its arguments are almost entirely speculative and are at times in conflict with large bodies of existing evidence.

Lack of empirical support: The paper's primary flaw is the weakness of its empirical evidence (1). offered in support of assertions made in the framing of the paper and (2). offered by their experiments, which the authors describe themselves as "qualitative". The authors then go on to make broad claims about the weaknesses of the dominant training paradigms and architectures on the basis of such qualitative evidence.

  • "In closed-world AI, both training and test examples are assumed to be drawn iid" -> The author's own definition of "inference time learning" means that non-IID structure present within the training/test examples (e.g., in-context) would qualify as non-IID structure. In fact, such non-IID structure is exactly what leads to the emergence of in-context learning in LLMs.
  • The authors go on to claim that in the current training paradigm, "Neither more task-specific iid data nor more scale reliably improves performance on the task at hand. This is not yet learning". There is evidence that both model-scale and the presence of task-data influence model learning outcomes (both in-weight and in-context, e.g., https://arxiv.org/abs/2303.03846)

Failure to engage seriously with relevant prior work: The paper fails to engage seriously with a wide range of related research including but not limited to the literature on continual learning, the entire literature around in-context learning, the literature around latent-cause inference (which is highly relevant to the authors claims about naturalistic data-distributions).

Clarity of presentation: The paper struggles with substantial issues with clarity that made it difficult to follow, particularly in the results sections.

Questions for authors

Could the authors clarify how their definition of "inference time learning" is distinct from traditional definitions of in-context learning? I found this point particularly confusing when reading the paper, and understanding it is necessary to evaluate the strength of the authors claims about current models lacking "inference-time learning".

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thanks to the reviewers and next steps

We thank the reviewers for their time and thoughtful feedback.

The introduction of this paper sets out an ambitious goal toward open-world AI, one that a single conference paper can hardly fully support. Colleagues who read the paper told us that it is too dense; the reviewers found it incomplete, noting the lack of a separate related-work section, the unclear distinction between inference-time learning and in-context learning, the question of consistency versus a sufficiently small learner error, and the missing one-stage + control, among others.

We see these as two views of the same problem: we tried to fit too much into one paper. Seen from one side, the result is dense; seen from the other, certain helpful explanations, context, and controls are missing.

Either way, it makes it hard for readers to engage with the paper's new theory and experiments. We will rewrite the work to be more accessible and complete, and return with either a longer version or two separate ones at a future venue. We thank the reviewers again; their comments will make the next version substantially better.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.