acceptodds
Under review as a conference paper at ICLR 2027

A Beginning of Infinity: Generalization by Dreaming with Neural Languages

Abstract

Generalization in neural networks is typically bounded by the amount and quality of the data used to train them. Recent neural models that learn a discrete language and its interpreter can represent programs far beyond their training distribution, but recovering these programs requires expensive search at test time. In this work, we show how this generalization capability can be transferred into a System 1 network, fully end to end. Once the learner has acquired a neural language, it can continue learning by dreaming. That is, it composes the primitives of its learned language into novel programs and executes them on inputs drawn from the training input distribution, generating new problems whose programs serve as exact training labels. The language thus becomes a source of new training problems, allowing the learner to acquire concepts that were never encoded in its original training set, without any domain-specific language, hand-written generator, or test-time search. If the learned language is universal for a problem class, dreaming can in principle reach every problem in it, even when the training data contains only a small fraction of the problems in that class. We evaluate our approach on several programming-by-example benchmarks and show that dreaming enables strong out-of-distribution generalization, matching or outperforming test-time search methods at orders of magnitude lower inference cost.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
neural program synthesis, neural language programming, compositional generalization, length generalization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gQjyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a program induction architecture which builds on previous Neurally Interpreted Language (NLI) approaches. Their method involves training (dreaming) on additional input output pairs generated according to the learned neural interpreter run on training inputs and novel recombinations of learned programs. This results in the inductor being exposed to novel problems which leads to better OOD generalization on various program induction tasks without the need for test-time search.

Critical strengths
  1. The paper’s approach is simple and elegant. The proposed approach is a clean extension of previous works (NLI). As a first time reader of the paper, it was easy to follow the motivation behind the approach, which is to amortize test-time search by consolidating the previously search-heavy inference of novel programs into the weights of the inductor (and crucially to ‘dream’ these novel training instances procedurally).
Critical weaknesses
  1. The paper introduces the notion of universality and a token reuse penalty to promote it, but severely lacks theoretical exposition or empirical results substantiating the relationship between them. As far as I can tell, there is no experimental setup investigating some notion of empirical universality in the learned tokens, or how the token reuse penalty impacts the effectiveness of the dreaming phase on generalization behavior.

  2. The method is under explored as a method for learning better program representations. Since the inductor and codebook are frozen after the initial training phase, the dreaming phase provides no learning signal for finding better representations for programs. The proposed novelty tends towards an incremental post-training strategy for a preexisting method (NLI). This would not be the case if ‘dreaming’ was interleaved throughout the main training phase, yielding fundamentally different program representations.

  3. The specific choice of benchmarks, e.g. Cell-Auto, needs to be motivated better. Currently there is little to no attempt made to convince the reader why this particular benchmark is interesting to look at.

Questions for authors
  1. Do you have any evidence of your approach being fundamentally different from NLI? As it stands, your approach can be viewed as mostly a data augmentation strategy which bridges the generalization gap by exposing the model to more data (as opposed to finding better representations). In other words, if the pre-training phase fails to discover appropriate decompositions of the training problems, it seems like the dreaming phase cannot overcome this (although you have shown that if such decompositions are indeed found, then the inductor can learn to use them in OOD settings).

  2. Are there any inductive biases that emerge from your method (not found in base NLI) that fundamentally improve generalization?

4 comments

Thank you for your feedback.

1. The paper introduces the notion of universality and a token reuse penalty to promote it, but severely lacks theoretical exposition or empirical results substantiating the relationship between them. As far as I can tell, there is no experimental setup investigating some notion of empirical universality in the learned tokens, or how the token reuse penalty impacts the effectiveness of the dreaming phase on generalization behavior.

The token reuse comes from the original NLI paper (Macfarlane et al., 2026), where they showed the need for the reuse loss (conf. their Figure 3, "no inductor loss"). We make this clear by adding citations to (Macfarlane et al., 2026) in both Section 1 and Section 3.2, where we discuss the reuse loss.

What we didn't show in our submission was the relationship between universality and dreaming. Following your comment, we present a study on the Comp-Reverse domain relating universality and dreaming in Figure 6, Appendix C.1, of the revised version. In this Figure, we show the results of dreaming for models that encode universal languages and models that don't encode universal languages. In short, the plot shows that dreaming without a universal language performs much worse than dreaming with universal ones. The reason is that without a universal language, there exist problems in the problem class that cannot be solved with the language. This prevents the system from learning how to solve such problems.

2. The method is under explored as a method for learning better program representations. Since the inductor and codebook are frozen after the initial training phase, the dreaming phase provides no learning signal for finding better representations for programs. The proposed novelty tends towards an incremental post-training strategy for a preexisting method (NLI). This would not be the case if ‘dreaming’ was interleaved throughout the main training phase, yielding fundamentally different program representations.

We agree that interleaving dreaming and training is an interesting research direction. However, we are studying the benefits of dreaming and what is required to make it work. This is where universality comes into play. As long as the learned language is universal, we don't need to interleave training and dreaming, because in principle, dreaming with a universal language already provides the mechanism for the model to learn to solve any problem from a given class.

3. The specific choice of benchmarks, e.g. Cell-Auto, needs to be motivated better. Currently there is little to no attempt made to convince the reader why this particular benchmark is interesting to look at.

We agree that the baseline needs to be better motivated. We added the following to Section 4.1 of the revised version.

Cellular automata provide a simple but expressive model of dynamical worlds, where complex global behavior emerges from local update rules (Wolfram, 1984). Some cellular automata, such as Conway's Game of Life, are Turing-complete (Rendell, 2002). They have also been used to model biological processes, such as cell differentiation and morphogenesis (Silva & Martins, 2003; Mordvintsev, 2020). Their compositional dynamics make cellular automata a natural testbed for studying neural networks learning reusable world models (Gilpin, 2019).

4. Are there any inductive biases that emerge from your method (not found in base NLI) that fundamentally improve generalization?

Thank you for bringing this up. In addition to solving problems out of distribution from training, dreaming also makes it easier to solve problems that are OOD to both training and dreaming through search. This is shown in the updated version of Figure 5(b) and Figure 11 in Appendix C.5, where we show that dreaming coupled with search is more efficient than dreaming with the base model. The intuition is that dreaming creates a larger pool of problems that the inductor can solve directly. When dealing with a problem that was not in training and not in dreaming, search is more likely to find a subproblem that the dreaming approach can solve, making the search easier. Dreaming effectively reduces the search depth of OOD problems, even for those never seen in either training or dreaming. As shown in the updated Figure 5(b) and Figure 11, depending on the problem, search is too expensive for the base model to solve but within reach after dreaming.

5. Do you have any evidence of your approach being fundamentally different from NLI? As it stands, your approach can be viewed as mostly a data augmentation strategy which bridges the generalization gap by exposing the model to more data (as opposed to finding better representations). In other words, if the pre-training phase fails to discover appropriate decompositions of the training problems, it seems like the dreaming phase cannot overcome this

The underlying model is identical to NLI, and to learn effectively through dreaming, we need a universal language. If NLI fails to learn a universal language, the performance of dreaming is weak, as we show in Appendix C.1 of the revised version. In practice, NLI reliably learns the primitives on all our benchmarks, as the dreaming results in Tables 1 and 2 show. This includes the hardest settings, where the primitives are not directly given (Comp-II) or fully hidden (Comp-Reverse) in the training tasks. The failure mode raised by the reviewer is therefore a limitation we characterize explicitly, but it does not arise in any of our experiments.

  1. The method is under explored as a method for learning better program representations. Since the inductor interpreter and codebook are frozen after the initial training phase, the dreaming phase provides no learning signal for finding better representations for programs. The proposed novelty tends towards an incremental post-training strategy for a preexisting method (NLI). This would not be the case if ‘dreaming’ was interleaved throughout the main training phase, yielding fundamentally different program representations.
  1. Are there any inductive biases that emerge from your method (not found in base NLI) that fundamentally improve generalization?

Note: I mistakenly wrote 'inductor' in place of 'interpreter' but your response shows that we are on the same page.

My two main concerns given above still seem difficult to reconcile. In my opinion, the original NLI paper's main weakness is its limitations around discovering new decompositions of OOD problems, i.e, new abstractions that help solve harder tasks. If the novel decomposition is not readily representable in its learned latent space, then gradient search is unlikely to find it. In other words, it depends heavily on the 'universality' of the discovered language.

Dreaming as a concept has the possibility of overcoming this key issue by interleaving codebook and interpreter updates with inductor updates on 'dreamt' data. However, because your approach freezes the codebook and interpreter during dreaming, the loss targets a fixed set of expressible programs. Therefore, it cannot learn novel decompositions of problems, and does not make progress on the universality bottleneck that NLI suffers from. Because the backbone of your method is entirely taken from the original NLI paper, I would need to see a more substantial addition to the intrinsic expressivity of the model.

As it stands, I maintain my belief that your approach serves more as a data augmentation technique to a backbone that is identical to the original NLI approach.

As long as the learned language is universal, we don't need to interleave training and dreaming, because in principle, dreaming with a universal language already provides the mechanism for the model to learn to solve any problem from a given class.

My belief is that your approach hinges on the universality of the learned inductor. This would be an acceptable assumption to make if the paper included explicit and rigorous justifications (i.e. theory) as to why the encoder regularization loss actually leads to universality, but this is absent as far as I can tell. The original NLI paper (Macfarlane et al., 2026) also lacks any formal justifications for why such a loss would lead to universality, in fact the term 'universality' is not mentioned in their paper. My opinion is that this paper glosses over the non-trivial nature of discovering such a universal language for describing programs, even though it is a crucial requirement for its practical success.

Reviewer_JXvjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes dreaming, a method for improving the compositional generalization of Neural Language Interpreters (NLIs). These models learn a discrete programming language and an interpreter end-to-end from input-output examples. Although the learned language can express programs outside the training distribution, recovering these programs ordinarily requires expensive test-time search.

The proposed method addresses this limitation by extracting reusable primitives from the learned language, composing them into new programs, and executing these programs with a frozen interpreter to generate synthetic training data. The model's program inductor is subsequently trained to recover the generating programs from their input-output behavior, thereby amortizing the inference previously performed through search.

The paper also introduces NLI-R, a recurrent variant of NLI that generates programs autoregressively, conditioned on intermediate execution states, allowing it to produce programs of variable length.

Experiments on three synthetic programming-by-example benchmarks demonstrate substantial improvements in out-of-distribution accuracy. Dreaming achieves near-perfect accuracy on two benchmarks and 84% on the third, outperforming test-time search methods while reducing inference computation by three to five orders of magnitude. The authors also demonstrate some length generalization beyond the maximum length of dreamed programs.

Critical strengths

(1) The paper provides a conceptually appealing mechanism for transferring the generalization capabilities of a learned programming language into a direct inference network. Unlike approaches that rely on hand-designed DSLs or manually specified primitives, the proposed method constructs synthetic problems from operations learned end-to-end from data.

(2) The improvements in Table 1 are striking. Dreaming raises NLI's out-of-distribution accuracy from effectively zero to near-perfect on Comp-I and Comp-Reverse, and NLI-R with dreaming achieves the best results on all three benchmarks.

The reduction in inference FLOPs relative to test-time search is also substantial. Furthermore, Figure 6 provides promising evidence that the recurrent architecture can generalize beyond the program lengths encountered during dreaming, suggesting that the model may learn a reusable procedure rather than simply memorizing compositions of fixed length.

Critical weaknesses

(1) The authors' central claim concerns compositional out-of-distribution generalization, but the principal experiments evaluate program lengths that are explicitly represented during the dreaming stage.

For example, Comp-I and Cell-Auto train initially on individual primitives and evaluate on compositions of length 2–4, while dreaming generates programs up to length 4. Likewise, Comp-Reverse evaluates on lengths 1 and 4, both within the range of lengths generated during dreaming.

Consequently, the strong results in Table 1 primarily demonstrate that a model can learn to recognize compositions that are novel relative to its original training data but not necessarily novel relative to its synthetic training distribution.

This is a useful result, but it is different from demonstrating systematic generalization to unseen compositions. The paper does not clearly establish what fraction of evaluation programs or distinct program behaviors are absent from the dreamed dataset.

Figure 6 partly addresses this concern by evaluating lengths beyond those encountered during dreaming. However, performance deteriorates substantially with length, and the results are reported for a single seed.

Given the paper's emphasis on generalization and universality, stronger evaluation on held-out compositions and longer programs would substantially strengthen the contribution.

(2) The method introduces several potentially important components, including a token-reuse loss, BPE-based primitive extraction, program composition, synthetic data generation, and additional supervised training of the inductor. However, the paper provides no systematic ablations establishing which components are necessary for its improvements.

In particular, the authors argue that learning reusable primitives rather than task-specific macros is essential for dreaming, but do not experimentally isolate the effect of the reuse loss or demonstrate how the composition and quality of the extracted library affect generalization.

(3) Although the inference savings are substantial, the dreaming procedure requires generating hundreds of thousands or millions of synthetic tasks and additional training. Without reporting this computational overhead, it is difficult to evaluate the overall efficiency of the approach or determine when amortization becomes beneficial.

(4) There are also potential confounds in the experimental comparisons. Appendix Tables 3 and 5 report four additional encoder layers and two causal head layers during dreaming for Comp-Reverse, despite the main text stating that the inference architecture is unchanged, but Table 1 reports identical inference FLOPs for the dreaming and non-dreaming variants.

Questions for authors

(1) How much of the reported OOD generalization is attributable to coverage of the evaluation distribution during dreaming? In particular, can the authors report what fraction of evaluation programs, or semantically distinct functions, are represented in the dreamed dataset? What happens when evaluation is restricted to program compositions that were not explicitly generated during dreaming?

(2) How important are the individual components of the dreaming procedure? I would be particularly interested in ablations removing the token-reuse loss, replacing BPE-extracted primitives with randomly sampled learned tokens, and varying the maximum dream length and number of dreamed tasks. These results would help distinguish the benefits of the learned compositional representation from those of additional synthetic supervision.

(3) How robust is the observed length generalization, and how faithfully does the learned interpreter execute novel compositions? Figure 6 reports a single seed. Would the authors provide results across multiple seeds? Additionally, measuring the agreement between the frozen learned interpreter and the ground-truth benchmark functions on increasingly long compositions would help determine whether declining accuracy originates from imperfect program induction, imperfect execution, or both.

(4) Can the authors clarify the architectural and computational comparisons? Appendix Tables 3 and 5 indicate that additional encoder and causal-head layers are introduced for Comp-Reverse during dreaming.

4 comments

Thank you for your feedback.

1. Consequently, the strong results in Table 1 primarily demonstrate that a model can learn to recognize compositions that are novel relative to its original training data but not necessarily novel relative to its synthetic training distribution.

This is correct, but note that this is only possible because the architecture learns a disentangled representation that allows it to learn how to solve problems that were not included in the training set. From the model's perspective, it generalizes because it trains on synthetic problems. However, from the user's perspective, who only provides the training set, the model truly generalizes OOD because the user never provided the synthetic data to the model.

2. How much of the reported OOD generalization is attributable to coverage of the evaluation distribution during dreaming? In particular, can the authors report what fraction of evaluation programs, or semantically distinct functions, are represented in the dreamed dataset? What happens when evaluation is restricted to program compositions that were not explicitly generated during dreaming?

To better answer this question, we added a vertical bar in Figure 5(b) of the revised version to denote the depth the dreaming process reaches. So, everything on the right-hand side of this vertical line is OOD for the dreaming process. When coupled with search, dreaming lets the learned model generalize beyond what it can achieve without it, even for data not encountered during dreaming. The dreaming advantage on data outside of the dreaming range comes from a bootstrapping effect of the search. Once the search finds a subproblem that is in-distribution with respect to dreaming, the search becomes easy. This is now explained in Section 4.5 of the revised version.

3. How important are the individual components of the dreaming procedure? I would be particularly interested in ablations removing the token-reuse loss, replacing BPE-extracted primitives with randomly sampled learned tokens, and varying the maximum dream length and number of dreamed tasks. These results would help distinguish the benefits of the learned compositional representation from those of additional synthetic supervision.

Thank you for bringing this up. The token reuse comes from the original NLI paper (Macfarlane et al., 2026), where they showed the need for the reuse loss (conf. their Figure 3, "no inductor loss"). We make this clear by adding citations to (Macfarlane et al., 2026) in both Section 1 and Section 3.2, where we discuss the reuse loss. What we didn't show in our submission was the relationship between universality and dreaming, which can now be found in Appendix C.1. We have added the following ablation studies:

  • Appendix C.1: Ablation on language universality
    • Dreaming improves with primitive coverage and achieves perfect accuracy only with a universal language.
  • Appendix C.2: Ablation on primitive extractor
    • BPE recovers multi-token primitives, but random selection misses them, which degrades dreaming.
  • Appendix C.3: Ablation on dream dataset size
    • Dreaming improves with the number of dreamed tasks, as more tasks provide more diverse examples for each program.
  • Appendix C.4: Ablation on dreaming loss
    • Dreaming relies on direct program supervision from the dreamed tasks.
  • Appendix C.5: Ablation on maximum dream depth
    • Beyond the maximum dream depth, the inductor fails but the interpreter remains correct, and search on the dreamed models recovers far more of these tasks than search on the base model.

4. Figure 6 reports a single seed. Would the authors provide results across multiple seeds?

We have added the average results of 5 seeds and standard deviation. As can be seen in Figure 5(b) of the revised version, the system generalizes nicely for the in-distribution of training, dreaming, and even OOD of dreaming provided that search is used.

5. Measuring the agreement between the frozen learned interpreter and the ground-truth benchmark functions on increasingly long compositions would help determine whether declining accuracy originates from imperfect program induction, imperfect execution, or both.

We evaluated the interpreter on tasks of length 1 to 15, and found that it matches the ground-truth execution exactly at every length. The interpreter therefore executes long programs correctly, even far beyond the maximum dream depth. We further applied rollout-based search to the dreamed models and found that it recovers many tasks beyond the maximum dream depth. The gap between the inference-only model after dreaming and the dreaming with search points at imperfect program induction as the cause of the decline. Please see Appendix C.5 of the revised version.

6. Can the authors clarify the architectural and computational comparisons? Appendix Tables 3 and 5 indicate that additional encoder and causal-head layers are introduced for Comp-Reverse during dreaming.

Thank you for bringing this up. We reran the experiments on Comp-Reverse with the same architecture used in the other domains (no additional encoder or causal-head layers). There was no noticeable change in the quality of the results. We updated the paper throughout accordingly, including the referred tables in the appendix.

The computation used with all approaches is measured in terms of FLOPs for inference only. We argue that the computation needed in the dream phase is amortized. That is, as the number of test problems we solve with the architecture grows large, the training cost goes to zero. That is why we focus on inference-time FLOPs in our comparisons. In the revised version, we also specify which hardware we used at the end of Section 4.1.

Thank you for addressing the questions in my review; the paper is experimentally stronger now. I raise my score.

Thank you for your thoughtful review and for raising your score. Your comments helped us strengthen the paper, and we truly appreciate the time you put into the review.

Reviewer_nCjDOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper consider the problem of Programming by Examples (PBE), where given a number of input-output pairs, one needs to approximate the program which was used to generate the observed outputs.

The proposed method builds on a previous method - Neural Language Interpreters (NLI), in which programs are expressed as discrete latent tokens. NLI uses an inductor to predict a latent program, that is executed by an interpreter on given input (both inductor and interpreter are neural networks).

The authors propose to introduce a second training phase to NLI, where they examine the available latent programs and detect repeating patterns of latent codes. These patterns are then composed in novel ways in order to generate new programs that could be encountered in the future. The new programs are used to generate a new training dataset, which the interpreter is finetuned on. As a result, the interpreter is better equipped to handle such programs at test time, leading to better generalisation performance.

Since NLI’s interpreter is confined to generating programs with a maximum length of T, the authors also introduce a recursive variant capable of generating longer programs.

Critical strengths

Provides new evidence about how incorporating a “dreaming” phase could help with out of distribution generalisation.

Critical weaknesses

Limited contributions. The method’s novelty builds upon the idea of a sleep phase, introduced in DreamCoder. There is a lack of analysis to characterize why and when this approach works, which might have made up for the limited novelty.

Limited experimental evidence. The experimental setting appears very simple (Comp-I and Cell-Auto directly teach the primitives, and the length generalization is 2-4), and not challenging enough to demonstrate the strength and weaknesses of the method. Moreover, claiming to dream to infinity, suggests a very strong generalization claim that is unsupported by the results.

Questions for authors

q1. New data is said to be generated by sampling the inputs uniformly from some input distribution. What input distribution is used?

q2. Could you help me understand why the experimental setup is challenging? The paper states that I/O pairs are generated by single primitives (Comp-I and Cell-Auto), and OOD evaluation requires composing them into programs of length 2-4.

3 comments

Thank you for your feedback.

1. Limited contributions. The method’s novelty builds upon the idea of a sleep phase, introduced in DreamCoder. There is a lack of analysis to characterize why and when this approach works, which might have made up for the limited novelty.

Thank you for giving us the opportunity to clarify the novelty issue. While we use the sleeping mechanism from DreamCoder, we use it in a completely novel setting. That is, DreamCoder uses a Lambda Calculus (LC) as its hypothesis space. LC is inherently modular and symbolic, which makes it considerably easier to dream. We use neural networks, fully differentiable end-to-end. Our system has no access to domain-specific languages; it must discover them via gradient descent and use the discovered symbols to learn through dreaming.

2. Limited experimental evidence. The experimental setting appears very simple (Comp-I and Cell-Auto directly teach the primitives, and the length generalization is 2-4), and not challenging enough to demonstrate the strength and weaknesses of the method.

Following your comment, we expanded the set of experiments in the paper. In addition to the setting we originally had (training in depth 1 and testing OOD on depths 2-4), we included version II of the domains, where we train on depths 1-3 and test OOD on 4-6. The primitives are not clearly given in the training set, so the model needs to disentangle them during training. We also note that our Comp-Reverse benchmark evaluates this, since training is on depths 2 and 3 and OOD testing is on depths 1 and 4.

We also added experiments on an image-based benchmark where the model needs to learn a language to perform image-editing operations on CIFAR-10 images (see Image-Edit-I and II in the revised version). See Section 4.1 of the revised version for details.

3. Claiming to dream to infinity, suggests a very strong generalization claim that is unsupported by the results.

We thank the reviewer for pointing this out. We agree that our original discussion did not distinguish between a language's universality and the generalization capabilities of a model trained through dreaming.

Our claim about dreaming to infinity is about the potential coverage of the problem class. If the learned language is universal and dreaming can generate arbitrarily long programs, every problem in the class is, in principle, reachable. However, this does not guarantee that the neural network will learn to solve every generated problem.

To address this, we added the following paragraph at the end of Section 4.5. Moreover, we changed the section title from “Dream to Infinity” to “Generalization Beyond Training”.

Importantly, dreaming does not guarantee that a model will generalize to every problem in a given problem class. If the learned language is universal for that class and the dreaming process can generate programs of arbitrary finite length, then every problem in the class can, in principle, be generated through dreaming. However, generating these problems does not guarantee that the model will learn to solve them. Our experiments provide evidence of generalization beyond the training distribution, but not of unbounded generalization. As shown in Figure 5(b) and Appendix C.5, dreaming allows the model to generalize to much longer tasks than those observed during training. Generalization to even longer tasks may require additional dreaming or test-time search.

4. New data is said to be generated by sampling the inputs uniformly from some input distribution. What input distribution is used?

Dreaming samples inputs from the same input distribution used to generate the training data. That is, for the Comp benchmark, each input is uniformly distributed over all permutations of the digits 0,1, ⋯\cdots, 9; for the Cell-Auto benchmark, the input distribution is over 5x5 binary grids, where each cell is independently alive with probability 0.35; for the Image-Edit benchmark, each input is uniformly distributed over images in the CIFAR-10 training set. We have clarified this in Section 3.3 and added a description of the input distribution for each benchmark in Appendix B.

5. Could you help me understand why the experimental setup is challenging? The paper states that I/O pairs are generated by single primitives (Comp-I and Cell-Auto), and OOD evaluation requires composing them into programs of length 2-4.

There are two axes in which the experimental setup is challenging. The first is that the model must perform more computation at test time than it was trained to perform at training time (e.g., longer compositions). The other axis is the model's ability to disentangle primitives from training data so they can be composed at test time.

In our submission, we explored mostly only the first axis and explored the second in a single benchmark (Comp-Reverse), so we thank the reviewer for pointing that out. In the revised version, we add four new benchmarks (Comp-II, Cell-Auto-II, Image-Edit-I, and Image-Edit-II), which thoroughly explore both axes. Taken together, the need to perform more computation at test time and to disentangle primitive operations from data form a particularly challenging setting for current neural architectures.

Thank you for your response and for adding more empirical evidence. Please see my remaining concerns below:

  1. Limited novelty. You made the point that this paper’s setting is more challenging, due to the use of NNs and the need to discover the language. I agree that this adds complexity, but it seems to me that this complexity is handled by NLI (previous work). This paper’s method operates on top of the discrete language discovered by NLI, and uses symbolic operations, e.g. BPE and program concatenation. Could you please help me by stating the methodological novelty of the paper: what are the major novel insights that made the method possible?
  2. Generalization bounded by the input distribution. During dreaming, the method uses inputs from the training distribution. In principle, this appears to place a limit on the diversity of programs that can be dreamed up and learned effectively: only ones for which inputs from the training distribution are valid inputs.
  3. Generalization bounded by the dreaming length. Generalization to programs longer than those seen during training appears to be bounded by the length of programs encountered during dreaming (Appendix C.5 shows a sharp drop right after the max dream depth). In principle, as the length of programs we want to support grows, the number of programs we need to dream grows exponentially, so it is not clear how this approach scales without falling back on test-time search.

Overall, my current concerns are that 1. points to a limited novelty, and 2. and 3. point to a method that works well on toy examples, however, will not scale to harder and more realistic problems.

Comments on the paper

1 comment

We thank all the reviewers for their time and thoughtful comments on our submission. We worked carefully to handle their comments and suggestions and we have submitted a revised version of our work. All modifications in the revised version are highlighted in blue. In summary,

  1. Following the comments of Reviewer nCjD, we added experiments with benchmarks whose primitive operations are entangled in the training set. We also added experiments on a benchmark that requires the model to learn operations over images.

  2. Following the questions and suggestions of Reviewer JXvj, we increased the number of seeds, provided experiments showing that the decrease in performance is due to poor program induction, and we included the following ablations in the revised version:

    • Ablation on language universality
    • Ablation on primitive extractor
    • Ablation on dream dataset size
    • Ablation on dreaming loss
    • Ablation on maximum dream depth
  3. Following the comments of Reviewer gQjy we studied empirically the relationship of universality and dreaming. We also showed empirically that dreaming makes searching for OOD problems easier, even those not seen in training or dreaming.

The details of these and other revisions are in our responses below. Thanks to the reviewers’ comments and suggestions, our paper has improved significantly. We would be happy to answer any further questions they may have during the discussion period.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.