- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose a program induction architecture which builds on previous Neurally Interpreted Language (NLI) approaches. Their method involves training (dreaming) on additional input output pairs generated according to the learned neural interpreter run on training inputs and novel recombinations of learned programs. This results in the inductor being exposed to novel problems which leads to better OOD generalization on various program induction tasks without the need for test-time search.
Critical strengths
- The paper’s approach is simple and elegant. The proposed approach is a clean extension of previous works (NLI). As a first time reader of the paper, it was easy to follow the motivation behind the approach, which is to amortize test-time search by consolidating the previously search-heavy inference of novel programs into the weights of the inductor (and crucially to ‘dream’ these novel training instances procedurally).
Critical weaknesses
-
The paper introduces the notion of universality and a token reuse penalty to promote it, but severely lacks theoretical exposition or empirical results substantiating the relationship between them. As far as I can tell, there is no experimental setup investigating some notion of empirical universality in the learned tokens, or how the token reuse penalty impacts the effectiveness of the dreaming phase on generalization behavior.
-
The method is under explored as a method for learning better program representations. Since the inductor and codebook are frozen after the initial training phase, the dreaming phase provides no learning signal for finding better representations for programs. The proposed novelty tends towards an incremental post-training strategy for a preexisting method (NLI). This would not be the case if ‘dreaming’ was interleaved throughout the main training phase, yielding fundamentally different program representations.
-
The specific choice of benchmarks, e.g. Cell-Auto, needs to be motivated better. Currently there is little to no attempt made to convince the reader why this particular benchmark is interesting to look at.
Questions for authors
-
Do you have any evidence of your approach being fundamentally different from NLI? As it stands, your approach can be viewed as mostly a data augmentation strategy which bridges the generalization gap by exposing the model to more data (as opposed to finding better representations). In other words, if the pre-training phase fails to discover appropriate decompositions of the training problems, it seems like the dreaming phase cannot overcome this (although you have shown that if such decompositions are indeed found, then the inductor can learn to use them in OOD settings).
-
Are there any inductive biases that emerge from your method (not found in base NLI) that fundamentally improve generalization?
4 comments
5. Do you have any evidence of your approach being fundamentally different from NLI? As it stands, your approach can be viewed as mostly a data augmentation strategy which bridges the generalization gap by exposing the model to more data (as opposed to finding better representations). In other words, if the pre-training phase fails to discover appropriate decompositions of the training problems, it seems like the dreaming phase cannot overcome this
The underlying model is identical to NLI, and to learn effectively through dreaming, we need a universal language. If NLI fails to learn a universal language, the performance of dreaming is weak, as we show in Appendix C.1 of the revised version. In practice, NLI reliably learns the primitives on all our benchmarks, as the dreaming results in Tables 1 and 2 show. This includes the hardest settings, where the primitives are not directly given (Comp-II) or fully hidden (Comp-Reverse) in the training tasks. The failure mode raised by the reviewer is therefore a limitation we characterize explicitly, but it does not arise in any of our experiments.
- The method is under explored as a method for learning better program representations. Since the
inductorinterpreter and codebook are frozen after the initial training phase, the dreaming phase provides no learning signal for finding better representations for programs. The proposed novelty tends towards an incremental post-training strategy for a preexisting method (NLI). This would not be the case if ‘dreaming’ was interleaved throughout the main training phase, yielding fundamentally different program representations.
- Are there any inductive biases that emerge from your method (not found in base NLI) that fundamentally improve generalization?
Note: I mistakenly wrote 'inductor' in place of 'interpreter' but your response shows that we are on the same page.
My two main concerns given above still seem difficult to reconcile. In my opinion, the original NLI paper's main weakness is its limitations around discovering new decompositions of OOD problems, i.e, new abstractions that help solve harder tasks. If the novel decomposition is not readily representable in its learned latent space, then gradient search is unlikely to find it. In other words, it depends heavily on the 'universality' of the discovered language.
Dreaming as a concept has the possibility of overcoming this key issue by interleaving codebook and interpreter updates with inductor updates on 'dreamt' data. However, because your approach freezes the codebook and interpreter during dreaming, the loss targets a fixed set of expressible programs. Therefore, it cannot learn novel decompositions of problems, and does not make progress on the universality bottleneck that NLI suffers from. Because the backbone of your method is entirely taken from the original NLI paper, I would need to see a more substantial addition to the intrinsic expressivity of the model.
As it stands, I maintain my belief that your approach serves more as a data augmentation technique to a backbone that is identical to the original NLI approach.
As long as the learned language is universal, we don't need to interleave training and dreaming, because in principle, dreaming with a universal language already provides the mechanism for the model to learn to solve any problem from a given class.
My belief is that your approach hinges on the universality of the learned inductor. This would be an acceptable assumption to make if the paper included explicit and rigorous justifications (i.e. theory) as to why the encoder regularization loss actually leads to universality, but this is absent as far as I can tell. The original NLI paper (Macfarlane et al., 2026) also lacks any formal justifications for why such a loss would lead to universality, in fact the term 'universality' is not mentioned in their paper. My opinion is that this paper glosses over the non-trivial nature of discovering such a universal language for describing programs, even though it is a crucial requirement for its practical success.
Thank you for your feedback.
1. The paper introduces the notion of universality and a token reuse penalty to promote it, but severely lacks theoretical exposition or empirical results substantiating the relationship between them. As far as I can tell, there is no experimental setup investigating some notion of empirical universality in the learned tokens, or how the token reuse penalty impacts the effectiveness of the dreaming phase on generalization behavior.
The token reuse comes from the original NLI paper (Macfarlane et al., 2026), where they showed the need for the reuse loss (conf. their Figure 3, "no inductor loss"). We make this clear by adding citations to (Macfarlane et al., 2026) in both Section 1 and Section 3.2, where we discuss the reuse loss.
What we didn't show in our submission was the relationship between universality and dreaming. Following your comment, we present a study on the Comp-Reverse domain relating universality and dreaming in Figure 6, Appendix C.1, of the revised version. In this Figure, we show the results of dreaming for models that encode universal languages and models that don't encode universal languages. In short, the plot shows that dreaming without a universal language performs much worse than dreaming with universal ones. The reason is that without a universal language, there exist problems in the problem class that cannot be solved with the language. This prevents the system from learning how to solve such problems.
2. The method is under explored as a method for learning better program representations. Since the inductor and codebook are frozen after the initial training phase, the dreaming phase provides no learning signal for finding better representations for programs. The proposed novelty tends towards an incremental post-training strategy for a preexisting method (NLI). This would not be the case if ‘dreaming’ was interleaved throughout the main training phase, yielding fundamentally different program representations.
We agree that interleaving dreaming and training is an interesting research direction. However, we are studying the benefits of dreaming and what is required to make it work. This is where universality comes into play. As long as the learned language is universal, we don't need to interleave training and dreaming, because in principle, dreaming with a universal language already provides the mechanism for the model to learn to solve any problem from a given class.
3. The specific choice of benchmarks, e.g. Cell-Auto, needs to be motivated better. Currently there is little to no attempt made to convince the reader why this particular benchmark is interesting to look at.
We agree that the baseline needs to be better motivated. We added the following to Section 4.1 of the revised version.
4. Are there any inductive biases that emerge from your method (not found in base NLI) that fundamentally improve generalization?
Thank you for bringing this up. In addition to solving problems out of distribution from training, dreaming also makes it easier to solve problems that are OOD to both training and dreaming through search. This is shown in the updated version of Figure 5(b) and Figure 11 in Appendix C.5, where we show that dreaming coupled with search is more efficient than dreaming with the base model. The intuition is that dreaming creates a larger pool of problems that the inductor can solve directly. When dealing with a problem that was not in training and not in dreaming, search is more likely to find a subproblem that the dreaming approach can solve, making the search easier. Dreaming effectively reduces the search depth of OOD problems, even for those never seen in either training or dreaming. As shown in the updated Figure 5(b) and Figure 11, depending on the problem, search is too expensive for the base model to solve but within reach after dreaming.