- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors introduce a new architecture, the prophecy transformer, that uses latent activity from the whole transformer rollout to generate latent variables through non-parametric means that can then inform the next iteration of the prophecy transformer. This, in principle, can be done an infinite number of times. They then show that the architecture beats simple autoregressive and other baselines on mostly NP complete tasks. Finally, they provide an analysis of the latent variable generated by the prophecy transformer to show that it is indeed doing something useful.
Critical strengths
- novel architecture
- interesting idea with decent motivations
- well written paper
- clear improvements over compared baselines
Critical weaknesses
- potentially weak baselines:
- I would need to see a figure that shows performance against flops to be convinced by the approach. CoT can solve the motivating problems of short-sightedness by increasing the length of the rollout. Recursive transformers can solve the issue by more recursive iterations.
- what is the scale and base model architecture for the baselines?
- are parameter counts held equal for the prophecy transformer and the baselines?
- methods could have expanded a bit more on the derichlet process (DP). I am not very familiar with such DPs and the paper's description wasn't detailed enough for me to implement the idea.
What would change my mind: if the authors provide a figure that shows performance against flops across model scales and rollout lengths, and the results are still strong, I will raise my score.
Questions for authors
- How does this improve upon recursive transformers?
- what is the scale and base model architecture for the baselines?
- are parameter counts held equal for the prophecy transformer and the baselines?
2 comments
Dear reviewer tCRF,
Since the discussion period is coming to an end, we would appreciate an acknowledgement of our rebuttal, and thanks for your time.
Thank you for writing this detailed review, and for the helpful feedback. We hope the following responses clarify your concerns.
Excluding vocabulary dependent embedding matrices which are identical between the two architectures, the exact parameter counts for PT and baseline are 142,762,753 and 141,580,032 respectively. This has been made more explicit in the updated paper.
Both PT and the baseline models share the same Qwen3 [1] transformer architecture backbone, whose details are described in Appendix A.4. The number of layers, hidden dim., attention heads, and MLP intermediate size are identical between PT and baselines. The only difference is that PT possesses exactly three additional projection matrices for the whole model to infer the parameters of the Dirichlet Process posterior.
[1] Yang, An, et al. "Qwen3 technical report." arXiv preprint arXiv:2505.09388 (2025).
The difference in parameter count is within 0.8% of the baseline parameter count, with the only addition in PT being in the three projection matrices used to infer the posterior Dirichlet Process.
Thank you for raising this point. We have run additional experiments scaling inference time compute on all of our search baselines, and have plotted their performance at three inference-time FLOP budgets for 3-SAT, graph coloring, TSP, and countdown. Our results/figures are presented in the updated version of our paper uploaded to openreview, in §5.3 ‘Scaling inference-time compute for search baselines.’ Modifications to the paper are colored blue for clarity.
In order to achieve the reported inference compute budgets, we scaled:
Models evaluated use the base model scale of 140M parameters trained on 500k unique examples for 25k steps. Details on the search baseline hyperparameters used in this analysis are presented in Appendix. A5.
Our results in Fig. 5 show that Prophecy Transformer continues to significantly outperform every search baseline across all tasks at every compute budget, including those utilizing more inference time compute.
Furthermore, in order to demonstrate that PT retains its advantage at larger model scales and at a larger number of unique training examples, we trained larger PT and baselines at 340M parameters and 1M unique training examples. We similarly plot their performance against FLOPs on all tasks (Fig. 5), and show that PT outperforms all of the search baselines we tested at all inference-time compute budgets on all tasks at this larger scale.
We note that because expended FLOPs for MCTS and Predictive Decoding depend on KV cache hits/misses when sampling rollouts, it is possible for the base scale models to expend more FLOPs than the large scale models.
We ran an experiment testing TRM (Tiny Recursive Model) [2] on 3-SAT, trained with the same data used in the base experiments (500k unique examples, 25k steps). Matching inference compute to PT at 128 GFLOPs/problem, TRM scored 2.72% on the 3-SAT test set (95% confidence interval [2.42%, 3.06%]). This is significantly worse than PT’s score of 96.5%.
We note that because TRM applies 336 forward passes per problem by default, the inference compute matched TRM has orders of magnitude fewer parameters than PT (0.62M vs 140M).
[2] Jolicoeur-Martineau, Alexia. "Less is more: Recursive reasoning with tiny networks." arXiv preprint arXiv:2510.04871 (2025).
Thank you for this feedback. We have added more detailed exposition regarding Dirichlet Processes and further references for clarity. Modifications to the paper are colored blue.