acceptodds
Under review as a conference paper at ICLR 2027

Unrolled Flow Models for Reasoning

Abstract

Flow matching enables language generation in few steps, but whether additional integration steps improve reasoning remains unclear. We prove that a flow parameterized by a two-layer Transformer can solve graph reachability, with the required number of integration steps increasing with the target’s distance from the root. Yet, standard flow language models can fail to benefit from additional steps on reasoning tasks. We attribute this limitation to objectives that supervise each time point independently, without explicitly training successive steps to build on one another. To address this, we instead train through the model’s own latent rollout over a randomly sampled subinterval of [0, 1], decoding only at the endpoint. On ProsQA, this raises accuracy to 97% and enables performance to improve with additional integration steps. For the longer rollouts required by reasoning tasks such as Sudoku and Maze, retracting the latent state onto a sphere stabilizes the dynamics and yields substantial gains over baselines with more than three times as many parameters. Sampling multiple rollouts further improves performance when paired with a parameter-free selection score, although reliable selection remains challenging for longer answers. Together, these results establish a theoretical basis for reasoning with flows and show how rollout training, stable latent dynamics, and rollout selection help realize this capacity in practice.

Primary area:
generative models
Keywords:
flow matching, flow language models, latent reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_aQQsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how flow models can leverage additional integration steps for reasoning. Then, it proves that a flow parameterized by a two-layer Transformer can solve graph reachability. Furthermore, it proposes UFM, which trains through the model’s own latent rollout using a single endpoint loss, and shows that it outperforms standard flow language models on several reasoning tasks.

Critical strengths
  1. The paper identifies a clear limitation of standard flow language models, that extra sampling steps barely help on reasoning tasks.
  2. Against the flow baselines it compares with, UFM performs clearly better on the two tasks (using a smaller model).
Critical weaknesses
  1. UFM is a flow model in name only. Its only training signal is a cross-entropy loss at the endpoint, and it does not uses the FM theoretical framework. Essentially, it is a recurrent network trained by backpropagating through its own iterations, which is already explored by the existing recursive reasoners and truncated sampler backpropagation. So, what's new?

  2. During training, UFM initializes its trajectory from a mixture of Gaussian noise and the ground-truth answer embedding, whereas inference starts from pure noise. This introduces privileged answer information into training, potentially allowing the model to learn denoising shortcuts rather than reasoning from the prompt.

  3. The experiments do not isolate UFM's gains. Baselines are limited to FLM and S-FLM, which are not built for reasoning, with no recursive or plain recurrent baseline under the same protocol. No component is properly ablated.

Questions for authors

.

4 comments
AuthorsOpenReview ↗

Response to Reviewer aQQs — Part 1/2

We thank the reviewer for appreciating the merits of our paper and their suggestions for improvement.

We have uploaded a revised version of the paper, with all changes highlighted in blue. All references to pages, sections, figures, tables, and equations use the revised version's numbering.

Q1. Flow formulation and novelty

“UFM is a flow model in name only.” “So, what's new?”

We respectfully disagree with the reviewer on this point. UFM retains a flow-parameterized latent update and uses unrolling of the latent states during training. We view this training strategy as an important contribution, as it enables stronger reasoning capabilities that scale with the number of integration steps, while using a flow-based architecture. Given the growing interest in flow models for language, we believe UFM offers a promising path toward extending their reasoning capabilities.

Building on the prompt-caching construction of Zhu et al., our theoretical contribution gives an explicit linear-attention reachability flow with a closed-form solution and an analysis of its finite Euler updates.

To clarify the role of latent-state unrolling during training, we have added an ablation to the revised manuscript comparing backpropagation through one versus six updates under the reference puzzle training recipe. We hold the architecture and all 24 forward updates fixed, varying only the gradient horizon. The six-update condition backpropagates through the final six calls.

The Sudoku-Hard test results across three training runs, each trained for 150 epochs, are:

Backpropagated updatesTest accuracy (%)
694.14 ± 2.02
12.09 ± 1.89

The table shows the mean and sample standard deviation across training runs. Each run's score averages five inference-noise seeds at 128 updates and one rollout. These results further underline the importance of backpropagating through several unrolled latent states under the reference update and training budget.

AuthorsOpenReview ↗

Response to Reviewer aQQs — Part 2/2

Q2. Answer-informed initialization

“This introduces privileged answer information into training”

To investigate answer-informed initialization, we trained a pure-Gaussian, t0=0t_0=0 configuration using the original scheduled update. It reaches 99.50% on Sudoku-Hard and 59.94% on the 2,000-puzzle Sudoku-Extreme subset. This demonstrates successful learning without an answer-derived starting state in that configuration. Fixing t0t_0 also changes the training rollout interval, so the comparison measures these choices together.

We also checked validation during training and final inference. Both start from pure noise, with answer labels excluded from the model inputs. Our split checks found no overlap for identical puzzles or solutions, Sudoku digit relabelling, or the eight Maze rotations and reflections. Sudoku row, column, and band permutations were outside these checks.

Q3. Component ablations

“No component is properly ablated.”

Following the reviewers' suggestions, we added controls for the gradient horizon, retraction, initialization, and state representation. The Sudoku-Hard controls use the separate 150-epoch budget. The gradient-horizon comparison keeps the backbone, decoder, training data, and all 24 forward updates fixed.

The table below reports single-training-run comparisons. Values are means and standard deviations over five inference-noise draws, using 128 uniform updates and one rollout.

ConditionSudoku-Hard (%)Maze-Hard (%)
Scheduled reference96.09 ± 0.2589.34 ± 0.21
One-update backpropagation3.66 ± 0.5570.38 ± 0.71

To examine the remaining components, we also trained separate Sudoku-Hard controls without retraction, with pure-noise initialization at t0=0t_0=0, and with a vocabulary-sized carried state. Using the same evaluation protocol, these configurations reach 97.53 ± 0.17%, 99.50 ± 0.07%, and 85.47 ± 0.51%, respectively. Values are means and standard deviations over five inference-noise draws, with one training run per condition.

The gradient-horizon comparison was also repeated across three independent Sudoku-Hard training runs. The scheduled reference reaches 94.14 ± 2.02% with six-update backpropagation and 2.09 ± 1.89% with one-update backpropagation. These values are means and sample standard deviations across training runs, with each run's score averaging five inference-noise draws.

The initialization and representation controls require additional qualification. Fixing t0=0t_0=0 removes the answer-derived initial component and changes the training rollout interval. The vocabulary-state variant also changes the input mapping, decoder, and noise distribution. We therefore report these as combined interventions, without attributing their effects to a single underlying factor.

For Maze-Hard, the reported checkpoints are selected on a separate generated validation set at 64 updates with one noise draw. Reporting uses the official released test set.

These ablations show a substantial benefit from longer gradients in the scheduled puzzle recipe. They also show that high Sudoku-Hard accuracy remains possible without retraction or answer-derived initialization. Detailed configurations and results are reported in the appendix under Additional controlled experiments.

We sincerely hope that we have addressed the concerns of the reviewer satisfactorily in the revised version and would kindly ask the reviewer to update their score accordingly.

Thank the authors for the response and additional experiments. My concerns are addressed, and I have updated the score to 3.

AuthorsOpenReview ↗

Thanking the Reviewer

We sincerely thank the reviewer for your positive assessment of our paper and for appreciating our additional results. We are glad we could address all of your concerns.

Reviewer_D8HAOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether additional flow integration steps can improve reasoning. It proposes Unrolled Flow Models (UFM), trained through their own latent trajectories using a loss on the final answer. Sphere retraction stabilizes longer trajectories, and truncated backpropagation reduces training memory. A graph-reachability construction motivates the approach. Experiments on ProsQA, Sudoku, and Maze show substantial improvements over the evaluated standard flow baselines and benefits from additional inference steps.

Critical strengths
  • Allowing more inference steps does not necessarily mean a model can use them effectively. The step-count experiments directly examine this distinction and show a convincing difference between UFM and the evaluated flow baselines.
  • On Sudoku-Hard, UFM achieves considerably higher accuracy with fewer parameters and less reported training time. This suggests that the proposed training recipe has practical value.
Critical weaknesses
  • The evaluation leaves some uncertainty about robustness and generalization. Maze-Hard uses the test set for checkpoint selection, and the reported error bars primarily measure inference noise rather than variation across independent training runs. The experiments also remain within structured tasks; notably, the ProsQA evaluation does not test longer reasoning paths than those included in training. These limitations make it difficult to assess how broadly the observed benefits extend.

  • UFM changes the training objective, intermediate representation, decoding strategy, and stabilization procedure. The comparisons therefore establish the effectiveness of the overall recipe more clearly than the specific benefit of backpropagating through multiple updates. A controlled comparison using the same architecture would strengthen the main claim. The advantage over closely related recurrent reasoning approaches is also less clear than the advantage over standard flow baselines.

Questions for authors
  • Do you have a matched ablation that keeps the architecture and training settings fixed while comparing backpropagation through one versus several rollout steps?

  • How does UFM compare with a closely related recurrent reasoner under the same data, training budget, and inference budget?

3 comments
AuthorsOpenReview ↗

Response to Reviewer D8HA

We thank the reviewer for the thoughtful review and for highlighting the step-scaling results and practical gains. We address your questions with the additional controls below.

We have uploaded a revised version of the paper, with all changes highlighted in blue. All references to pages, sections, figures, tables, and equations use the revised version's numbering.

Q1. Gradient-horizon comparison

“Do you have a matched ablation that keeps the architecture and training settings fixed while comparing backpropagation through one versus several rollout steps?”

We thank the reviewer for this comment. To clarify the role of latent-state unrolling during training, we have added an ablation to the revised manuscript comparing backpropagation through one versus six updates under the reference puzzle training recipe. We hold the architecture and all 24 forward updates fixed, varying only the gradient horizon. We verified that the one-update condition includes only the final network call in the computation graph for backpropagation and that all 22 parameter tensors receive nonzero gradients. The six-update condition backpropagates through the final six calls.

The Sudoku-Hard test results across three training runs, each trained for 150 epochs, are:

Backpropagated updatesTest accuracy (%)
694.14 ± 2.02
12.09 ± 1.89

The table shows the mean and sample standard deviation across training runs. Each run's score averages five inference-noise seeds at 128 updates and one rollout. These results further underline the importance of backpropagating through several unrolled latent states under the reference update and training budget.

Q2. Comparison with recurrent reasoners

“How does UFM compare with a closely related recurrent reasoner under the same data, training budget, and inference budget?”

As discussed in our paper, the evaluated flow models remain below the strongest reported recursive reasoners on Sudoku-Extreme. The aim of our paper is to close this gap as much as possible.

To do that, we have identified unrolling of the latent states during training, while retaining a flow-parameterized latent update. We view this training strategy as an important contribution, as it enables stronger reasoning capabilities that scale with the number of integration steps, while using a flow-based architecture. Given the growing interest in flow models for language, we believe UFM offers a promising path toward extending their reasoning capabilities.

Q3. Longer reasoning paths

“the ProsQA evaluation does not test longer reasoning paths than those included in training.”

We thank the reviewer for this very interesting comment. To assess generalization beyond the training hop range, we trained on three- and four-hop ProsQA questions and evaluated five- and six-hop questions using five inference updates. The five- and six-hop sets contain 2,646 and 579 questions and are used only for evaluation.

For the reference model accuracy with five inference updates is 93.3% on the combined in-range three- and four-hop questions, 90.1% on five-hop questions, and 87.9% on six-hop questions. These scores describe one training run.

These experiments extend evaluation beyond the training hop range.

We sincerely hope that we have addressed the concerns of the reviewer satisfactorily in the revised version and would kindly ask the reviewer to update their score accordingly.

Thanks the author for the additional results which addresses my concern, I will update my score to 3 and I'm also lower my confidence to 1 as I think I may not fully understand all the mechanism of the method after reread the paper and the rebuttal.

AuthorsOpenReview ↗

Thanking the Reviewer

We sincerely thank the reviewer for your positive assessment of our paper and for appreciating our rebuttal. We are glad we could address all of your concerns.

Reviewer_oMHzOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether extra flow-integration steps can provide useful reasoning computation. It constructs a two-layer architecture with a linear-attention propagation operator whose exact flow solves graph reachability; its Euler discretization propagates support one hop per step. Unrolled Flow Models (UFM) train through model-generated latent rollouts with a terminal answer loss, using truncated backpropagation for longer tasks and sphere retraction for stability. Experiments on ProsQA, Sudoku, and Maze show stronger step scaling and higher accuracy than the evaluated FLM/S-FLM baselines. On Sudoku-Hard, UFM reports 86.9% with 8.4M parameters versus 51.9%/50.9% for 28.6M baselines. Multiple-rollout selection is also studied.

Critical strengths
  1. The conceptual distinction between integration accuracy and computational depth is made concrete. The reachability construction explicitly separates the exact flow, which has positive mass at all reachable vertices for any positive time, from the finite Euler rollout, whose support expands by graph distance. This is a useful existence result, and the paper states the orthonormal-embedding, linear-attention, and candidate-restricted-readout assumptions rather than claiming that trained Transformers necessarily implement it.

  2. The empirical improvement over standard flow training is substantial. ProsQA step scaling and the large Sudoku/Maze gaps support the premise that training on one's own rollout can make extra steps useful. The Sudoku-Hard comparison is especially persuasive because UFM uses a separate validation split while baseline checkpoints are selected on the reporting set, a choice favoring the baselines. The full 422,786-puzzle Sudoku-Extreme single-rollout evaluation and explicit separation of selected accuracy from Pass@100 further strengthen the reporting.

Critical weaknesses
  1. The distinct role of flow-specific design is not cleanly isolated from recurrent training and other changes. Relative to the flow baselines, UFM changes latent parameterization, decoder placement, training-state distribution, terminal backpropagation, and, on long tasks, normalization and loss details. Appendix D.5 states that the main long-task checkpoints disable explicit time conditioning. A same-backbone recurrent-update control and an objective-only ablation are therefore important to determine whether the benefit comes from the proposed flow formulation or primarily from recurrent latent refinement. The retraction ablation uses a different recipe and best recorded test accuracy (Appendix B), so it is suggestive rather than a clean explanation of the main results. Published FRM/recursive-reasoner rows are useful context but not matched comparisons.

  2. Some reported generalization and uncertainty are weaker than the headline task results suggest. Maze-Hard checkpoints are selected on the test set for all methods (Section 5.1), and the reported five-seed deviations measure initialization noise at inference rather than independent training runs. Thus, the Maze result and training stability need confirmation on untouched validation/test partitions and across training seeds. K=100 with 128 steps also entails much more inference computation than a single rollout; parameter count alone does not establish compute efficiency relative to recursive alternatives.

Questions for authors
  1. Attribution request: With backbone, training compute, state dimension, and final decoder held fixed, how do terminal-rollout training and a plain recurrent refinement control compare? Which gains persist when only the supervision objective is changed?
  2. Generalization request: Does the Maze-Hard result persist when checkpoint and hyperparameter selection use a disjoint validation set? What variability is observed across independent training runs, separately from noise-seed variability?
  3. Compute clarification: Can you report accuracy against inference FLOPs or wall-clock latency for single and multiple rollouts, including the chooser, and a matched comparison with a contemporary recurrent/flow reasoner? This would clarify the practical value beyond beating independently supervised FLM baselines.
2 comments
AuthorsOpenReview ↗

Response to Reviewer oMHz — Part 1/2

We thank the reviewer for the careful reading and for recognizing the value of our theoretical construction and empirical evaluation. We address your questions about the individual design choices below.

We have uploaded a revised version of the paper, with all changes highlighted in blue. All references to pages, sections, figures, tables, and equations use the revised version's numbering.

Q1. Attribution and recurrent controls

“With backbone, training compute, state dimension, and final decoder held fixed, how do terminal-rollout training and a plain recurrent refinement control compare? Which gains persist when only the supervision objective is changed?”

As acknowledged in our paper, the evaluated flow models remain below the strongest reported recursive reasoners on Sudoku-Extreme. The aim of our paper is thus to close this gap as much as possible.

For our new Sudoku-Hard controls, we use a 150-epoch training budget and evaluate each checkpoint at 128 updates with one rollout.

Across three training runs on Sudoku-Hard, the reference reaches 94.14 ± 2.02% and direct recurrence 99.75 ± 0.09%. These values give the mean and sample standard deviation across training runs.

The gradient-horizon controls clarify the interpretation. Reducing backpropagation from six updates to one changes the reference from 94.14% to 2.09% . The one-update condition still trains on states produced by preceding model updates. These results separate exposure to model-generated states from credit assignment through several updates. Longer gradients are particularly valuable for the scheduled update under this recipe and training budget.

The reference uses a retracted update base. All variants read the raw latent state and return the predicted endpoint at the final step. Appendix C provides the full training and inference algorithms.

We also checked readout-weight initialization. The default readout is zero-initialized, and direct recurrence uses truncated LeCun-normal initialization. A reference with that same initialization reaches 92.03 ± 0.30%, compared with 99.83% for the reported direct-recurrence run. The control's error bar measures inference-noise variability.

Under a uniform inference grid, the 2,000-puzzle Sudoku-Extreme comparison gives:

CheckpointAccuracy (%)
Direct recurrence81.93 ± 0.18
Reference74.24 ± 0.37

Values are means and standard deviations across five inference-noise seeds for each fixed checkpoint, with 128 updates and one rollout.

These update controls share the endpoint training objective. Attribution to the choice of supervision objective alone remains open.

The validation-selected Maze reference and direct-recurrence checkpoints reach 89.34% and 89.62% on the released test set, respectively.

AuthorsOpenReview ↗

Response to Reviewer oMHz — Part 2/2

Q2. Validation-based Maze selection and variability

“Does the Maze-Hard result persist when checkpoint and hyperparameter selection use a disjoint validation set? What variability is observed across independent training runs, separately from noise-seed variability?”

For the new Maze runs, checkpoints are selected using exact-match accuracy on a disjoint generated validation set at 64 updates. The selected checkpoints are then evaluated on the official released test set at 128 updates with one rollout and five inference-noise seeds.

CheckpointReleased test (%)
Reference89.34 ± 0.21
One-update backpropagation70.38 ± 0.71

The error bars measure inference noise at fixed checkpoints, with one training run per condition.

FLM and S-FLM have not been evaluated under this new selection protocol, so these results compare our update and gradient-horizon controls.

The Sudoku-Hard results in Part 1 are now evaluated across three independent training runs on the test set. Their error bars describe training-run variability, separately from inference-noise variability.

Q3. Inference cost and comparison with flow baselines

“Can you report accuracy against inference FLOPs or wall-clock latency for single and multiple rollouts, including the chooser, and a matched comparison with a contemporary recurrent/flow reasoner?”

To assess inference cost within the flow-model setting studied in our paper, we benchmarked UFM against FLM and S-FLM on one A100-SXM4-40GB. Each method uses its native evaluation settings, 128 network evaluations, and one rollout.

The checkpoint associated with the original 86.9% result has not been recovered, so this comparison uses the available 105-epoch UFM reproduction.

MethodAccuracy (%)Batch 1 latency (ms)Batch 8 time per puzzle (ms)
UFM (reproduction)81.11330.841.2
FLM51.991478.6185.9
S-FLM50.581509.1191.1

Accuracies average five inference-noise draws for one checkpoint per method. Batch-8 values are median batch times divided by eight.

Under these settings, UFM is approximately 4.5 times faster at both batch sizes. At 16 updates and batch size 8, it reaches 62.46% at 5.2 ms per puzzle, exceeding the baseline accuracies at every tested update count from 16 to 128.

The 100-rollout benchmark at 128 updates takes 5.10 seconds per batch of eight puzzles, or approximately 0.64 seconds per puzzle. Selected-answer accuracy for this checkpoint was not measured, and multiple-rollout baseline selection was not timed.

Timings include conditioning, sampling, and decoding with inputs already on the GPU. Data loading and accuracy scoring are excluded. We report medians over 20 synchronized batches after five warm-ups. These are comparisons of the native implementations, whose model sizes, sequence lengths, and precision settings are not controlled to be identical.

We thank the reviewer again for the constructive suggestions and hope these clarifications are helpful.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.