acceptodds
Under review as a conference paper at ICLR 2027

Self-Conditioning Enables Deep Sequential Reasoning in Flow Matching

Abstract

Recent work has explored visual reasoning through image generation, where models solve visual-reasoning problems by directly generating solution images. Intuitively, iterative samplers such as flow matching are well suited to this task, since each sampling step may resemble one reasoning step. However, in this work, from a circuit complexity perspective, we show that this intuition is misleading. We prove that as long as the model outputs satisfy certain mild regularity conditions, the trajectory of the vanilla flow-matching sampler can be simulated by a constant number of sequential rounds of parallel model evaluations. Consequently, additional sampling steps cannot increase the expressive power of a constant-depth backbone model under the standard complexity-theoretic model of transformers. We then reveal that self-conditioning, which conditions the model on the previous step's estimate of the final image, plays a previously unrecognized role in removing this bottleneck. Controlled experiments on diverse visual-reasoning tasks show that flow matching with self-conditioning achieves higher accuracy and greater reasoning depth than vanilla flow matching, and its performance improves with more sampling steps. Theoretically, we prove that with self-conditioning, a constant-depth backbone can achieve expressive power equivalent to that of space-bounded Turing machines. Our study suggests a design principle for future visual reasoning research: carry more information between sampling steps, so that additional steps translate into deeper reasoning.

Primary area:
generative models
Keywords:
flow matching, self-conditioning, circuit complexity, expressive power, visual reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_7Mx9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper analyzes sequential computation in flow-matching samplers. Under boundedness, Lipschitz, precision, and acceptance-margin assumptions, it simulates vanilla sampling with a constant number of parallel evaluation rounds. Feeding endpoint predictions back into subsequent calls removes this restriction in a constructive expressivity result. Experiments compare vanilla and self-conditioned models on synthetic tasks and structured visual reasoning.

Critical strengths

The paper gives a substantive computational interpretation of self-conditioning. An explicit prediction state can change substantially between calls, whereas the ordinary sampler state evolves through small integration increments. The Picard simulation and Turing-machine construction formalize this distinction under stated assumptions. This contributes a useful theoretical perspective on an existing mechanism.

The controlled empirical gains are substantial and account for important compute differences. The five-task average rises from 65.9% to 76.7%; three-seed line-tracing and multiplication results support the trend. Training comparisons account for the extra self-conditioning pass and longer inputs, and the appendix includes inference-FLOP comparisons (Tables 1–2; Appendices D.1–D.3). This makes the practical contribution credible beyond equal-NFE comparisons.

Critical weaknesses

The upper bound does not quantitatively explain saturation at the tested small sampling budgets. Lemma 1 requires O(L) parallel rounds, while empirical Lipschitz estimates are around 400, reaching 580, and are local search estimates rather than certified global bounds. The asymptotic result remains interesting, but its constants and assumptions limit its use as a finite-budget explanation. This distinction should be maintained consistently in the introductory and concluding claims.

The connection between complexity classes and learned behavior needs a narrower interpretation. Long multiplication and text rendering are TC0-complete, so success on them does not demonstrate computation beyond a TC0 upper bound. The relevant TC0 separation for line tracing is conjectural, and finite-size accuracy does not establish asymptotic separation. Likewise, an existence construction for self-conditioned models does not establish that training learns that mechanism. These qualifications affect the explanatory claims, although they do not negate the empirical advantage.

Questions for authors

Given the measured magnitude of L, what finite-budget prediction follows from the theorem, if any? Can the distinction between an asymptotic expressivity result and an explanation of low-NFE saturation be made explicit?

Which task results are intended to test the AC0 barrier and which the conjectured TC0 barrier, given the actual backbone implementations? Please avoid treating TC0-complete arithmetic or text rendering as evidence of surpassing TC0.

Do the trained models retain the advantage on held-out longer line-tracing instances? If existing results only cover m = 50, please state that scope explicitly rather than equating interpolation accuracy with scaling of sequential complexity.

1 comment
AuthorsOpenReview ↗

Response to 7Mx9

Thank you for your careful reading and constructive feedback. Below we address your concerns point by point.

W1&Q1 (Asymptotic upper bound vs. low-NFE saturation) Thanks. Regarding local search estimates versus certified global bounds, we acknowledge this limitation in Section 6 (lines 501–502). Meanwhile, simulating a trajectory x0,…,xKx_0, \dots, x_K in our theory only requires Lipschitzness with respect to the trajectory: ∥u(xi′,ti;w)−u(xi,ti;w)∥∞≤L∥xi′−xi∥∞\Vert u(x_i', t_i; w) - u(x_i, t_i; w)\Vert _ \infty \le L \Vert x_i' - x_i\Vert _ \infty for all xi′∈Fbnx_i' \in F_b^n, or more specifically, only at the Picard iterates xi′=yi(r)x_i' = y_i^{(r)}. We state global Lipschitzness in Theorem 1 for conciseness. Probing LL across multiple scales around the real trajectories in Appendix D.5.2 thus provides a useful estimate of this requirement (Appendix A.2.4, lines 1247–1255).

Regarding Q1, Theorem 1 bounds worst-case error accumulation, which rarely occurs in trained models. Thus, L≈400L\approx 400 supports the asymptotic separation (as LL levels off with mm) rather than yielding a finite-budget prediction for VanillaFM's low-NFE saturation. We have made this distinction between our asymptotic expressivity result and low-NFE saturation explicit in Section 6 (lines 502–506) and Appendix D.5.2, and maintained it in our introductory and concluding claims in Section 1 (line 041) and Section 7 (line 515).

Moreover, in an empirical Picard simulation on line tracing, the Lipschitz constant between the Picard iterates and the trajectory, LPicard:=max⁡i,r∥u(yi(r),ti;w)−u(xi,ti;w)∥∞/∥yi(r)−xi∥∞L _ {\mathrm{Picard}} := \max _ {i,r} \Vert u(y_i^{(r)}, t_i; w) - u(x_i, t_i; w)\Vert _ \infty / \Vert y_i^{(r)} - x_i\Vert _ \infty, is only 17.9, and 12 rounds suffice to reproduce the line coloring of 100-NFE VanillaFM. Taken together, LPicard≈18L _ {\mathrm{Picard}}\approx 18, the 12 Picard rounds needed to simulate VanillaFM, and VanillaFM's early saturation (around NFE=6 with an LCP of ~8) are all of a similar order of magnitude.

W2 (Narrower interpretation of complexity classes and learned behavior) Thanks. We agree with these qualifications: the TC⁰ versus NC¹ separation remains conjectural (Section 3.3, lines 297–298), finite-size accuracy provides empirical support rather than an asymptotic proof, and Theorem 2 is an existential construction rather than a guarantee on what gradient training learns. We have made these points explicit in Section 4.2 (lines 342–347) and Section 6 (lines 508–510).

Q2 (Correspondence between task complexity, backbones, and learned behavior) Thanks. Multiplication and text rendering are TC⁰-complete and test the AC⁰ barrier, whereas line tracing is NC¹-complete and tests the conjectured TC⁰ barrier. We do not treat the TC⁰-complete tasks as evidence of surpassing TC⁰, and we have clarified this in Section 4.2 (lines 342–347) and Appendix D.1 (lines 2172–2189).

Since practical implementations use fixed hardware precision (in our experiments, BF16 activations and FP32 accumulation/reduction) on finite problem sizes, it is hard to say whether this precision should be treated as constant or logarithmic relative to the problem size, and thus whether a practical backbone strictly corresponds to AC⁰ (constant precision) or TC⁰ (logarithmic precision). Instead, our theoretical result serves as a mathematical modeling of practical flow matching. This modeling yields useful insights: it suggests that in many cases, SelfCondFM can enable deeper sequential reasoning than VanillaFM. We use this insight to guide our experimental design and empirically validate this advantage.

Prior works have similarly modeled constant-depth Transformers as both AC⁰ (Li et al., 2024; Svete et al., 2026) and TC⁰ (Merrill & Sabharwal, 2023; Chiang, 2025; Chen et al., 2025). In general, lower precision is regarded as closer to AC⁰ and higher precision as closer to TC⁰. This modeling has yielded useful conclusions, such as Chain-of-Thought enabling constant-depth Transformers to solve inherently serial problems (Feng et al., 2023; Merrill & Sabharwal, 2024; Li et al., 2024). Following Li et al. (2024), our experiments therefore include tasks for both barriers.

Q3 (Held-out longer line-tracing instances) Thanks. In Figure 3 of Section 4.2, we intended to test the in-distribution capability of the two models at m=50 by varying NFE, rather than OOD generalization to longer instances. For NFE > 10, the flat curve reflects the m=50 instance ceiling, and we have explicitly stated this m=50 scope in Section 4.2 (lines 371–387).

Nevertheless, we also added an evaluation on held-out longer line-tracing instances in Section 4.2 (lines 387–390) and Appendix D.3 (Figure 12 of the latest version). Although trained only on m=50, SelfCondFM retains its advantage and generalizes to a mean LCP of ~65 when evaluated on m>50, which demonstrates an interesting OOD scaling capability of SelfCondFM.

Thank you again for your thoughtful feedback. We look forward to our further discussions!

Reviewer_MYQ6OpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how self conditioning can support sequential reasoning in flow matching models by feeding the previous prediction of the final image into the next sampling step. Its theoretical analysis shows that, under some assumptions, regular flow matching has limits on the computational expressiveness that cannot be overcome by more sampling steps, but adding self conditioning can overcome these limits. Experiments show improvements on line tracing, multiplication, text rendering, and several visual reasoning tasks.

Critical strengths
  1. The paper asks an important question: does running a generative model for more steps actually allow it to do more reasoning? The theoretical analysis gives a useful explanation of when this may be limited and how keeping previous predictions can change it.
  2. The approach is simple and shows clear improvements across several tasks. The comparisons with matched training compute are useful, and make the results more convincing than simply giving the proposed method more training.
Critical weaknesses
  1. Theoretical analysis soundness. But firstly, I want to mention that I have limited knowledge in this field so could be wrong: The authors estimate the model’s sensitivity by perturbing sampled inputs. However, it requires a bound that holds for all inputs, which these experiments do not establish. The paper acknowledges this, but the broad claim that regular flow matching cannot gain deeper reasoning from more steps still seems stronger than what the evidence supports now.
  2. The results show that providing the prediction inputs helps, but it is less clear whether the gain comes from keeping useful computation across steps. Matching compute is useful, but does not fully answer this. A comparison that recomputes a fresh draft from the current noisy input at each step, instead of keeping the previous draft, would help. Also, another test is to eval how the required sampling steps change with task length.
Questions for authors

Please see the weakness section.

3 comments
AuthorsOpenReview ↗

Response to MYQ6

Thank you for your constructive feedback. We address your concerns point by point.

W1 (Empirical Lipschitz estimation & scope of claim) Thanks. We acknowledge that our Lipschitz estimation is a local empirical estimate rather than a certified global upper bound, and we have noted this in Section 6 (lines 501–502) in the latest version. Also, to clarify the role of the Lipschitz condition, we remark that a correct simulation of a trajectory x0,…,xKx_0, \dots, x_K in our theory only requires Lipschitzness with respect to the trajectory: ∥u(xi′,ti;w)−u(xi,ti;w)∥∞≤L∥xi′−xi∥∞\Vert u(x_i', t_i; w) - u(x_i, t_i; w)\Vert _ \infty \le L \Vert x_i' - x_i\Vert _ \infty for all xi′∈Fbnx_i' \in F_b^n, or more specifically, it suffices for this bound to hold only at the Picard iterates xi′=yi(r)x_i' = y_i^{(r)}. We state global Lipschitzness in Theorem 1 to keep the theorem statement concise. Since we probe the Lipschitz constant across multiple scales around the real trajectories in Appendix D.5.2, our results provide a useful estimation of this requirement.

In addition, we newly ran an empirical Picard simulation on line tracing, where the Lipschitz constant between the Picard iterates and the trajectory, LPicard:=max⁡i,r∥u(yi(r),ti;w)−u(xi,ti;w)∥∞/∥yi(r)−xi∥∞L _ {\mathrm{Picard}} := \max _ {i,r} \Vert u(y_i^{(r)}, t_i; w) - u(x_i, t_i; w)\Vert _ \infty / \Vert y_i^{(r)} - x_i\Vert _ \infty, is only 17.9, and 12 rounds suffice to produce the same line coloring as the 100-NFE VanillaFM. We have added these clarifications and new results to Appendix A.2.4 (lines 1247–1255) and Appendix D.5.2.

As for the claim that regular flow matching cannot gain deeper reasoning from more steps, we only claimed the case where our stated regularity assumptions hold, such as boundedness, Lipschitzness, and a constant error margin. To avoid ambiguity, we have further refined our wording in Section 1 (line 041) and Section 7 (line 515) to ensure the scope of our conclusions remains clear.

W2 (Fresh draft at each step & required steps vs. task length) Thank you for your suggestions. Using the trained SelfCondFM weights, we have tested two variants of the inference algorithm without cross-step recurrence in Figure 18 of the latest version: (a) No feedback, which omits the self-conditioning input at inference and conditions only on the input image, and (b) Fresh draft, which recomputes a fresh draft from the current noisy input at each step (via the same two-pass procedure as in training) instead of carrying predictions across steps. With No feedback, the model plateaus at an LCP of ~11, performing similarly to VanillaFM (~8) and showing that the training objective alone brings little gain. With Fresh draft, the model achieves an LCP of ~31, similar to 4–5 NFEs of SelfCondFM, whereas keeping the previous draft reaches ~50 in 10 NFEs. This is reasonable considering that VanillaFM is similar to 2 NFEs of SelfCondFM and within-step feedback doubles the single-step computational depth.

This ablation confirms that carrying predictions across steps is essential for the gain, and we have added it to Section 4.2 (lines 392–399) and Appendix D.5.1.

Second, regarding how the required sampling steps change with task length, we would like to point out that our experiments on performance across different NFEs in Figure 3 of Section 4.2 do exactly this: since LCP means the largest subtask length the model successfully solves, flipping the x- and y-axes in Figure 3 from NFE=1 to NFE=10 directly gives the required sampling steps versus the subtask length. We have remarked this in Appendix D.3 (lines 2544–2546).

We sincerely appreciate your constructive feedback. Please let us know if you have further concerns or questions.

Reviewer_MYQ6OpenReview ↗

Review update

Thank you for the additional experiments and clarifications. My main concerns have been addressed, and I will raise my score.

Thank you for the constructive feedback and for raising your score. We are glad our responses addressed your concerns, and the corresponding revisions are reflected in the updated manuscript.

Reviewer_iXPUOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates self-conditioning for visual reasoning in flow-matching models. The central mechanism is to feed the predicted clean latent from the previous diffusion step into the input of the next diffusion step, alongside the current noisy latent and the original condition. This prediction is obtained by extrapolating the noisy latent toward the clean endpoint using the predicted velocity, providing an explicit channel for carrying information across sampling steps. Under boundedness, Lipschitz, and finite-precision assumptions, the authors show that vanilla flow-matching inference remains within the backbone’s circuit complexity class, whereas self-conditioning enables substantially greater sequential computational power. They implement this mechanism through a two-pass training procedure and demonstrate improvements on synthetic and structured visual-reasoning tasks. In particular, self-conditioned models benefit more from additional sampling steps on line tracing.

Critical strengths

Clear theoretical contribution. The parallel Picard argument provides a useful perspective on the difference between repeatedly evaluating a network and performing inherently sequential computation. The treatment of finite-precision errors and approximate prefix sums is particularly valuable: the AC0AC^0 result does not incorrectly rely on exact iterated addition being available in AC0AC^0.

Carefully constrained expressivity construction is provided. The self-conditioning lower bound preserves boundedness and a constant Lipschitz bound, rather than obtaining additional computational power through unbounded velocities or arbitrarily sharp transitions. The extensions to other norms, fixed explicit Runge–Kutta methods, and growing Lipschitz constants strengthen the theoretical analysis.

Consistent empirical improvements with meaningful compute controls. The training-FLOP accounting includes the additional forward pass and longer token sequence. Self-conditioning improves all five tasks in the controlled benchmark comparison, raising average accuracy from 65.9% to 76.7%. The inference-FLOP comparison for line tracing and the multi-seed results for line tracing and multiplication are also useful.

Critical weaknesses

The applicability of the upper-bound assumptions remains insufficiently established. Theorem 1 requires a global Lipschitz bound that is uniform over time and independent of problem size. However, if we deduce from the flow-matching formulation.For a fixed interpolation w, xt=t y(w)+(1−t)ϵx_t=t\,y(w)+(1-t)\epsilon

implies that the ideal velocity field is

=y(w)-\epsilon =\frac{y(w)-x}{1-t},\qquad t<1.$$ Consequently, $$\frac{\partial v^*}{\partial x} =-\frac{I}{1-t}, \qquad L_x(t)=\frac{1}{1-t},$$ which grows without bound as $t\to1$. Thus, even this simple conditional flow does not admit a time-uniform Lipschitz bound in its ideal continuous formulation. This calculation is not a counterexample to Theorem 1, but it shows that the required regularity is not automatically guaranteed by the flow-matching formulation. The authors should explain why their trained models satisfy a sufficiently strong bound, particularly near the endpoint, or clarify how endpoint truncation or regularization would affect the result. **The correspondence between task complexity and the implemented backbones needs clarification.** The $NC^1$-complete line-tracing task addresses the conjectured $TC^0$ barrier. However, multiplication and text rendering are $TC^0$-complete and therefore do not, by themselves, exceed a $TC^0$ backbone’s expressivity. Their interpretation as separation tasks depends on establishing the stronger $AC^0$ characterization for the implemented models. **A few minor formal details.** Section 2.2 should require \$ n \geq 1\$, since the acceptance definition becomes vacuous at $n=0$ and the upper-bound proof reads the first output coordinate. Proposition 6 should also specify how the counting reduction respects the no-leading-zeros convention. Both appear readily repairable and do not undermine the central results.
Questions for authors
  1. Can the authors provide stronger evidence that the relevant trained checkpoints satisfy the regularity regime of Theorem 1, or formulate a weaker condition restricted to the states needed by the simulation? How should the observed L≈400L\approx400 be related to the much earlier empirical NFE saturation?

  2. Which precise arithmetic model and circuit-class characterization apply to each experimental backbone? In particular, how are network precision, accumulation and reduction operations, and the sampler’s precision requirement reconciled?

  3. How sensitive are the results to the mismatch between same-time two-pass training and previous-step recursive conditioning at inference? Would adjacent-time conditioning or a short training unroll change the results?

  4. Can the authors disentangle the contributions of recurrent feedback, the additional conditioning tokens, and the two-pass training objective through compute-matched ablations? In particular, how much of the improvement depends on carrying predictions across sampling steps, rather than on the modified training procedure alone?

  5. Can the authors include a VanillaFM baseline using the same Qwen-Image-Edit-2511 backbone, training data, and compute budget as the large-scale SelfCondFM experiments?

4 comments
AuthorsOpenReview ↗

Response to iXPU

Thank you for your constructive feedback. Below we address your concerns point by point.

W1 (Endpoint regularity) Thank you for your comment. To address this concern, we further extend our theory with endpoint truncation near t=1 so that the Lipschitz condition can be weakened to hold only on t in [0, 1-τ]. This is now included in Section 3.4 (lines 309–310) and Appendix B.4 (Proposition 4). The main idea is to still bound the error on t in [0, 1-τ] by Lemma 2, while bounding the error on t in [1-τ, 1] using only the bounded norm of the velocity.

Q1 (Trajectory-level regularity & NFE saturation) Thanks. For the first subquestion, the response regarding endpoint singularity can be found in W1. Besides, we also remark that a correct simulation of a trajectory x0,…,xKx_0, \dots, x_K in our theory only requires Lipschitzness with respect to the trajectory: ∥u(xi′,ti;w)−u(xi,ti;w)∥∞≤L∥xi′−xi∥∞\Vert u(x_i', t_i; w) - u(x_i, t_i; w)\Vert _ \infty \le L \Vert x_i' - x_i\Vert _ \infty for all xi′∈Fbnx_i' \in F_b^n, or more specifically, it suffices for this bound to hold only at the Picard iterates xi′=yi(r)x_i' = y_i^{(r)}. We state global Lipschitzness in Theorem 1 to keep the theorem statement concise. Since we probe the Lipschitz constant across multiple scales around the real trajectories in Appendix D.5.2, our results provide a useful estimation of this requirement. We have clarified this in Appendix A.2.4 (lines 1247–1255) and Appendix D.5.2.

For the second subquestion, our analyses in Theorem 1 bound worst-case error accumulation, which rarely occurs in actual trained models, so the adversarial estimate L≈400L\approx 400 is used to support the asymptotic separation (as LL levels off with mm) rather than to directly predict the small NFE at which VanillaFM saturates. To further investigate this, we additionally ran an empirical Picard simulation on line tracing in the revised version: the Lipschitz constant between the Picard iterates and the trajectory, LPicard:=max⁡i,r∥u(yi(r),ti;w)−u(xi,ti;w)∥∞/∥yi(r)−xi∥∞L _ {\mathrm{Picard}} := \max _ {i,r} \Vert u(y_i^{(r)}, t_i; w) - u(x_i, t_i; w)\Vert _ \infty / \Vert y_i^{(r)} - x_i\Vert _ \infty, is only 17.9, and 12 rounds suffice to produce the same line coloring as the 100-NFE VanillaFM. Taken together, LPicard≈18L _ {\mathrm{Picard}}\approx 18, the 12 Picard rounds needed to simulate VanillaFM, and VanillaFM's early saturation (around NFE=6 with an LCP of ~8) are all of a similar order of magnitude. We have added these new experimental results and discussions in Section 6 (lines 502–506) and Appendix D.5.2.

W2&Q2 (Task complexity, backbone circuit classes, and arithmetic precision) Thank you for these insightful questions. In our experiments, the backbones use BF16 activations, GPU kernels perform accumulation and reduction operations in FP32, and the sampler computes step updates in FP32 before casting states back to BF16. Since practical implementations use fixed hardware precision on finite problem sizes, it is hard to say whether this precision should be treated as constant or logarithmic relative to the problem size, and thus whether a practical model strictly corresponds to AC⁰ (constant precision) or TC⁰ (logarithmic precision). Instead, our theoretical result serves as a mathematical modeling of practical flow matching, and this theoretical modeling yields useful insights: it suggests that in many cases, SelfCondFM can enable deeper sequential reasoning than VanillaFM. We use this insight to guide our experimental design and empirically validate this sequential reasoning advantage.

Prior works have similarly modeled constant-depth Transformers as both AC⁰ (Li et al., 2024; Svete et al., 2026) and TC⁰ (Merrill & Sabharwal, 2023; Chiang, 2025; Chen et al., 2025). In general, lower precision is regarded as closer to AC⁰ and higher precision as closer to TC⁰. This modeling has yielded useful conclusions, such as Chain-of-Thought enabling constant-depth Transformers to solve inherently serial problems (Feng et al., 2023; Merrill & Sabharwal, 2024; Li et al., 2024). Following Li et al. (2024), our experiments therefore include tasks for both barriers: TC⁰-complete multiplication and text rendering test the AC⁰ barrier, while NC¹-complete line tracing tests the conjectured TC⁰ barrier. We have incorporated this discussion into the latest version in Section 4.2 (lines 342–347) and Appendix D.1 (lines 2172–2189).

W3 (Formal details) Thank you for pointing this out. We have fixed both details in Section 2.2 (line 158) and Appendix C.4 (Proposition 7 in the revised manuscript, formerly Proposition 6).

Q3&Q4 (Train-inference mismatch & compute-matched ablations) Thank you for these insightful questions. We added compute-matched ablations on line tracing to decouple (1) recurrent feedback, (2) additional conditioning tokens, and (3) the two-pass training objective:

(1) Recurrent feedback vs. No recurrent feedback: Using the trained SelfCondFM weights, we test two variants without cross-step recurrence in Figure 18 of the latest version: (a) No feedback, which omits the self-conditioning input at inference and conditions only on the input image, and (b) Fresh draft, which recomputes a fresh draft from the current noisy input at each step (via the same two-pass procedure as in training) instead of carrying predictions across steps. With No feedback, the model plateaus at an LCP of ~11, performing similarly to VanillaFM (~8) and showing that the training objective alone brings little gain. With Fresh draft, the model achieves an LCP of ~31, similar to 4–5 NFEs of SelfCondFM, whereas keeping the previous draft reaches ~50 in 10 NFEs. This is reasonable considering that VanillaFM is similar to 2 NFEs of SelfCondFM and within-step feedback doubles the single-step computational depth.

These ablations confirm that recurrent feedback across steps is crucial, as variants without recurrent feedback perform substantially worse.

(2) Additional conditioning tokens vs. VanillaFM: We feed VanillaFM with additional uninformative conditioning tokens by duplicating the condition image to match the token count of SelfCondFM. As shown in Figure 19 of the latest version, this Extra tokens variant does not boost performance.

(3) Two-pass training objective vs. Short training unroll: To better match previous-step recursive conditioning at inference, we replace the self-conditioning estimate at the same timestep with k=2,3,4 unrolled SelfCondFM inference steps during training (Unroll k). As shown in Figure 19 of the latest version, short-unroll training achieves similar performance to same-time two-pass training, reaching the full LCP of 50.

All added experiments support our theoretical analysis and are included in Section 4.2 (lines 392–406) and Appendix D.5.1. Combined, for Q3, our results are robust to these training and inference variations; for Q4, a large portion of the gain comes from carrying predictions across sampling steps, while single-step feedback itself only doubles the depth of VanillaFM.

Q5 (Large-scale VanillaFM baseline) Thanks. Due to limited budget, we are unable to train another large-scale VanillaFM from scratch. However, we controlled our data distribution to be identical to EndoCoT (Dai et al., 2026), where the 5-epoch DiffThinker model trained in the EndoCoT paper adopts the standard VanillaFM formulation on the same Qwen-Image-Edit-2511 backbone. Since SelfCondFM costs ~2.48x FLOPs per training sample compared to VanillaFM, we evaluated our 2-epoch SelfCondFM checkpoint at matched compute (0.99x FLOPs). As shown below, SelfCondFM still outperforms VanillaFM by large margins:

Method (Unified training)VSP 16VSP 32Maze 8Maze 16Maze 32TSP 12TSP 15TSP 18Sudoku 45Sudoku 40Sudoku 35Sudoku 30Avg.
EndoCoT1008097985264554610088805976.6
DiffThinker (VanillaFM @ 5 epoch)97849899666449349445263766.1
SelfCondFM @ 2 epoch9982100997073735410097885282.3
SelfCondFM @ 5 epoch1008610010096787565100100978590.2

We have added the results to Section 4.3 (lines 458–461) and Appendix D.4 (Table 3).

Thank you again for your thoughtful feedback. Please let us know if you have further concerns or questions.

Reviewer_iXPUOpenReview ↗

Responce to author

Thank you for the detailed response and the additional theoretical and experimental work, and I have no further questions.

I would only encourage the authors to be careful with the wording regarding the connection to practical models. The intuition that lower precision is “closer to” AC0AC^0 and higher precision to TC0TC^0 is useful for modeling, but does not establish a correspondence. Applying the asymptotic results also requires specifying how arithmetic and summation procedures are defined as input size grows, even for fixed-precision hardware.

I appreciate the authors' efforts and will maintain my original positive score of 3.

Thank you for the careful reading and for maintaining your positive assessment. We agree that the analogy between numerical precision and circuit classes serves only as a modeling heuristic. We have removed this wording from the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

A Kind General Response to All Reviewers and AC

We sincerely thank all reviewers for their thoughtful and constructive feedback. We are encouraged that the reviewers recognized the importance of studying whether more sampling steps enable deeper sequential reasoning in flow matching (MYQ6), our clear theoretical contribution via parallel Picard simulation (iXPU, 7Mx9) with rigorous treatment of finite-precision errors and approximate prefix sums (iXPU), our carefully constrained expressivity construction alongside extensions to general norms and Runge-Kutta samplers (iXPU, 7Mx9), and our consistent empirical improvements under controlled compute (iXPU, MYQ6, 7Mx9). These comments capture the central strengths of our work, and we address all concerns carefully below.

In this response, we have strengthened the paper in the following aspects.

(1) Controlled ablations disentangling recurrent feedback, conditioning tokens, and training objectives.

Following suggestions from iXPU and MYQ6, we conducted ablations under matched compute on line tracing to isolate the source of SelfCondFM's gains (iXPU-Q3&Q4, MYQ6-W2). Using trained SelfCondFM weights, omitting self-conditioning at inference plateaus at an LCP of ~11 (similar to VanillaFM's ~8), and recomputing a fresh draft at each step without carrying it across steps plateaus at ~31. In contrast, carrying predictions across steps reaches the full LCP of ~50 in 10 NFEs. Furthermore, giving VanillaFM duplicated condition tokens yields no gain, while training SelfCondFM with a short inference unroll achieves similar performance to our default two-pass training and reaches the full LCP of 50. Together, these results confirm that carrying intermediate predictions across sampling steps is the key driver of the gain, and that our method is robust to training and inference variations.

(2) Large-scale VanillaFM comparison and OOD generalization to longer instances.

Following iXPU-Q5, we added a large-scale comparison against VanillaFM on the Qwen-Image-Edit-2511 backbone. Under matched training compute, SelfCondFM achieves 82.3% average accuracy, substantially outperforming VanillaFM (66.1%). In addition, we evaluated our line-tracing checkpoints trained at m=50 on held-out longer instances (m>50), where SelfCondFM retains its advantage and generalizes zero-shot to a mean LCP of ~65 (7Mx9-Q3).

(3) Extended endpoint regularity theory and clearer complexity positioning.

We extended Theorem 1 with endpoint truncation near t=1, so that the Lipschitz condition can be weakened to hold only on t in [0, 1-τ] (iXPU-W1&Q1). We clarified that our simulation only requires Lipschitzness with respect to the sampling trajectory, and added empirical Picard simulation results on line tracing to better explain VanillaFM's early saturation (iXPU-Q1, MYQ6-W1, 7Mx9-W1&Q1). We also clarified how our practical backbones and task choices relate to theoretical AC⁰ and TC⁰ characterizations (iXPU-W2&Q2, 7Mx9-Q2). Finally, we refined the wording for precision and fixed the technical details pointed out by the reviewers (iXPU-W3, MYQ6-W1, 7Mx9-W2).

Overall, these revisions have further clarified and strengthened our work: showing that self-conditioning enables deeper sequential reasoning in flow matching, grounded in complexity-theoretic modeling and supported by mechanism ablations, large-scale comparisons under matched compute, and various ablation studies. We sincerely appreciate the reviewers' comments, which have helped us sharpen the scope, strengthen the evidence, and improve the presentation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.