- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper uses a recurrent self conditioning technique in order to improve performance in reasoning tasks such as sudoku, zebra or maze. In their sampling algorithm, using a fixed time point with only iteration noising and denoising improve performance drastically. They also use a stopping criterion which uses the differences of the probabilities between updates. They claim that their algorithm promotes a fixed point iteration and show empirically, that when the change of probabilities is low, the chance of predicting the correct token is high.
Critical strengths
The idea is interesting, addresses an important problem, and improves performance compared to the vanilla algorithm strongly. There are many informative experiments presented which illustrate their claims.
Critical weaknesses
- There is a bit of theory missing, which would motivate further, why the algorithm should be "fixed point forcing".
- Their performance gains are mainly only existent, when they use fixed time sampling with stochastic renoising. Although this is mentioned in the main text, the reasons for that are underexplored.
- some of their contribution claims are a bit too strong for the given evidence.
Questions for authors
- Figure 6c already gives a hint on that correct predictions may be critical points. Did you do/can you do experiments using your trained models, that really show empirically, that correct predictions are attractive critical points and that incorrect predictions are instable? Currently Figure 1A is a strong claim, which should be backed by either more precise experiments or theory.
- The best performing sampling method is for fixed time and with stochastic renoising. How much does this contribute to escaping unstable predictions?
- Is there some reason why the fixed time is so important for the sampling performance?
- The sampling procedures should be specified more explicitely in the appendix, especially given that no code was provided.
- In the appendix A2, how long was the pure self conditioning model trained? only 100k, or did it match the 275k of the FRM?
- It is reported, that there was additional training just for the time point which was also used for fixed time sampling. How good is the performance for fixed point sampling without this additional training?
- In section 3.1 the training/samping mismatch for are discussed. It should be made more clear that only the mismatch is addressed later, since for the training update, still is used if i understand correctly.
- Where in the paper is the claim "We discover that exposure bias can cause these models to converge to spurious fixed points." addressed exactly?
- it should be made clear that the claim "FPF mitigates this mismatch by training on rollout-derived conditioning states, making additional recurrent depth productive and convergence predictive of correctness." is backed empirically, not theoretically.
- in general the name "Fixed-Point Forcing" is a bit strong without further evidence.
- it should be noted more clearly in the main text, that the performance depends a lot on the chosen fixed time , and that the model had substantially extra training for that time. The best would be to report also performance without extra training for that time point. In general I find the paper interesting, and I'm of course willing to raise my score if my questions are sufficiently addressed.
7 comments
B. The role of rollout-derived carries, and the name
There is a bit of theory missing, which would motivate further, why the algorithm should be "fixed point forcing". [...] in general the name "Fixed-Point Forcing" is a bit strong without further evidence.
The name describes the behavior the objective encourages. For each sampled rollout carry, the supervised target is the same puzzle solution . This trains the model to correct erroneous predictions and preserve correct ones on the sampled input–carry pairs. Figure 6 tests whether that behavior persists under repeated denoising and re-noising.
In section 3.1 the training/samping mismatch for are discussed. It should be made more clear that only the mismatch is addressed later, since for the training update, still is used if i understand correctly.
Yes. FPF targets the carry-side mismatch. At each supervised step, the noisy input still comes from the ground-truth interpolant, while the carry comes from a detached model rollout. Training therefore exposes the denoiser to errors that develop over multiple recurrent steps.
it should be made clear that the claim "FPF mitigates this mismatch by training on rollout-derived conditioning states, making additional recurrent depth productive and convergence predictive of correctness." is backed empirically, not theoretically.
We revised the introduction and Sec. 4.3 to describe this as an empirical interpretation.
C. Samplers and held time
Their performance gains are mainly only existent, when they use fixed time sampling with stochastic renoising. [...]
We realize the original submission did not make this clear. FPF's gains hold under Euler, annealed SDE and fixed-time sampling with fresh noise, on every task: FPF reaches 49.5% with Euler and 90.1% with annealed SDE, compared with 40.7% for the strongest baseline at its best sampler, and fixed-time sampling saturates Sudoku-Extreme (App. Table 5).
[...] How much does this contribute to escaping unstable predictions?
Adding fresh noise at each step is essential. At held and the same 8,192 model evaluations, FPF solves 7.6% of Sudoku puzzles when it reuses one noise draw and 99.7% when it draws fresh noise at each step (Table 5).
Is there some reason why the fixed time is so important for the sampling performance?
Holding the noise level at an intermediate value, instead of annealing it to zero, lets every update still revise the prediction. The intermediate Sudoku optimum is consistent with retaining enough of the current grid to refine it while injecting enough noise to revise errors. Additional updates therefore keep helping as long as the correct solution is the attracting state, which is what FPF provides (A above).
it should be noted more clearly in the main text, that the performance depends a lot on the chosen fixed time , and that the model had substantially extra training for that time. [...]
The extra training is not needed for accuracy (D below). Performance does depend on the held time for Sudoku-Extreme, which works best in a band around , while Zebra and Maze-Unique are robust over a broad range (App. B.2, Fig. 8).
The sampling procedures should be specified more explicitely in the appendix, especially given that no code was provided.
We now give the update as Eq. 4 in the main text, with an algorithm block and per-task hyperparameters in App. B.
D. Comparisons at matched optimizer steps
In the appendix A2, how long was the pure self conditioning model trained? only 100k, or did it match the 275k of the FRM?
Originally 100K steps. We now train every method from scratch for 275K steps and report final checkpoints: FPF still reaches 99.7%, versus 40.7% for the base flow and 33.7% for self-conditioning (App. Table 5). This matches optimizer steps across recipes. FPF uses additional rollout evaluations and, by default, additional supervised predictions.
[...] How good is the performance for fixed point sampling without this additional training?
Without Stage C, fixed-time sampling solves 99.7% of Sudoku-Extreme puzzles in the single-trajectory evaluation, compared with 99.5% with Stage C at matched optimizer steps. On Sudoku-Lite with restarts, both recipes reach 99.89%, while Stage C reduces mean inference compute from 2.52 to 0.83 TFLOPs per puzzle (App. A.2).
I thank the author for their answers and think that the paper improved, especially Figure 6 is helpful. I have some additional questions:
- In table 6, for the fixed time t training, do you still sample uniformely? so the network technically sees also earlier time steps?
- Using only a fixed time point for sampling does only work, if for every clue there is a unique solution, is that right? If yes that should be made clear.
- Are the converging to a fixed point arguments also true for multiple possible solutions?
We thank the reviewers for their thoughtful feedback, and are glad they recognize that FRMs deliver:
In response to your specific suggestions, we made these core clarifications:
We hope that if our clarifications resolve the reviewers primary concerns that they would consider raising their score.
A. Testing the fixed-point attractor picture
We agree that this perspective should be backed directly, and we now test it empirically from two directions: whether wrong and correct states attract the model back after perturbation (Fig. 6a), and whether the model converges on them (Fig. 6b, c).
Yes. On the same perturbed starting grids, self-conditioning usually recovers its original wrong answer, while FPF usually recovers the correct answer.
To test the behavior depicted in Fig. 1A, we perturbed wrong Sudoku grids on which self-conditioning had converged, then asked both models to re-solve the same puzzles (Sec. 4.3, Fig. 6a). At perturbation strength 0.3, self-conditioning returned to its original wrong grid 96.5% of the time. FPF returned to that grid at most 0.1% of the time and recovered the correct solution about 95% of the time. Starting from correct solutions, FPF returns to them 95% of the time even at strength 0.9, compared with 28% for self-conditioning.
We revised the manuscript to highlight this result: Sec. 4.3 and the new Fig. 6b,c document the spurious convergence, and the introduction's claim now points there. We measure convergence by thresholding the mean change between consecutive predictions over the final 64 steps. On 1,024 Sudoku-Extreme puzzles, 81% of the solutions meeting this criterion under conventional self-conditioning were wrong. Under FPF, every solution meeting it was correct.
Our evidence for the role of exposure bias comes from changing which carries the model sees during training. With supervision held at one state per rollout, increasing training rollout length from one to eight steps raises solve rate from 7.2% to 98.0% (App. Fig. 9). This supports our interpretation that training on one-pass carries leaves the model poorly prepared to correct errors along its recurrent trajectories.