acceptodds
Under review as a conference paper at ICLR 2027

Invariant-Measure Reasoners: Stable Representations for Latent Reasoning

Abstract

Latent reasoning models repeatedly update a latent state using the same recurrent block. As the recurrent depth increases, the sequence of latent states may converge to a compact subset of the state space without necessarily converging to a fixed point. Existing models typically predict by applying a prediction head to a single latent state. However, the latent state can continue to change even after many updates, potentially making predictions unstable across recurrent depths. To address this instability, we introduce invariant-measure reasoners (ImR), a framework that uses an invariant measure as a stable representation. This measure describes the long-run distribution of latent states on the compact subset and is invariant under updates by the recurrent block. ImR predicts from the expectation of the prediction head's output under this measure. We use ImR in two ways: fine-tuning only the prediction head of existing models and training models from scratch. Both approaches reduce prediction instability and improve accuracy in many settings on maze and Sudoku tasks. In some settings, models trained with ImR exhibit non-fixed-point behavior more frequently than existing models yet achieve high accuracy even with such behavior, unlike existing models. These results suggest that ImR can leverage otherwise destabilizing dynamics for latent reasoning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
latent reasoning, prediction stability, invariant measures

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_duQ8OpenReview ↗
Rating
2
Confidence
2(+1)
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper proposes Invariant-Measure Reasoners (ImR) to address prediction instability across recurrent depths. It replaces prediction from the final latent state with an average of prediction-head logits along a trajectory, interpreted as an approximation to an expectation under an invariant measure. The authors study prediction-head fine-tuning and training from scratch using EqR and deep thinking networks on maze and Sudoku tasks. The results show improved stability and accuracy in several settings, including accurate predictions from trajectories whose latent states continue to change.

Critical strengths

The paper presents a useful way to obtain stable predictions without requiring individual latent states to converge to a fixed point. The implementation is simple, preserves the architecture, and introduces no additional trainable parameters. Its application to both a linear EqR head and a nonlinear DT head provides evidence beyond a single implementation.

Critical weaknesses
  1. The comparisons do not adequately isolate ImR’s contribution. Missing controls include averaging with the original pretrained head and alternative temporal aggregation methods. The Sudoku comparison without the puzzle prefix also uses different training horizons and gradient budgets. These gaps weaken the attribution of improvements to the proposed method.

  2. The practical significance remains insufficiently established. Evaluation is restricted to structured puzzle tasks, with some gains emerging only after thousands of recurrent updates. Although accuracy-versus-depth curves are provided, the paper does not establish whether these gains translate into a favorable accuracy–runtime tradeoff in a practical use setting.

Questions for authors
  1. How does ImR compare with averaging using the original pretrained head and other temporal aggregation methods under matched inference budgets?

  2. What are the results of the Sudoku EqR baseline with the same training horizon and gradient budget as ImR?

  3. Could the authors clarify what broader reasoning challenges the chosen maze and Sudoku settings are intended to capture, and which conclusions about latent reasoning are expected to generalize beyond these controlled tasks?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer duQ8 Part 1/2

Thank you for your careful review and constructive feedback. We quote and respond to your comments below. Section, appendix, table, and figure numbers in our responses refer to the revised manuscript.

CW1 and Q1 Comparing averaging with and without ImR training

CW1: The comparisons do not adequately isolate ImR’s contribution. Missing controls include averaging with the original pretrained head and alternative temporal aggregation methods.

Q1: How does ImR compare with averaging using the original pretrained head and other temporal aggregation methods under matched inference budgets?

We added comparisons to separate the effects of averaging and training in Appendix C.2. Keeping each model's parameters fixed, we evaluated prediction from the final state (Final) and from averaged logits (Average) with the same number of recurrent updates. These comparisons include the unchanged pretrained model and the conventionally fine-tuned model under Average, and ImR (FT) under Final. We also evaluated ImR trained from scratch under both rules.

Figures 8–10 show accuracy across recurrent depth, and Tables 7–10 report prediction stability and accuracy at the final depth.

Prediction stability. EqR stability is evaluated over the final 64 depths, ending at depth 512 on Maze-Unique and 1,024 on Sudoku-Extreme (Table 7). For models evaluated with three training seeds, we report the means.

For brevity, the table below presents selected prediction stability results on Maze-Unique under Final and Average.

ModelPuzzle prefixPrediction ruleNon-fixed rate (%)Stable correct (%)Stable incorrect (%)Unstable (%)
EqRWithFinal14.716.30.783.0
EqRWithAverage14.744.927.927.2
EqR (FT)WithFinal14.716.30.783.0
EqR (FT)WithAverage14.750.124.525.4
ImR (FT)WithFinal14.716.30.083.7
ImR (FT)WithAverage14.751.723.824.5
ImRWithFinal3.639.82.557.6
ImRWithAverage3.671.924.93.2
EqRWithoutFinal18.611.70.288.1
EqRWithoutAverage18.610.632.457.0
ImRWithoutFinal50.894.30.05.7
ImRWithoutAverage50.897.52.10.5

The table below presents the corresponding prediction stability results on Sudoku-Extreme under Final and Average.

ModelPuzzle prefixPrediction ruleNon-fixed rate (%)Stable correct (%)Stable incorrect (%)Unstable (%)
EqRWithFinal4.13.60.096.4
EqRWithAverage4.13.479.517.1
EqR (FT)WithFinal4.13.60.096.4
EqR (FT)WithAverage4.13.479.517.1
ImR (FT)WithFinal4.13.60.096.4
ImR (FT)WithAverage4.13.479.716.9
ImRWithFinal1.29.20.090.8
ImRWithAverage1.28.044.547.5
EqRWithoutFinal4.511.60.088.4
EqRWithoutAverage4.511.271.417.4
ImRWithoutFinal50.398.30.01.7
ImRWithoutAverage50.397.90.81.4

The tables show that averaging alone reduces instability but can stabilize incorrect predictions. Under Average, head fine-tuning increases the stable correct proportion on Maze-Unique, while the three heads give similar results on Sudoku-Extreme. ImR (FT) also has the highest stable correct proportion under Average on DT (Table 8). ImR trained from scratch without the prefix produces predominantly stable correct predictions even under Final, and averaging further reduces instability. Its higher accuracy than EqR under both prediction rules (Figures 8–9 and Table 9) shows that the gains are not solely due to averaging at inference. These results suggest additional benefits from training g and learning stable representations through f.

Appendix C.3 (Figures 11–12) also compares the main methods using measured inference time, including head and averaging costs, for EqR with the prefix and for DT. The results suggest that ImR can improve accuracy at comparable inference times within the evaluated range.

AuthorsOpenReview ↗

Reply to Reviewer duQ8 Part 2/2

CW1 and Q2 Comparisons with matched training horizon L and backpropagation length K

CW1: The Sudoku comparison without the puzzle prefix also uses different training horizons and gradient budgets. These gaps weaken the attribution of improvements to the proposed method.

Q2: What are the results of the Sudoku EqR baseline with the same training horizon and gradient budget as ImR?

The original EqR hyperparameter search for Sudoku-Extreme without the prefix already included the setting (L, K) = (36, 4), while the (3, 1) setting was selected by validation accuracy. The (36, 4) setting was also selected for EqR on Maze-Unique without the prefix (Table 6). To make the comparison more informative, we additionally evaluated the existing checkpoint trained with the (36, 4) setting on the test set.

The table below reports test accuracy on Sudoku-Extreme without the prefix at depth 1,024 (Appendix B.3 and Table 9). It compares EqR and ImR with the same training horizon L = 36 and backpropagation length K = 4, alongside the selected EqR setting (3, 1). Entries with three seeds are means.

ModelTraining setting (L, K)Accuracy (%)Training seeds
EqR(3, 1)96.23
EqR(36, 4)69.01
ImR(36, 4)99.03

In this evaluation, EqR with the (36, 4) setting achieved lower accuracy than the selected (3, 1) baseline. ImR remained more accurate than EqR when both models used the (36, 4) setting.

CW2 Accuracy versus inference time

Although accuracy-versus-depth curves are provided, the paper does not establish whether these gains translate into a favorable accuracy–runtime tradeoff in a practical use setting.

We added measurements of accuracy versus inference time in Appendix C.3 (Figures 11–12), including the cost of the prediction head and logit averaging. The linear prediction head in the EqR architecture adds little cost relative to the recurrent block. For DT, ImR (FT) takes slightly longer at matched recurrent depths because it evaluates the nonlinear prediction head at every step. Despite this additional cost, the accuracy gains observed at large recurrent depths can also be obtained at comparable inference times. These results support test-time scaling with ImR within the evaluated range.

CW2 and Q3 Choice of tasks and limitations

CW2: The practical significance remains insufficiently established. Evaluation is restricted to structured puzzle tasks, with some gains emerging only after thousands of recurrent updates.

Q3: Could the authors clarify what broader reasoning challenges the chosen maze and Sudoku settings are intended to capture, and which conclusions about latent reasoning are expected to generalize beyond these controlled tasks?

We chose maze and Sudoku to examine prediction instability in existing latent reasoning models and evaluate how averaging and training with ImR affect stability and accuracy. The tasks involve pathfinding and constraint satisfaction, respectively, and allow us to track how predictions change as recurrent computation is extended. The maze experiments also test controlled changes in problem size and the degree of the start cell. The improvements observed across multiple settings of these different reasoning tasks suggest that ImR may also be useful for other reasoning problems.

We expanded the paragraph on limitations in Section 6 to clarify which conclusions extend beyond these tasks. Non-fixed-point behavior has also been reported in research on large language models [1], suggesting that prediction instability may also arise in other domains. Under the assumptions in Sections 3.1–3.2, we expect averaged logits to stabilize predictions in other domains. Our analyses further suggest that learning stable representations through f and training g can provide benefits beyond averaging at inference alone. Further evaluation is needed to determine whether ImR improves performance in natural language and other domains.

References

[1] Geiping, J., et al. "Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach." NeurIPS, 2025.

Thank you for the response and the additional ablations and timing experiments. Comparisons with other temporal aggregation methods would still be helpful, as would further validation of practical utility beyond the current puzzle settings. I have raised some individual assessment scores and my confidence to reflect these improvements, while keeping my overall rating unchanged for now.

Reviewer_M6tROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ImR, which predicts by averaging readouts across recurrent latent states instead of using only the final state. The authors interpret this average as an approximation to an expectation under an invariant measure, allowing predictions to stabilize even when latent states continue to change. They apply the method through prediction-head fine-tuning and training from scratch on maze and Sudoku tasks. Results show improved accuracy and stability in several settings, although stabilization sometimes produces consistently incorrect predictions or reduces accuracy.

Critical strengths
  • The method is simple and adds no trainable parameters. For EqR’s linear head, averaging latent states before applying the head provides an efficient implementation (Appendix B.2).
  • The fine-tuning results show benefits without changing the recurrent dynamics. ImR improves Maze-Unique accuracy from 89.4% to 91.6%, and degree-2 OOD maze accuracy from 82.83% to 84.50% over ordinary head fine-tuning (Table 7).
  • The analysis and reporting are transparent. The paper reports three training seeds, detailed validation procedures, full accuracy-versus-depth curves, and cases where stability does not improve correctness.
Critical weaknesses
  • The most direct baseline—averaging predictions from the unchanged pretrained model—is missing. The authors should also evaluate conventionally fine-tuned heads with averaged readouts and ImR-trained heads with final-state readouts. These controls would show how much improvement requires ImR training and how much comes from averaging alone.

  • On no-prefix Sudoku, ImR uses L=36,K=4L=36,K=4, whereas EqR uses L=3,K=1L=3,K=1, with the same number of optimizer updates (Table 6). Equal update counts therefore do not imply equal computation. Appendix B.3 states that the EqR search included (36,4)(36,4); reporting that result alongside the selected baseline would make the comparison more informative. Matched-compute results would further strengthen the attribution.

  • The degree-3 improvement from 26.32% to 28.42% corresponds to approximately two additional solved mazes out of 95 (Tables 3 and 7). Training-seed variation does not capture uncertainty from the test sample. Paired confidence intervals or additional independently generated test sets would clarify the reliability of these gains.

Questions for authors

Please refer to Critical Weaknesses. If I've missed anything, please let me know.

4 comments
AuthorsOpenReview ↗

Reply to Reviewer M6tR Part 1/2

Thank you for your careful review and constructive feedback. We quote and respond to your comments below. Section, appendix, table, and figure numbers in our responses refer to the revised manuscript.

CW1 Comparing averaging with and without ImR training

The most direct baseline—averaging predictions from the unchanged pretrained model—is missing. The authors should also evaluate conventionally fine-tuned heads with averaged readouts and ImR-trained heads with final-state readouts. These controls would show how much improvement requires ImR training and how much comes from averaging alone.

We added all three requested comparisons in Appendix C.2. Keeping each model's parameters fixed, we evaluated prediction from the final state (Final) and from averaged logits (Average) with the same number of recurrent updates. These comparisons include the unchanged pretrained model and the conventionally fine-tuned model under Average, and ImR (FT) under Final. We also evaluated ImR trained from scratch under both rules.

Figures 8–10 show accuracy across recurrent depth, and Tables 7–10 report prediction stability and accuracy at the final depth.

Prediction stability. EqR stability is evaluated over the final 64 depths, ending at depth 512 on Maze-Unique and 1,024 on Sudoku-Extreme (Table 7). For models evaluated with three training seeds, we report the means.

For brevity, the table below presents selected prediction stability results on Maze-Unique under Final and Average.

ModelPuzzle prefixPrediction ruleNon-fixed rate (%)Stable correct (%)Stable incorrect (%)Unstable (%)
EqRWithFinal14.716.30.783.0
EqRWithAverage14.744.927.927.2
EqR (FT)WithFinal14.716.30.783.0
EqR (FT)WithAverage14.750.124.525.4
ImR (FT)WithFinal14.716.30.083.7
ImR (FT)WithAverage14.751.723.824.5
ImRWithFinal3.639.82.557.6
ImRWithAverage3.671.924.93.2
EqRWithoutFinal18.611.70.288.1
EqRWithoutAverage18.610.632.457.0
ImRWithoutFinal50.894.30.05.7
ImRWithoutAverage50.897.52.10.5

The table below presents the corresponding prediction stability results on Sudoku-Extreme under Final and Average.

ModelPuzzle prefixPrediction ruleNon-fixed rate (%)Stable correct (%)Stable incorrect (%)Unstable (%)
EqRWithFinal4.13.60.096.4
EqRWithAverage4.13.479.517.1
EqR (FT)WithFinal4.13.60.096.4
EqR (FT)WithAverage4.13.479.517.1
ImR (FT)WithFinal4.13.60.096.4
ImR (FT)WithAverage4.13.479.716.9
ImRWithFinal1.29.20.090.8
ImRWithAverage1.28.044.547.5
EqRWithoutFinal4.511.60.088.4
EqRWithoutAverage4.511.271.417.4
ImRWithoutFinal50.398.30.01.7
ImRWithoutAverage50.397.90.81.4

The tables show that averaging alone reduces instability but can stabilize incorrect predictions. Under Average, head fine-tuning increases the stable correct proportion on Maze-Unique, while the three heads give similar results on Sudoku-Extreme. ImR (FT) also has the highest stable correct proportion under Average on DT (Table 8). ImR trained from scratch without the prefix produces predominantly stable correct predictions even under Final, and averaging further reduces instability. Its higher accuracy than EqR under both prediction rules (Figures 8–9 and Table 9) shows that the gains are not solely due to averaging at inference. These results suggest additional benefits from training g and learning stable representations through f.

AuthorsOpenReview ↗

Reply to Reviewer M6tR Part 2/2

CW2 Comparisons with matched training horizon L and backpropagation length K

On no-prefix Sudoku, ImR uses (L=36,K=4), whereas EqR uses (L=3,K=1), with the same number of optimizer updates (Table 6). Equal update counts therefore do not imply equal computation. Appendix B.3 states that the EqR search included ((36,4)); reporting that result alongside the selected baseline would make the comparison more informative. Matched-compute results would further strengthen the attribution.

The original EqR hyperparameter search for Sudoku-Extreme without the prefix already included the setting (L, K) = (36, 4), while the (3, 1) setting was selected by validation accuracy. To make the comparison more informative, we additionally evaluated the existing checkpoint trained with the (36, 4) setting on the test set.

The table below reports test accuracy on Sudoku-Extreme without the prefix at depth 1,024 (Appendix B.3 and Table 9). It compares EqR and ImR with the same training horizon L = 36 and backpropagation length K = 4, alongside the selected EqR setting (3, 1). Entries with three seeds are means.

ModelTraining setting (L, K)Accuracy (%)Training seeds
EqR(3, 1)96.23
EqR(36, 4)69.01
ImR(36, 4)99.03

In this evaluation, EqR with the (36, 4) setting achieved lower accuracy than the selected (3, 1) baseline. ImR remained more accurate than EqR when both models used the (36, 4) setting.

CW3 Evaluation on the expanded degree-3 test set

The degree-3 improvement from 26.32% to 28.42% corresponds to approximately two additional solved mazes out of 95 (Tables 3 and 7). Training-seed variation does not capture uncertainty from the test sample. Paired confidence intervals or additional independently generated test sets would clarify the reliability of these gains.

We expanded the degree-3 test set from 95 to 190 mazes using the original generation procedure and excluding overlaps with the existing test set (Appendix B.1). Section 5.2 and Appendix C.2 report the updated results. The table below summarizes accuracy and prediction stability from Tables 2 and 10. Fine-tuned models report means across three training seeds.

ModelAccuracy (%)Stable correct (%)Unstable (%)
DT26.3226.312.6
DT (FT)26.8426.310.9
ImR (FT)28.6028.61.1

On the expanded set, ImR (FT) remains more accurate than both baselines and has the highest stable correct proportion, consistent with the results on the original test set of 95 degree-3 mazes.

Thank you for your rebuttal. I'd like to keep my positive rating.

Thank you again for your time, careful consideration of our rebuttal, and positive assessment.

Reviewer_HwH1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes ImR, a method that stabilizes predictions in latent reasoning models by averaging predictions across recurrent steps. Using the empirical measure of latent states along the recurrent trajectory, it approximates an invariant measure. The paper evaluates the method on Sudoku and maze tasks.

Critical strengths
  • ImR provides a simple approach to stabilizing predictions without modifying the architecture.
  • The analysis of fixed and non-fixed trajectories helps characterize the behavior of recurrent latent reasoning models.
  • The paper evaluates both prediction accuracy and stability, capturing fluctuations that accuracy alone cannot reflect.
Critical weaknesses
  • The paper assumes that the empirical measure of latent states converges to an invariant measure but does not verify this assumption experimentally.
  • The paper does not establish its advantages over temporal aggregation methods. ImR is implemented simply by uniformly averaging logits.
  • ImR uses 36 recurrent updates per inner loop, compared with only 3 for EqR, raising concerns about its computational cost.
Questions for authors
  • How does ImR compare with other aggregation methods, such as sliding-window averaging or directly averaging logits from a pretrained EqR model without fine-tuning?
  • Why does removing the puzzle prefix lead to a larger increase in the proportion of non-fixed trajectories for ImR than for EqR?
5 comments
AuthorsOpenReview ↗

Reply to Reviewer HwH1 Part 1/2

Thank you for your careful review and constructive feedback. We quote and respond to your comments below. Section, appendix, table, and figure numbers in our responses refer to the revised manuscript.

CW1 Convergence of empirical measures

The paper assumes that the empirical measure of latent states converges to an invariant measure but does not verify this assumption experimentally.

We added an experimental analysis of non-fixed trajectories in Section 5.3 (Figure 4) and Appendix C.1 (Figures 6–7). Trajectories classified as fixed already show approximate convergence to fixed points, providing empirical evidence of convergence to point measures. Using Wasserstein distance, we compared empirical measures over recurrent depths 3,073–3,584 and 3,585–4,096 for Maze-Unique, and 2,049–3,072 and 3,073–4,096 for Sudoku-Extreme. For ImR, these distances concentrate near zero in most settings, providing empirical support for the convergence assumption.

On Sudoku-Extreme with the prefix, ImR's main peak remains away from zero, indicating persistent differences between the measured empirical distributions. Only 1.2% of its evaluated trajectories are non-fixed (Table 7). One possible explanation is that training mainly optimized predictions for inputs with fixed trajectories. The empirical measures along the remaining non-fixed trajectories may therefore not have been sufficiently stabilized.

CW2 and Q1 Comparing averaging with and without ImR training

CW2: The paper does not establish its advantages over temporal aggregation methods. ImR is implemented simply by uniformly averaging logits.

Q1: How does ImR compare with other aggregation methods, such as sliding-window averaging or directly averaging logits from a pretrained EqR model without fine-tuning?

We added comparisons of prediction stability and accuracy in Appendix C.2. Keeping each model's parameters fixed, we evaluated prediction from the final state (Final) and from averaged logits (Average) with the same number of recurrent updates. These comparisons include the unchanged pretrained model and the conventionally fine-tuned model under Average, and ImR (FT) under Final. We also evaluated ImR trained from scratch under both rules.

Figures 8–10 show accuracy across recurrent depth, and Tables 7–10 report prediction stability and accuracy at the final depth.

Prediction stability. EqR stability is evaluated over the final 64 depths, ending at depth 512 on Maze-Unique and 1,024 on Sudoku-Extreme (Table 7). For models evaluated with three training seeds, we report the means.

For brevity, the table below presents selected prediction stability results on Maze-Unique under Final and Average.

ModelPuzzle prefixPrediction ruleNon-fixed rate (%)Stable correct (%)Stable incorrect (%)Unstable (%)
EqRWithFinal14.716.30.783.0
EqRWithAverage14.744.927.927.2
EqR (FT)WithFinal14.716.30.783.0
EqR (FT)WithAverage14.750.124.525.4
ImR (FT)WithFinal14.716.30.083.7
ImR (FT)WithAverage14.751.723.824.5
ImRWithFinal3.639.82.557.6
ImRWithAverage3.671.924.93.2
EqRWithoutFinal18.611.70.288.1
EqRWithoutAverage18.610.632.457.0
ImRWithoutFinal50.894.30.05.7
ImRWithoutAverage50.897.52.10.5

The table below presents the corresponding prediction stability results on Sudoku-Extreme under Final and Average.

ModelPuzzle prefixPrediction ruleNon-fixed rate (%)Stable correct (%)Stable incorrect (%)Unstable (%)
EqRWithFinal4.13.60.096.4
EqRWithAverage4.13.479.517.1
EqR (FT)WithFinal4.13.60.096.4
EqR (FT)WithAverage4.13.479.517.1
ImR (FT)WithFinal4.13.60.096.4
ImR (FT)WithAverage4.13.479.716.9
ImRWithFinal1.29.20.090.8
ImRWithAverage1.28.044.547.5
EqRWithoutFinal4.511.60.088.4
EqRWithoutAverage4.511.271.417.4
ImRWithoutFinal50.398.30.01.7
ImRWithoutAverage50.397.90.81.4
AuthorsOpenReview ↗

Reply to Reviewer HwH1 Part 2/2

The tables show that averaging alone reduces instability but can stabilize incorrect predictions. Under Average, head fine-tuning increases the stable correct proportion on Maze-Unique, while the three heads give similar results on Sudoku-Extreme. ImR (FT) also has the highest stable correct proportion under Average on DT (Table 8). ImR trained from scratch without the prefix produces predominantly stable correct predictions even under Final, and averaging further reduces instability. Its higher accuracy than EqR under both prediction rules (Figures 8–9 and Table 9) shows that the gains are not solely due to averaging at inference. Across settings, the smaller Wasserstein distances reported in our response to CW1 also tend to coincide with lower unstable proportions under Average. These results suggest additional benefits from training g and learning stable representations through f.

CW3 Comparisons with matched training horizon L and backpropagation length K

ImR uses 36 recurrent updates per inner loop, compared with only 3 for EqR, raising concerns about its computational cost.

To clarify, EqR and ImR already use the setting (L, K) = (36, 4) on Maze-Unique without the prefix, with the same batch size and optimizer-update budget (Table 6).

The original EqR hyperparameter search for Sudoku-Extreme without the prefix already included the setting (L, K) = (36, 4), while the (3, 1) setting was selected by validation accuracy. To make the comparison more informative, we additionally evaluated the existing checkpoint trained with the (36, 4) setting on the test set.

The table below reports test accuracy on Sudoku-Extreme without the prefix at depth 1,024 (Appendix B.3 and Table 9). It compares EqR and ImR with the same training horizon L = 36 and backpropagation length K = 4, alongside the selected EqR setting (3, 1). Entries with three seeds are means.

ModelTraining setting (L, K)Accuracy (%)Training seeds
EqR(3, 1)96.23
EqR(36, 4)69.01
ImR(36, 4)99.03

In this evaluation, EqR with the (36, 4) setting achieved lower accuracy than the selected (3, 1) baseline. ImR remained more accurate than EqR when both models used the (36, 4) setting.

Q2 Non-fixed trajectories without the puzzle prefix

Why does removing the puzzle prefix lead to a larger increase in the proportion of non-fixed trajectories for ImR than for EqR?

We clarified the discussion in Section 6. Removing the puzzle prefix reduces the latent dimensions available for recurrent computation. We conjecture that this may constrain the model's ability to represent correct solutions through convergence to a fixed point. ImR has the flexibility to form stable representations from non-fixed trajectories, so training with ImR may favor such trajectories when fewer latent dimensions are available.

Thank you for the detailed rebuttal and additional experiments. While the responses address several of my concerns, the advantages of the invariant-measure formulation over generic temporal aggregation remain insufficiently established, and the evaluation is limited to Sudoku and maze tasks. I therefore maintain my original score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary for reviewers and AC

Summary for reviewers and AC

Thank you for the constructive reviews. Below, we summarize the revisions addressing the reviewers' Critical Weaknesses (CW) and Questions for Authors (Q). The sections, appendices, tables, and figures cited below are from the revised manuscript.

  1. Averaging with and without ImR training — HwH1 CW2/Q1; M6tR CW1; duQ8 CW1/Q1. We added an ablation of prediction stability and accuracy, comparing prediction from the final state and from averaged logits for released models, conventional and ImR head fine-tuning, and ImR trained from scratch. The results suggest that training g and learning stable representations through f provide benefits beyond averaging logits at inference alone. Appendix C.2 (Figures 8–10 and Tables 7–10) reports the detailed ablation results.

  2. Comparisons with matched training horizon L and backpropagation length K — HwH1 CW3; M6tR CW2; duQ8 CW1/Q2. EqR was already trained with the setting (L, K) = (36, 4) during the hyperparameter search reported in the original submission. We additionally evaluated the existing checkpoint on the test set and report the results in Appendix B.3. On Sudoku-Extreme without the prefix, it achieved 69.0% for one training seed, compared with 96.2% for the selected EqR setting (3, 1) and 99.0% for ImR (Table 9). On Maze-Unique without the prefix, EqR and ImR already share L, K, batch size, and optimizer-update budget (Table 6).

  3. Convergence of empirical measures — HwH1 CW1. We added an experimental analysis in Section 5.3 (Figure 4) and Appendix C.1 (Figures 6–7), comparing empirical measures across finite depth intervals using Wasserstein distance. Small distances between later intervals provide empirical support for the convergence assumption in most evaluated settings for ImR.

  4. Non-fixed trajectories without the puzzle prefix — HwH1 Q2. We clarified the conjecture in Section 6 that reducing latent dimensions may constrain the representation of correct solutions through convergence to a fixed point. ImR's flexibility to form stable representations from non-fixed trajectories may therefore favor such trajectories during training.

  5. Evaluation on the expanded degree-3 test set — M6tR CW3. We generated 95 additional mazes using the original procedure, checked for overlap, and reevaluated all models on the expanded set of 190. The updated results in Section 5.2 and Appendix C.2 show that ImR (FT) remains more accurate than DT and DT (FT). Appendix B.1 describes the expanded test set.

  6. Accuracy versus inference time — duQ8 CW2. We added measurements of accuracy versus inference time in Appendix C.3 (Figures 11–12), including the cost of the prediction head and logit averaging. The results suggest that ImR can improve accuracy at comparable inference times within the evaluated range.

  7. Choice of tasks and limitations — duQ8 CW2/Q3. We clarified why we chose maze and Sudoku for our evaluation and expanded the discussion of limitations in Section 6. The improvements observed across multiple settings of these different reasoning tasks suggest that ImR may also be useful for other reasoning problems. Under our assumptions, averaged logits are expected to stabilize predictions. Further evaluation is needed to determine whether ImR improves performance in other domains.

Below, we summarize the additional revisions we made in response to the reviewers’ follow-up comments. Blue text in the revised manuscript marks additions and revisions since the original submission.

  1. Experimental comparison with other temporal aggregation methods — HwH1; duQ8. We compared ImR’s uniform averaging with sliding-window averaging (Window), exponential moving average (EMA), and max pooling (Max) on the released EqR and DT checkpoints. With the original head and with heads fine-tuned for each aggregation method, uniform averaging yielded fewer unstable predictions than any tested Window or EMA setting on non-fixed EqR trajectories from Maze-Unique and Sudoku-Extreme. The same ordering held in most DT settings. At the final recurrent depth, uniform averaging was about as accurate as Window and EMA on Maze-Unique and slightly less accurate on Sudoku-Extreme. On DT’s degree-2 and degree-3 OOD mazes, uniform averaging yielded a higher proportion of stable correct predictions and slightly higher accuracy than Window or EMA. Max often produced stable incorrect predictions, consistent with its ability to retain large logits from transient states that do not yet represent a solution. Appendix C.6 reports the full results in Tables 11–14.

  2. Advantage of the invariant-measure formulation over generic temporal aggregation — HwH1; duQ8. Our invariant-measure formulation differs from generic temporal aggregation in that it interprets the aggregated logits as the expected output of the prediction head under a measure on latent states. In this framework, we can also express the averaged logits for Window and EMA as expectations under the empirical measures defined by their respective temporal weights. This lets us analyze prediction stability through weak convergence of those measures. ImR’s empirical measure at depth TT assigns weight 1/T1/T to the state visited at each depth from 11 to TT. Under the conditions of Section 3.2, ImR’s averaged logits converge when this measure converges weakly to an invariant measure as TT increases, even if the states keep changing. On the same trajectory, the empirical measure over a fixed-length window of the most recent states need not converge weakly. The window length needed for a stable average can also vary with the input, so using a single length can leave some predictions unstable. We also proved that, under the assumptions of Appendix C.6, the empirical measure for EMA with fixed 0<β<10 < \beta < 1 converges weakly if and only if the states converge. Its limit is then the point measure at that state. Thus, if ImR’s empirical measure converges weakly to an invariant measure that is not concentrated at a single state, the empirical measure for EMA does not converge weakly on that trajectory. Window and EMA can therefore yield unstable predictions across recurrent depths even when ImR’s averaged logits converge for the same input. Consistent with this analysis, Tables 11–13 show that ImR yielded more stable predictions than Window and EMA on non-fixed EqR trajectories and in most DT settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.