acceptodds
Under review as a conference paper at ICLR 2027

Quantizing Recursive Reasoning Models

70 views

Abstract

Recursive reasoning models solve hard puzzles by applying compact, weight-tied blocks over many refinement steps. Because these blocks are reused many times, quantizing them creates a severe error accumulation challenge: the quantization error is incurred at every step. While 8-bit quantization (integer or float) preserves accuracy, moving to a per-tensor 4-bit format causes a systematic bias to accumulate. The ensuing drift catastrophically degrades exact-solution accuracy on Sudoku from 84.1% to 0.0% (only  25% of cells correct). In this work, we show that this collapse is caused by activation-scaling granularity rather than bit-width or number format. Crucially, moving to per-block scaling restores stable contraction and most of the lost accuracy. To implement this, we apply MXINT4, a blockwise integer format for weights and activations, to recursive reasoning models. It is competitive with blockwise float formats on our tasks, while keeping integer elements and power-of-two block scales. Finally, recursion depth and reuse modulate quantization sensitivity, with the deepest trained model we test (EqR) the most sensitive. Yet blockwise scaling overcomes this vulnerability, staying robust across the evaluated recursive models and transferring to the open-ended ARC-AGI benchmark.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
quantization, recursive reasoning models, weight-tied recursion, post-training quantization, microscaling formats, MXINT4, low-bit inference, tiny recursive models, activation quantization, error accumulation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Sk9nOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies quantization at the level of these reused transition operators, including how transition perturbation, blockwise integer transition rules, and recursion depth and reuse modulate quantizability. Experiments on Sudoku and Maze show the effectiveness of the proposed method.

Critical strengths

Quantization is useful for neural networks, and this paper studies quantization at multiple levels.

Critical weaknesses
  1. The effectiveness of the proposed method is not sufficiently verified. For example, how about the performance with different backbones?
  2. Quantization has been studied for several years. Due to the lack of comparisons with the SOTA methods, the advantages of the proposed method are not well demonstrated.
  3. As discussed in the limitations. Quantization is general, and it can be applied to many tasks, such as LLMs and vision tasks. Is the proposed method effective on those tasks?
Questions for authors

Due to the lack of a deep discussion of recent methods, the advantages of the proposed method are not well demonstrated. See weaknesses.

3 comments

Dear reviewer, thank you for your review. The revised PDF is uploaded with these replies.

W1, backbones. Four recursive architectures now, with HRM new on its public checkpoints: 55.1% in full precision, 0.0% with per-tensor 4-bit, 53.8% with MXINT4. On HRM Maze: 74.2%, 0.0 and 74.3.

W2 / Q1, state of the art. The contribution is the comparison across four recursive architectures: which standard fixes recover exact accuracy, and what accuracy is traded for a shared-exponent integer format.

exact %, 4-bit weights and activationsSudoku TRM (3 seeds)EqRHRM Sudokuneeds at inference
full precision83.386.455.1
per-tensor scales0.80.00.0one float scale per tensor
QuaRot-style rotation baseline79.982.453.9a float scale per channel and token, and the rotation
group-32, float scales81.584.454.8a float scale per block of 32
MXINT479.380.253.8a power-of-two scale per block of 32, integer elements

Float scales of the same granularity are 1.0 to 4.2 points better than MXINT4. That is the measured price of the power-of-two rule, which buys integer elements with one shared exponent per block. We claim no accuracy advantage for MXINT4. The other models and methods are in the general response, part 1.

GPTQ and AWQ act on the weights, which are the small term here: 4-bit weights with per-channel scales cost 1.0 point on Sudoku (one seed), while the activation scale alone moves accuracy between 1% and 82%. We therefore compared against methods that act on the activations and did not run those two. LoopQ is concurrent work on looped language models that adds loop-aware adapters and trajectory-aware calibration. We study post-training quantization without adapters on exact-solution tasks, and Sec. 2 states the difference.

The format family runs on a shipping accelerator. On a Tenstorrent Blackhole card, 4-bit weights are lossless and 3.55x smaller, with 4.9% higher device throughput than bf16 (general response, part 2).

W3, scope. Our claim is for models that reuse one small block many times per forward pass (7M parameters for TRM, 27M for HRM), which is what makes them candidates for memory-constrained inference. We make no claim for feed-forward LLMs or vision models. For those, 4-bit microscaling and rotation are covered by the work cited in Sec. 2. The language-model case with our structure is the looped transformer, which LoopQ studies.

Thanks for the feedback. After reading the response and other comments. I think my main concerns about the backbones and comparisons are addressed. I would like to raise the rating.

AuthorsOpenReview ↗

Thank you for your review

Dear reviewer Sk9n, thank you for reading the response and the discussion, and for the updated rating.

Reviewer_i9cjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies low-bit quantization of recursive reasoning models, including TRM and EqR. It finds that standard per-tensor W4A4 quantization can severely degrade performance, reducing Sudoku exact-solve accuracy from approximately 84% to 0%, while blockwise MXINT4 restores it to approximately 80% without retraining. The paper interprets these failures through repeated quantization perturbations in recursive computation and presents a conditional contraction-based error bound to motivate its stability analysis. Evaluation spans Sudoku, maze solving, and ARC, with comparisons across quantization formats, weight and activation quantization ablations, experiments varying parameter reuse, trajectory diagnostics, and quantization-aware training. The main empirical contribution is demonstrating substantial performance recovery under suitable four-bit quantization configurations. The theoretical analysis and diagnostic experiments offer possible explanations for the observed failures, though their causal role is not conclusively established.

Critical strengths
  • A clear and consequential empirical finding. The drop from roughly 84% to 0% Sudoku exact-solve accuracy under per-tensor W4A4 is striking and makes this a problem worth studying.

  • A useful fix without retraining. Blockwise MXINT4 brings Sudoku accuracy back to roughly 80%. Even though the technique itself is established, showing that it works this well for these recursive solvers has practical value.

  • Broad experimental coverage. The paper explores multiple models, tasks, quantization formats, and training options. This gives readers useful information about which configurations work beyond the main Sudoku example.

  • Artifacts that support independent checks. The released code, evaluation records, and predictions make several important results independently checkable. In particular, rescoring the supplied ARC predictions reproduces the reported aggregate accuracies.

  • A useful numerical perspective. Framing quantization as a perturbation to an iterative computation raises good questions about error propagation and stability. This is a helpful starting point for further work, even if the proposed explanation remains only partly validated.

Critical weaknesses
  • The experiments do not yet separate a recursion-specific problem from ordinary quantization error. The central question is whether recursive computation makes a given local error more harmful, or whether the tested quantizer simply introduces too much error in these models. The reuse and weight-untying experiments change other relevant factors, including model capacity and full-precision performance, so they do not settle this distinction. Comparing sensitivity at matched local error levels, and testing repeated versus independently sampled perturbations with comparable per-step statistics, would help establish what recursion actually contributes to the failure.

  • The proposed mechanism is not cleanly connected to the measured failure or recovery. A key diagnostic compares two evolving trajectories, which mixes fresh quantization error with previously accumulated state differences. The contraction bound is conditional, and the experiments do not establish whether its assumptions hold in the evaluated models. Meanwhile, the strongest recovery changes both weight and activation quantization, leaving their respective contributions unclear. Measuring fresh error on identical input states and separately varying weight and activation grouping would make the explanation much more convincing. As it stands, the paper establishes an effective configuration more clearly than it explains why that configuration works.

Questions for authors

I currently slightly lean towards rejection, but I do want to acknowledge the practical value of the method. These questions represent a few points that, in my opinion, can help the authors improve the scientific value of this work in next steps:

  1. Does recursion make comparable local errors more damaging? Could you test this with controlled perturbations in a fixed model, comparing a single injection, repeated use of a fixed parameter perturbation, and independently sampled parameter perturbations across steps? Matching the resulting per-step error magnitudes as closely as possible would help distinguish sensitivity to error size from sensitivity to its persistence and temporal structure.

  2. Can the proposed error-propagation mechanism be measured directly? At the same input state, could you measure the difference between the quantized and full-precision updates, then separately track how an isolated perturbation evolves under subsequent full-precision updates? This would separate fresh error from propagated error and test whether perturbations decay, persist, or grow along actual reasoning trajectories. Relating these measurements to exact-solve outcomes would also help connect numerical behavior to task failure.

  3. Which part of the successful quantization configuration drives the recovery? Could you cross per-tensor versus blockwise weight quantization with per-tensor versus blockwise activation quantization, while keeping other choices fixed? Separately testing the scale rule would help clarify whether the gain comes mainly from weight grouping, activation grouping, or their interaction, and give readers a more transferable explanation of the fix.

[Post-Review Update:] The authors have addressed many aspects I brought up in the Weakness and Questions sections. I thus now lean towards acceptance by a small margin.

3 comments

Dear reviewer, thank you for the review and for the three questions; we could run as specified. Sudoku numbers are for three training seeds unless marked otherwise.

W1. We tested whether reuse explains the 4-bit failure, and it explains part of it. Isotropic noise with the same per-step error already removes about 60 of the 83 points on Sudoku. The quantizer removes 22 to 25 more, and we have not shown that repetition is the reason. You are right that the reuse and untying experiments of the submission change capacity and full-precision accuracy. The within-model experiments below replace them as the main evidence. We have no non-recursive network trained on these tasks, and the paper says so.

Q1, matched error size and reuse. Noise at the per-tensor quantizer's own per-step error keeps 24.6 / 24.0 / 24.6% where the quantizer keeps 2.2 / 0.0 / 0.1% (general response, part 2). For reuse, ΔW is a random matrix with the Frobenius norm of the real Q(W) − W at the plain per-tensor weight scale.

Sudoku TRM, exact %3 seeds
per-tensor W4A4 during step 1 only82.2 ± 1.5
per-tensor W4A4 during the final step only83.0 ± 0.7
per-tensor W4A4 at every step0.8 ± 1.0
random ΔW, reused at every step64.9 ± 3.6
random ΔW, redrawn at every step81.5 ± 2.2

The two ΔW arms match the norm of the weight error and not the per-step error it produces. This is our only direct test of reuse.

Q2, fresh error at identical states and the fate of an isolated error. At every step we feed the full-precision (FP) state to both transitions and record the fresh error ε. We also inject the fresh error at one step and propagate it with FP updates.

quantizerfresh error εone-step ratio L₁ after injectionε / (1 − L₁)observed divergence at step 16
Int823.9 ± 6.40.14 ± 0.0428.036.8 ± 6.4
MXINT443.1 ± 1.50.11 ± 0.0148.456.3 ± 0.7
static per-tensor Int4160.9 ± 4.80.10 ± 0.02178.9262.4 ± 12.3
dynamic per-tensor Int4149.1 ± 7.60.16 ± 0.09178.7234.1 ± 10.4
  • An isolated error is contracted. For puzzles the FP model solves, a per-tensor error injected at step 4 or later falls from 138 to 9.6 in one FP step.
  • Accumulated damage recovers slowly. After fifteen quantized steps, 1, 2 and 4 FP steps bring exact accuracy back to 10%, 46% and 67%.
  • Proposition 1. No single contraction ratio makes it a useful estimate. With the one-step ratio it underestimates the observed divergence by a factor of 1.1 to 1.7, and with the ratio measured after fifteen quantized steps (0.6 to 0.7) the estimate exceeds it (460 against 262 for static per-tensor). The revised appendix keeps it as a qualitative scaffold, with this measured gap.
  • The divergence norm does not predict accuracy. On seed A, quantizing only the final step and quantizing steps 1 to 15 end at the same divergence (153.6 and 153.1) with 83.9% and 12.9% exact.

Q3, which part of the configuration drives the recovery. Sudoku TRM, seed A, per-call scales, exact %:

weights \ activationsper-tensorper-tensor, clipped at 99.5%per-tokengroup-32
per-tensor (MSE-clipped scale)2.249.665.776.1
per-channel2.366.976.182.2
group-322.3not run77.282.3
  • Activation grouping is the first term. Moving the activations alone to group-32 takes the per-tensor-weight row from 2.2 to 76.1.
  • Weight grouping is the second term: 6 to 12 points once the activations are fine-grained.
  • The scale rule costs 2.0 points. At group-32 on both sides, float scales give 82.3 and power-of-two scales 80.3 (this is MXINT4). Over three seeds the gap is 2.2 ± 0.5 (4.2 on EqR, 1.0 on HRM).

W2. The revised paper no longer reads the stochastic-inference experiments or the coherence diagnostic as evidence for a coherent bias, and the mechanism figure of Sec. 3 now shows the measurements of this reply.

Reviewer_i9cjOpenReview ↗

I like and sincerely appreciate your response.

Thank you for the substantial additional experiments and please pardon my untimely response. The weight/activation cross-ablation directly addresses my concern about what drives recovery: activation grouping accounts for most of the gain, with finer weight quantization providing further improvement. I also appreciate the same-state measurements of fresh error, the isolated-perturbation experiments, and the clearer acknowledgment of the contraction bound’s limitations. The repeated-versus-resampled weight perturbation comparison provides useful evidence that temporal structure matters. My remaining reservation is that this comparison matches parameter-error norms rather than the resulting per-step state errors, so the role of recurrence in the full W4A4 collapse remains only partly resolved. The revised positioning of MXINT4 relative to strong rotation and floating-scale baselines is appropriate; its deployment benefits should likewise be scoped to the configurations actually measured. Finally, the abstract and TL;DR should reflect these qualifications, particularly the observation that isolated errors contract even under configurations that fail with repeated quantization. Overall, the new evidence meaningfully addresses my main concerns and strengthens the paper’s empirical contribution. I am inclined to raise my recommendation from weak rejection to weak acceptance.

AuthorsOpenReview ↗

Dear reviewer

Dear reviewer i9cj thank you for the reply and for reconsidering your recommendation. We would like to also try to address the three remaining points you pointed out!

Reuse comparison. Our earlier sentence was imprecise and we have corrected it. The two arms are matched in per-step error as well as in weight-error norm. Reused and redrawn perturbations produce the same output error per call (4.40% ± 0.21 and 4.39% ± 0.22 of the output norm, equal on each of the three training seeds) and the same latent divergence after the first step (118 and 120). They separate only afterward. At step 16, the divergence is 107 with reuse and 85 with redrawing, and exact accuracy is 64.9 ± 3.6 and 81.5 ± 2.2. The part that stays open is the size of the perturbation. It is smaller than the per-tensor W4A4 error, which is about 10% of the output norm per call, and zero-mean noise of that size alone removes most of the accuracy. The test therefore shows that reuse matters at matched per-step error. It does not isolate the share of reuse in the full W4A4 collapse. Sec. 3 and App. A.14 now state both.

Deployment scope. The introduction and Sec. 4 now state that the simpler integer datapath is a motivation we do not benchmark, and that MXINT4 has no accuracy advantage over float block scales or the rotation baseline.

Abstract and TL;DR. The abstract now states that under the configuration that collapses an isolated error contracts, that quantizing a single step costs about one point, and that the collapse arises only when every step is quantized. We did want to modify the TL;DR like you suggested but the revision form does not let us edit the TL;DR field during the discussion. We will change it at the first opportunity to: "Per-tensor 4-bit quantization collapses weight-tied recursive reasoning models only when every step is quantized, while an isolated error contracts. Per-block activation scaling, integer or float, restores most of the accuracy on TRM, EqR and HRM."

The revised PDF contains these changes in blue.

Reviewer_3wq4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies quantization in recursive reasoning models that repeatedly apply shared network modules. It compares quantization formats and scaling granularity, tracks errors across recursive steps, and evaluates quantization-aware training. On Sudoku, per-tensor W4A4 quantization reduces accuracy from 84.1% to 0%, while blockwise MXINT4 recovers 80.1% without retraining. Experiments also cover maze solving, equilibrium reasoning models, and ARC-AGI.

Critical strengths
  1. The method preserves substantial accuracy without retraining. MXINT4 recovers most of the full-precision Sudoku performance and performs well on other tasks. On ARC-AGI-1, it achieves 44.0% versus the full-precision model’s 45.25%, supporting the feasibility of 4-bit quantization for these models.
  2. The experiments examine behavior throughout recursion. Tracking hidden-state and output errors and varying recursive depth provide useful information beyond final accuracy. The results also show that additional recursion does not always worsen quantization damage, helping clarify when error accumulation occurs.
Critical weaknesses
  1. The headline result relies heavily on a coarse-grained baseline that completely fails. The drop from 84.1% to 0%, followed by recovery to 80.1%, is striking. However, the failing baseline uses per-tensor quantization for both weights and activations. The paper needs to distinguish a problem specific to recursive reasoning from a generally unsuitable quantization configuration. Blockwise MXFP4 already reaches 72.7%, so the more informative comparison is the improvement over reasonable fine-grained W4A4 baselines, rather than the recovery from zero.
  2. The existing controls do not adequately establish that conventional quantization techniques cannot address the failure. The Hadamard rotation baseline only rotates layers whose input dimensions are powers of two. Other layers, including certain SwiGLU down-projections, retain ordinary per-tensor Int4 quantization. Yet the paper identifies down-projections as the largest sources of quantization error. Consequently, this baseline’s 0% accuracy does not convincingly demonstrate that a rotation-based approach addressing the critical layers would also fail.
Questions for authors

Could the authors explain the collapse from 84.1% to 0% through comparisons with fine-grained W4A4 baselines that use matched quantization coverage and appropriately handle both weights and activations, showing how much improvement MXINT4 retains and whether the collapse reflects recursive-model sensitivity or the coarse quantization configuration?

3 comments

Dear reviewer, thank you for your review. Both weaknesses you identified are correct, and both have changed the paper.

W1 / Q1, the comparison you asked for. MXINT4 has no accuracy advantage over the best fine-grained W4A4 baselines at matched coverage. On Sudoku (three seeds) it is 2.2 points below float group-32 scales and level with a QuaRot-style baseline. It is 7.7 above per-channel/per-token scaling and 4.3 above MXFP4. The table over five models is in the general response, part 1. The revised paper leads with this comparison and keeps the recovery from zero only as the motivating observation.

What MXINT4 offers is a simpler datapath. Against float group-32 scales it replaces the float scale by a power-of-two one, at a measured price of 1.0 to 4.2 points. Against the rotation baseline it needs no rotation and no per-channel or per-token float scale.

Matched-coverage cross (Sudoku TRM, one seed with full precision 84.3, per-call scales, exact %)

weights \ activationsper-tensorper-tensor, clipped at 99.5%per-tokengroup-32
per-tensor (MSE-clipped scale)2.249.665.776.1
per-channel2.366.976.182.2
group-322.3not run77.282.3

MXINT4 is the group-32 cell with power-of-two scales (80.3 on this seed). The clipping threshold was tuned on this seed. Over three seeds the 99.9th percentile gives 45.3 ± 10.6.

W1, recursion or an unsuitable configuration? Mostly the configuration.

  • The per-tensor error of one step is 145 to 160 against a state norm of 204. Isotropic noise of that size already removes about 60 of the 83 points on Sudoku, and the quantizer removes 22 to 25 more at equal size.
  • An untied model with 7 times less reuse per weight collapses in the same way (0.1% against 0.0% for the tied model).
  • The only direct test of reuse is on the weights: a random perturbation reused at every step costs 16.7 ± 1.9 points more than one redrawn at every step at the plain per-tensor weight scale, and 2 to 4 points more at better weight scales (one seed).

We have no non-recursive network trained on these tasks, so these are within-model controls and the revised paper states that limit.

W2, the rotation control. You were right. The submitted control rotated only layers with power-of-two input width and left the 1536-wide down-projection at plain per-tensor Int4. We reran it with every layer rotated and with the random sign diagonal that QuaRot uses (the two rotation rows of the general table).

  • With per-channel weights and per-token activations, rotation reaches 79.9 ± 0.8 on Sudoku, level with MXINT4. It adds 8.3 points on Sudoku TRM, 22.0 on HRM and 25.7 on EqR over the same scales without rotation.
  • With per-tensor scales, it repairs the attention models, as you anticipated: fully on Maze, to 50.6 of 55.1 on HRM and to 36.5 of 45.0 on ARC-AGI-1. It still gives 0.6 on Sudoku TRM and 0.0 on EqR, the two MLP-mixing models.

The revised paper replaces the statement that outlier controls fail by these results.

AuthorsOpenReview ↗

Update before discussion closes.

Dear reviewer 3wq4, a short update before the discussion closes, since the final PDF sharpens two points of our reply to you.

W1 and Q1. The paper now leads with the matched-coverage comparison you asked for (the baseline table over five models in Sec. 4). The abstract and introduction state that MXINT4 has no accuracy advantage over float block scales or a full rotation baseline. The recovery from zero is kept as the motivating observation.

Recursion or configuration. The reused and redrawn weight perturbations are matched in per-step error (4.4% of the output norm per call on each of three seeds) and in latent divergence after the first step (118 and 120). They separate afterwards, to 107 and 85 at step 16. This perturbation is smaller than the full W4A4 error, so the paper states that the test does not isolate the share of reuse in the full collapse.

W2. The rotation control with every layer rotated is in the main baseline table, and the appendix reports it on ARC-AGI-2 as well.

If any part of your two concerns remains open, we would be glad to address it.

Thank you for the thoughtful response. The additional experiments have addressed my main concerns and strengthened the paper substantially. I will therefore raise my score.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General response (1/2): what the revised paper shows

We thank the three reviewers. The revised PDF is uploaded, with changes in blue. Quantization-aware training and ARC-AGI-2 are left out for now because their runs on the newly trained models are still in progress. We will add them by Tuesday 17 November.

  • Finding. With an unclipped per-tensor activation scale, 4-bit weights and activations leave 0.0 to 0.8% exact accuracy on five recursive models from four architectures (TRM with MLP mixing, TRM with attention, EqR and, new, HRM). Group-32 float scales recover every model to within 2 points of full precision without retraining. Rotation repairs per-tensor scales on the attention models and fails on the MLP-mixing ones. Range clipping reaches 80% on Maze and 45% on Sudoku.
  • Recursion. Quantizing one block call costs at most 0.2 points. On Sudoku, isotropic noise with the same per-step error already removes about 60 of the 83 points, and the quantizer removes 22 to 25 more. A weight perturbation reused at every step costs 16.7 ± 1.9 points more than one redrawn at every step (2 to 4 points at better weight scales). We have no non-recursive network trained on these tasks, so these are within-model controls (part 2).
  • MXINT4. We claim no accuracy advantage. It is level with a QuaRot-style rotation baseline on four of the five models (2.2 points lower on EqR), and float scales of the same granularity are 1.0 to 4.2 points more accurate. What it offers is the datapath: one power-of-two scale per block of 32 and integer elements.
  • Against the submission. The headline numbers reproduce within one point on newly trained models (Sudoku, three seeds: full precision 83.3, MXINT4 79.3, per-tensor 0.8; submitted 84.1, 80.1, 0.0). The corrections are listed in part 2.

Setup. Weights and inputs of every linear layer are quantized, embeddings stay in full precision, full test sets. Sudoku TRM: three training seeds, mean ± standard deviation. Maze TRM: mean of two seeds (one standard error about 1.2 points). EqR and HRM: public checkpoints. ARC-AGI-1: one model, pass@2 (one standard error about 2.5 points).

Exact accuracy (%), 4-bit weights W and activations A unless stated

methodSudoku TRM (MLP)Maze TRM (attn)EqRARC-AGI-1HRM Sudokuneeds at inference
full precision83.3 ± 0.783.986.445.055.1
Int8, per-tensor82.3 ± 1.384.280.945.855.3one scale per tensor
per-tensor W and A, unclipped0.8 ± 1.00.00.00.00.0one float scale per tensor
per-channel W, per-tensor A clipped (99.9th percentile)45.3 ± 10.680.26.233.531.1plus a percentile per call
rotation (randomized Hadamard), per-tensor W and A0.6 ± 0.584.20.036.550.6plus a rotation of every linear input
per-channel W, per-token A71.6 ± 4.183.856.742.131.9float scale per channel and token
QuaRot-style: rotation, per-channel W, per-token A79.9 ± 0.884.382.444.253.9plus the rotation
per-channel W, group-32 A81.4 ± 0.884.082.445.154.0float scale per channel and per block
group-32 W and A, float scales81.5 ± 0.683.884.446.454.8float scale per block of 32
MXINT479.3 ± 0.983.980.245.053.8power-of-two scale per block, integer elements
MXFP475.0 ± 1.884.1not run45.449.2power-of-two scale per block, float elements

On Maze and on ARC pass@2 the last six rows are within noise of each other. Except for Int8 and MXFP4, all scales are computed per call and no calibration data is used.

AuthorsOpenReview ↗

General response (2/2): within-model measurements, and what changed against the submission

A supervision step is 21 block calls on Sudoku TRM, and every model runs 16 steps.

Which block calls must be quantized (unclipped per-tensor W4A4 inside a window, full precision elsewhere; exact %)

quantized windowSudoku TRM (3 seeds)Maze TRM (2 seeds)EqRHRM Sudoku
none (full precision)84.3 / 82.6 / 83.084.8 / 83.186.455.1
the last block call and the output heads84.2 / 82.6 / 83.084.8 / 83.186.254.9
the final supervision step83.9 / 82.3 / 82.71.4 / 0.065.536.6
all steps except the final one12.9 / 8.8 / 9.177.1 / 75.56.40.0
every call2.2 / 0.0 / 0.10.0 / 0.00.00.0
  • One call is tolerated. One quantized block call costs at most 0.2 points on every model. On Sudoku TRM, after fifteen quantized steps, 1, 2 and 4 full-precision steps bring exact accuracy back to only 10%, 46% and 67%. On Maze the collapse happens inside one supervision step.
  • Size. On Sudoku the per-tensor per-step error is 145 to 160 against a state norm of 204. Isotropic zero-mean noise with the same per-step error keeps 24.6 / 24.0 / 24.6% on the three seeds, so it already removes about 60 of the 83 points.
  • Structure. The quantizer keeps 2.2 / 0.0 / 0.1% at that error, 22 to 25 points less than the noise (Maze: 0.0 / 0.0 against 30.2 / 16.5). We have not shown that this is because the error repeats across calls.
  • Reuse, tested directly on the weights. A random weight perturbation reused at every step costs 16.7 ± 1.9 points more than one of equal norm redrawn at every step, at the plain per-tensor weight scale (three Sudoku seeds, paired). At the MSE-clipped and per-channel weight scales the gap is 4.1 and 2.2 points (one seed). This is our only direct test of reuse.
  • Proposition 1 stays a qualitative scaffold. With the contraction ratio measured after one injected error (about 0.1) it underestimates the observed divergence by a factor of 1.1 to 1.7, and with the ratio measured after fifteen quantized steps (0.6 to 0.7) the estimate exceeds it.

What changed against the submission

submissionnow
Outlier controls (rotation) failThe control left the largest layer unrotated. With full coverage, rotation with per-channel and per-token scales is level with MXINT4.
The collapse is a coherent biasSize and structure both matter. The stochastic-inference experiments and the coherence diagnostic are no longer read as evidence for a coherent bias, and severity is no longer said to track effective depth.
"Clipping ... ruled out"The submission had no clipping experiment. Clipped rows are in part 1.
Untied-weight controlRepeated on a new model. With 7 times less reuse per weight, per-tensor W4A4 still gives 0.1% (tied 0.0%), and 4-bit weights alone cost 0.5 points against 5.7. The control also changes capacity.
Maze, static per-tensor Int4: 73.5 to 77.77.4 and 79.4 on two seeds. The static quantizer clips the range. Unclipped per-tensor gives 0.0 on both.
Maze, per-tensor FP4: 0.474.8 (75.4 and 74.3 on two seeds). On Sudoku it gives 0.0.
MXFP4 on Sudoku: 72.7 ± 0.475.0 ± 1.8 over three training seeds.
"8-bit preserves accuracy"True on TRM and HRM. On EqR per-tensor Int8 gives 80.9 against 86.4.
Granularity ladder with per-tensor weights: 4.6 and 14.519.4 and 37.0 as means over three training seeds.
EqR per-token activations: 60.656.7 with linear layers only, as in every other row.

New: a device measurement. On a Tenstorrent Blackhole card in its 4-bit block-float format, 4-bit weights are 3.55x smaller than bf16 and raise device throughput by 4.9%, with no accuracy loss (83.1 against 82.6). The card's native conversion of activations truncates and leaves 13.1%. Round-to-nearest in software reaches 80.8 and is about 30 times slower.

AuthorsOpenReview ↗

Update: the revised PDF now includes quantization-aware training and ARC-AGI-2

As announced, we uploaded a second revision. It adds the two result sets that were still running. Both are measured on the newly trained models with the coverage of every other row (weights and inputs of the linear layers). Changes are in blue, and nothing else changed.

Quantization-aware training (Sec. 4.5, App. A.6). The conclusions of the submission hold: weight-LSQ adds 8.0 points on Sudoku and is tied with naive QAT on Maze.

exact %Sudoku TRM (3 seeds)Maze TRM (2 seeds)
per-tensor Int4, post-training0.07.4 and 79.4
naive Int4 QAT74.0 ± 1.2 (submitted 71.8)82.5 ± 0.8 (submitted 82.2)
+ weight-LSQ82.0 ± 0.3 (submitted 81.1)83.4 ± 1.5 (submitted 81.8)
full precision83.3 ± 0.783.9 ± 0.9

One statement is removed. The submission reported activation-LSQ as unstable on Maze (64.9). With linear-layer coverage it gives 83.5 ± 0.3.

ARC-AGI-2 (Sec. 4.6, App. A.12). One newly trained model, 120 evaluation tasks, one standard error of pass@2 about 2.3 points.

pass@2 %nowsubmitted
full precision6.76.25
Int8, per-tensor6.72.9
Int4, per-tensor0.00.0
MXINT44.66.25
MXFP44.65.4

Per-tensor Int4 gives 0.0 and the block formats recover most of the score. MXINT4 is 2.5 tasks below full precision, which is within one standard error. Two statements of the submission are removed: that MXINT4 matches full precision on ARC-AGI-2, and that Int8 loses about half its score there. For the QAT models on ARC we report exact accuracy per augmented puzzle (33.5 against 33.1 in full precision on ARC-AGI-1, 1.9 against 2.1 on ARC-AGI-2), because their evaluation during training does not compute pass@2.

AuthorsOpenReview ↗

Closing comment

We thank the reviewers and the area chair for a discussion that made the paper stronger. A short summary.

Contribution. The paper establishes when 4-bit quantization of recursive reasoning models fails and how to repair it. Activation-scaling granularity governs the collapse, and per-block scaling restores most of the lost accuracy on all five models we test, without retraining. MXINT4 achieves this with integer elements and power-of-two block scales, at accuracy competitive with float block scales and a rotation baseline.

What the discussion added. Every experiment the reviewers asked for was run and is in the revised PDF: matched-coverage baselines with float group scales and rotation on every layer, a weight/activation cross-ablation, same-state and isolated-perturbation measurements, quantization-aware training, HRM and ARC-AGI-2. The main finding held under all of them and now rests on five models from four architectures. We tightened two claims where the new controls called for it: MXINT4 is positioned as competitive with the strongest baselines, with no accuracy advantage over them, and the role of weight reuse in the full collapse is stated as partly resolved.

Reviewers. Reviewers i9cj and Sk9n wrote that their main concerns are addressed and raised their ratings to weak acceptance. The two weaknesses raised by reviewer 3wq4 led to the two most informative additions, the matched fine-grained baselines and the full rotation control. Both are in the main baseline table, and we believe they answer both points.

The TL;DR field cannot be edited during the discussion. Its replacement text is in our reply to reviewer i9cj.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.