acceptodds
Under review as a conference paper at ICLR 2027

Megrez: Reconstructing Intermediate Supervision for Ternary Looped Transformers

Abstract

Ternary looped transformers reuse the same quantized weights across recurrent calls, coupling the approximation of intermediate teacher states. We present Megrez, a calibration framework that adapts the supervision targets to the current student. We reconstruct teacher targets to minimize alignment error while requiring the teacher's unchanged native readout to return its original output on those targets. Each token's normalized teacher states are jointly transformed across recurrent calls by a single orthogonal map. The constrained alignment has an exact low-dimensional solution. Megrez alternates target reconstruction with shared ternary weight updates, accommodating gated and terminal readouts without changing the inference architecture. Across Ouro-1.4B, Parcae-1.3B and LoopFormer, reconstruction reduces squared target error by 20.2–55.5% on average per window at fixed student weights. On Ouro-1.4B, the complete framework lowers C4 perplexity from 45.44 to 29.54 and raises HellaSwag accuracy from 39.83% to 44.92% compared with the restricted TernaryLLM adaptation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
ternary quantization, looped transformers

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CJPROpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies teacher–student state mismatch in recurrent ternary transformers and proposes Megrez to reconstruct teacher targets before calibration. The key technical idea is a neat formulation of target reconstruction as a constrained orthogonal Procrustes problem with a closed-form solution in a low-dimensional subspace. Experiments on several recurrent ternary models show substantial target alignment improvements and promising quantization results.

Critical strengths
  1. The method is well tailored to the shared-weight setting of recurrent ternary transformers, and the idea of adapting the intermediate targets to the current student is intuitive.

  2. The authors formulate target reconstruction as a constrained orthogonal Procrustes problem in the relevant low-dimensional subspace, resulting in a neat closed-form solution.

  3. The experiments provide evidence at both the geometric and end-to-end levels, showing improved target alignment and promising quantization performance. Weaknesses

Critical weaknesses
  1. The paper shows that intermediate teacher and student targets can be mismatched, but it is less clear whether this mismatch is actually a major optimization bottleneck for ternary calibration. More direct evidence connecting target mismatch to calibration difficulty would make the motivation much stronger.

  2. The concept of “readout-preserving” could be stated more precisely. The proposed transformation preserves the constraints imposed by the native readout, but it does not mean that the reconstructed targets are fully equivalent to the original teacher targets. Making this distinction clearer would help better convey what is actually preserved by the reconstruction.

  3. The connection between geometric alignment and quantization gains is still somewhat indirect. The substantial reduction in target reconstruction error does not consistently translate into better PPL or accuracy, as shown by the matched controls in Table 2. This makes it difficult to tell how much the geometric improvement itself contributes to the final quantization performance.

Questions for authors
  1. Is Megrez a general solution for recurrent quantization, or does the constraint need to be designed separately for each readout structure?

  2. How much of the final gain comes from target reconstruction itself, rather than the auxiliary loss or target refresh strategy?

2 comments

Thank you for asking how geometric alignment connects to calibration and which properties a reconstructed target retains.

What the four labels mean. All four methods match the teacher's original output probabilities via Kullback–Leibler (KL) divergence.

MethodIntermediate targets
KL-onlyNone
FixedOriginal teacher states
Free Alignment (Free)Reconstructed teacher states without the readout-preserving constraint
MegrezReconstructed teacher states that give the original teacher output under its frozen readout

Free and Megrez both use one orthogonal map per token to bring the teacher's recurrent states closer to the current student's. The student fits the resulting targets through its shared ternary weights.

W1. Do reconstructed states help the student make predictions? We replaced intermediate states at all token positions in frozen KL-only students, after call two in Ouro and after call four in LoopFormer, then ran each student's remaining calls. Each comparison uses 32 windows per corpus and measures full-vocabulary next-token negative log-likelihood (NLL).

Frozen student and corpusMegrez minus FixedMegrez minus FreeWindows favoring Megrez over Free
Ouro, Colossal Clean Crawled Corpus (C4)−0.0098−0.538432/32
Ouro, WikiText-2−0.0103−0.395832/32
LoopFormer, C4+0.1445−0.017831/32
LoopFormer, WikiText-2+0.1011−0.010530/32

Values are in nats/token. With sparse substitution in Ouro, the advantage over Free was 0.0212 on C4 and 0.0151 on WikiText-2. The Megrez–Fixed interval spans zero on Ouro C4. The LoopFormer student scored best with Fixed, even though Megrez lowered the target mismatch by 53.4% on C4 and 55.9% on WikiText-2. Direct substitution tests what frozen students can do when given these states. Learning to produce them requires changing shared weights, so whether mismatch is a dominant training bottleneck remains open. See Appendix H.4 (p. 26) for the Ouro intervention and the text after Eq. (7) (p. 5) for the distinction between the fixed-state solve and the shared-weight update.

We also measured auxiliary gradients from different recurrent calls at one common Ouro initialization. Each of eight windows supplies six call pairs.

Target methodCall pairs with negative gradient cosine
Fixed3/48
Free23/48
Megrez2/48

The constraint changes gradient agreement in this diagnostic. The result leaves its effect on the final training outcome open.

W2 / Q1. What is preserved, and which readouts are covered? In exact arithmetic, the frozen teacher's native readout returns its original output when applied to the reconstructed targets. One orthogonal map per token preserves the inner products among its normalized teacher states. The individual targets can move, and the recurrent model may be unable to generate the resulting trajectory. The explanation following Proposition 1 (pp. 4–5) states this scope.

The solver is shared once the fixed directions have been derived for each readout. In Ouro, Megrez fixes the gain-adjusted gate direction and original weighted aggregate in normalized coordinates. This preserves the gate weights and the input to the output head. In Parcae and LoopFormer, fixing the final-state sequence preserves the downstream readout, including the input to Parcae's layers after the recurrence.

The first two rows show Ouro's initial snapshot from Figure 4 (p. 8), where relative logit drift was measured in half precision. The last row is a separate frozen-snapshot diagnostic over 64 held-out windows.

Measurement on reconstructed targetsFreeMegrez
Squared state error reduction84.6%20.2%
Relative teacher logit drift45.01%0.048%
Median KL from original teacher output (nats per sampled token)0.9681.23×10−51.23\times 10^{-5}

All four methods match the teacher's original output probabilities using Kullback–Leibler (KL) divergence. KL-only uses only the output loss. Fixed adds the original intermediate targets. Free Alignment (Free) and Megrez use reconstructed targets, with Megrez requiring the teacher's frozen readout to return its original output on those targets.

W3 / Q2. How much of the prediction gain comes from reconstruction? The available controls estimate an increment for each calibration setting. Megrez versus Fixed measures switching from original to periodically reconstructed targets while retaining intermediate supervision. Megrez versus KL-only measures adding the full intermediate supervision, and Megrez versus Free measures the readout constraint at the same refresh rate. The Fixed comparison changes target construction and refresh together. Isolating refresh would require varying its schedule within one target family.

Matched training evidence. Below are both LoopFormer cohorts on 1,024 held-out Colossal Clean Crawled Corpus (C4) documents per cohort after 256 matched updates. Each cell gives mean Megrez-minus-control negative log-likelihood (NLL) in 10−310^{-3} nats/token, with wins out of three training blocks.

CohortInference callsvs KL-onlyvs Fixedvs Free
Existing checkpoints4−0.239 (2/3)−0.305 (3/3)−0.441 (3/3)
Existing checkpoints8−0.040 (1/3)−0.156 (2/3)−0.092 (3/3)
New training blocks4+0.415 (1/3)+0.203 (1/3)−0.168 (1/3)
New training blocks8−0.091 (2/3)−0.217 (2/3)−0.183 (2/3)

The eight-call mean against Fixed favored Megrez in both cohorts. At four calls the sign changed. In the new four-call cohort, adding fixed intermediate supervision increased mean NLL by 0.212 in the table's units (Fixed minus KL-only). The increments vary by cohort and depth, so a single fraction of the total calibration gain would be misleading (Table 2 and Appendix H.3).

Ouro one-hop color tracking. At four calls, Megrez lowered one-hop candidate NLL relative to Fixed in both earlier 256-update blocks, by 0.09107 nats/question on average. The rate of answering both counterfactual questions correctly rose by 5.08 percentage points. We evaluated both new training blocks on a separate set of 256 graph pairs. Megrez had lower NLL than all three controls in one block and higher NLL than all three in the other. The second result persisted after retraining on 4090D GPUs. Appendix H.4 gives the earlier mean and the differences for each new block.

What changes when we hold the evaluation data fixed? We trained Ouro for the first 32 updates of the original 256-update schedule. At initial auxiliary weight 0.100.10, we evaluated the original and newly trained blocks on the same scoring device. The table reports differences in nats/token.

Training blockWindowsvs KL-onlyvs Fixedvs Free
Development48 development−0.05507−0.00480−0.03624
Development128 held-out−0.03321−0.00206−0.02435
New 148 development+0.00010+0.00138+0.00339
New 1128 held-out+0.00021+0.00086+0.00383
New 248 development+0.00789−0.00231+0.00410
New 2128 held-out+0.03719−0.00587+0.00679

All nine contrasts kept their sign across evaluation sets. Rankings changed across training blocks on the same evaluation set. The roles of training-window selection, order, and numerical variation remain unresolved (Appendix H.6, p. 27).

Reviewer_Q25TOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a constrained orthogonal alignment formulation that accommodates two distinct readout mechanisms and admits a low-dimensional solution. The experiments span three recurrent architectures and distinguish target reconstruction quality from downstream predictive performance.

Critical strengths

Megrez formulates target reconstruction as a constrained orthogonal alignment problem and derives an exact low-dimensional solution via orthogonal Procrustes. The formulation explicitly accounts for different readout structures, including Ouro's gated readout, while preserving the relevant teacher-output constraints.

Critical weaknesses

The theoretical guarantees are narrower than the presentation of the method may suggest. The Procrustes formulation establishes an exact constrained solution and guarantees non-increasing reconstruction error at a fixed student state. However, this does not imply that alternating target reconstruction and weight optimization improves the final predictive objective(Eq. 7).

Questions for authors

Provide a more detailed description of Table 2. Provide a experiment that isolates the effect of Megrez's target reconstruction, Specifically, comparing KL-only, fixed-target, free-alignment, and Megrez under identical settings would clarify how much of the gain is attributable to the proposed target reconstruction mechanism.

2 comments

Thank you for asking us to clarify what Megrez guarantees, explain Table 2, and isolate the benefit of target reconstruction.

1. What do the four methods test?

All methods use KL distillation to match the teacher's original output probabilities. They differ in how they supervise intermediate states:

MethodIntermediate targetsPurpose
KL-onlyNoneOutput distillation alone
FixedOriginal teacher statesIntermediate supervision without reconstruction
FreePeriodically reconstructed targets, without readout preservationUnconstrained alignment
Megrez (ours)Periodically reconstructed targets preserving the teacher's logitsReadout-preserving alignment

These controls identify the contribution of our design. Megrez vs Fixed tests reconstruction together with target refresh; Megrez vs Free isolates readout preservation because the alignment objective and refresh rate are identical. KL-only shows the result without intermediate supervision.

2. What does the theory guarantee?

Megrez exactly solves target reconstruction for a fixed student; it does not claim to solve the subsequent weight-training problem.

Given fixed teacher states, student states, and readout constraints, Proposition 2 finds the feasible orthogonal map with the smallest squared state error. Since the identity map is feasible, the reconstructed targets cannot have greater error than the original states. Proposition 1 further guarantees that the frozen teacher readout recovers the original teacher logits in exact arithmetic.

This is the theoretical contribution: Megrez makes targets as close as possible to the current student while preserving the teacher's original predictions. Free does not impose this constraint.

Equation (7) then updates shared ternary weights using both target supervision and output KL (§3.4). These updates change the student states, so the fixed-state result does not guarantee lower final predictive loss or convergence. We have clarified this distinction in the revision and evaluate predictive benefits separately.

3. How should the original Table 2 be read?

The two panels use different metrics and training cohorts (full results: revised Table 16, p. 25).

PanelMetric and evaluation setTraining cohort
APerplexity on WikiText-2 and C4Three original seeds
BMean per-document NLL on 256 C4 documentsTwo additional Ouro/Parcae seeds; three separate four-method LoopFormer reruns

The panels should not be pooled as one experiment. In the original LoopFormer row, "Megrez (0.241)" against KL-only means an NLL reduction of 0.241 × 10⁻³ nats/token. Revised Table 16 uses signed Megrez-minus-control differences to make this clear.

4. Does reconstruction improve final prediction under matched settings?

We added two LoopFormer cohorts with three matched runs per method, 256 calibration updates per run, and 1,024 held-out C4 documents per cohort. The replication uses new calibration subsets and data orders (revised Table 2, p. 8; Appendix H.3).

The table reports mean Megrez-minus-control NLL at eight inference calls, in 10⁻³ nats/token. Negative values favor Megrez; parentheses give the number of runs (out of three) favoring Megrez.

Cohortvs KL-onlyvs Fixedvs Free
Original−0.040 (1/3)−0.156 (2/3)−0.092 (3/3)
Replication−0.091 (2/3)−0.217 (2/3)−0.183 (2/3)

Megrez achieves lower mean NLL than Fixed and Free in both cohorts, including the replication. This demonstrates a setting where reconstruction-and-refresh improves on fixed targets, and readout preservation improves on unconstrained alignment.

For Megrez vs Fixed, paired-document 95% intervals were [−0.275, −0.031] and [−0.313, −0.120] in the table's units. They reflect document-level variation for fitted models, not uncertainty across training runs.

Scope of the claim. The mean advantage is small and depends on inference depth: four-call results are less consistent. In the original three-seed Ouro C4 comparison, Megrez and KL-only were nearly tied (29.5358 vs 29.5221 perplexity), and some other matched comparisons favor Fixed or Free (Table 16, Appendix H.3). We report these outcomes rather than claim universal superiority.

Conclusion. Megrez provides an exact, readout-preserving solution to fixed-state target reconstruction, and our new matched experiments show that it can improve final prediction under identifiable conditions.

The theoretical guarantee establishes the quality and consistency of reconstructed targets. The eight-call LoopFormer experiments demonstrate that this design can also improve predictive performance, including under new calibration conditions. We distinguish this guaranteed reconstruction property from the empirical, setting-dependent outcomes of weight calibration.

Q2.What do the matched comparisons tell us?

Megrez can improve prediction over both fixed and unconstrained targets, but the gain depends on the model, inference depth, and calibration run. We use three controls to identify where the benefit comes from: KL-only learns from teacher outputs alone; Fixed also uses original teacher states; Free reconstructs intermediate targets without preserving the teacher's readout. Megrez reconstructs targets while preserving that readout.

1. Four methods under matched training conditions

Within each matched pairing, methods share the initialization, calibration data and order, optimizer, and 256 updates. Fixed, Free, and Megrez use the same auxiliary-loss normalization and weight schedule. Free and Megrez refresh targets every eight updates.

The table reports mean Megrez minus control next-token NLL in 10⁻³ nats/token. Negative values favor Megrez. Parentheses show the number of runs favoring Megrez.

Model / evaluationCallsvs KL-onlyvs Fixedvs Free
Ouro, 256 C4 docsNative+6.465+4.657+7.396
Parcae, 256 C4 docsNative+18.979+25.613+35.294
LoopFormer, 256 C4 docsNative−0.241 (3/3)−0.014 (1/3)−0.403 (2/3)
LoopFormer, original, 1,024 held-out docs4−0.239 (2/3)−0.305 (3/3)−0.441 (3/3)
LoopFormer, original, same docs8−0.040 (1/3)−0.156 (2/3)−0.092 (3/3)
LoopFormer, new training, 1,024 held-out docs4+0.415 (1/3)+0.203 (1/3)−0.168 (1/3)
LoopFormer, new training, same docs8−0.091 (2/3)−0.217 (2/3)−0.183 (2/3)

These results identify a replicated predictive benefit: at eight LoopFormer calls, Megrez achieves lower mean NLL than Fixed and Free in both the original and new-training cohorts. Megrez versus Fixed tests reconstruction together with target refresh; Megrez versus Free isolates readout preservation at the same refresh rate.

The benefit is setting-dependent. Ouro and Parcae favor controls in these comparisons, and the four-call replication changes sign against Fixed. The 256-document LoopFormer interval against Fixed spans zero (−0.000191 to +0.000162 nats/token).

Table 1 separately evaluates the full Parcae recipe: C4 perplexity falls from 70.86 at the ternary initializer to 28.59 after Megrez calibration (KL-only: 28.58). This demonstrates effective calibration without attributing the entire gain to reconstruction.

2. Does readout preservation help on a downstream task?

We conducted a matched Ouro single-needle retrieval pilot using 32 updates and 128 development questions.

MethodAnswer NLL ↓Exact match
KL-only1.83385122/128
Fixed1.70413121/128
Free2.15306107/128
Megrez1.75095118/128

Megrez clearly improves over Free in this pilot: it reduces answer NLL by 0.40211 nats/answer and retrieves 11 more answers. This provides direct evidence that readout preservation can improve prediction beyond geometric alignment alone.

Fixed achieves lower NLL, while KL-only achieves higher exact match. Across 30 completed 32-update development pilots, Megrez lowers NLL versus Fixed in 15, versus KL-only in 10, and versus Free in 11. Thus, the retrieval result establishes a benefit over Free in this setting, not universal superiority.

3. Does auxiliary weighting affect the benefit?

In a matched Ouro C4 screen, Fixed, Free, and Megrez each achieved their lowest development NLL at an initial auxiliary weight of 0.10 among three tested values. Megrez reached 3.40127 nats/token after the first 32 updates. The table reports Megrez-minus-control NLL in nats/token.

Training blockC4 windowsvs KL-onlyvs Fixedvs Free
Original48 development−0.05507−0.00480−0.03624
Original128 held-out−0.03321−0.00206−0.02435
New 1128 held-out+0.00021+0.00086+0.00383
New 2128 held-out+0.03719−0.00587+0.00679

With this matched weight, Megrez outperforms all three controls on both development and held-out windows for the original training block. This demonstrates that its predictive benefit extends beyond the data used for weight selection.

The advantage does not persist across both new training blocks: KL-only and Free achieve lower NLL, while comparisons with Fixed are mixed (Appendix H.6, p. 27)

Conclusion. Megrez's contribution is not a claim of universal predictive superiority. It provides an exact, readout-preserving reconstruction method for a fixed student, and the matched experiments demonstrate settings where this design improves prediction.

The eight-call LoopFormer replication and the retrieval improvement over Free provide concrete evidence of its value. The less favorable comparisons clarify when that benefit is limited, without weakening the mathematical reconstruction guarantee.

Reviewer_s8v2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Megrez, an intermediate-supervision method for ternary looped Transformers. It aligns teacher states from repeated applications of shared weights with student states using constrained orthogonal transformations. The constraints preserve the teacher's native readout, and the resulting targets are used as auxiliary supervision alongside output-level KL distillation. Experiments cover Ouro, Parcae, and LoopFormer, including matched training controls.

Critical strengths
  1. Well-motivated mathematical construction. The readout-preserving constraints address the fact that changing intermediate states can alter a looped model's final output, especially with gated aggregation. The reduced Procrustes formulation is clearly derived and computationally tractable.

  2. Careful controlled evaluation. The paper includes matched KL-only, fixed-target, and unconstrained-alignment comparisons, as well as additional seeds and held-out documents. Reporting results that do not favor Megrez makes the empirical findings more informative.

Critical weaknesses
  1. The benefit of target reconstruction is not established. Despite substantial improvements over some external quantization baselines, the matched KL-only control performs almost identically to Megrez. For example, on Ouro, KL-only achieves a C4 perplexity of 29.5221 versus 29.54 for Megrez. Fixed targets and unconstrained alignment also outperform Megrez in several matched comparisons. It is therefore unclear what predictive benefit can be attributed to the proposed reconstruction itself.

  2. The readout-preserving constraint is not sufficiently justified as a training objective. Preserving the teacher's output and reducing state mismatch are mathematically meaningful, but neither guarantees that the targets are easier to learn with shared ternary weights. Since unconstrained alignment sometimes performs better, the experiments do not establish why the additional constraint should improve student optimization.

Questions for authors
  1. Independent contribution: Given the nearly identical KL-only results and the cases where fixed or unconstrained targets outperform Megrez, in which settings does the proposed reconstruction provide a consistent improvement in final prediction quality?

  2. Readout-preserving constraint: What evidence justifies imposing readout preservation rather than using unconstrained alignment? Can the authors explain when this constraint helps or hurts the optimization of shared ternary weights?

  3. Optimization mechanism: Why does the reduction in intermediate-state mismatch not consistently translate into better predictions? Could gradient alignment or conflict across recurrent steps help explain this result? The reported auxiliary-gradient contribution is small; how sensitive are the conclusions to its weighting?

3 comments

Thank you for asking us to isolate what target reconstruction adds to ternary calibration.

W1 / Q1. When does reconstruction improve prediction? Megrez improves average predictive performance over Fixed and Free at eight inference calls in LoopFormer, including in a replication with new calibration data and training orders. These results provide evidence that reconstructing intermediate targets while preserving the teacher's readout can improve prediction. The improvement is modest, but it is observed in both evaluation cohorts.

What do the four methods compare? All four methods use Kullback–Leibler (KL) distillation to match the teacher's original output probabilities. They differ in how they supervise the student's intermediate states:

MethodIntermediate targetsWhat it does
KL-onlyNoneMatches only the teacher's output probabilities
FixedOriginal teacher statesAdds intermediate-state supervision without reconstruction
FreeReconstructed targetsPeriodically adjusts targets to the student without preserving the teacher's readout
Megrez (ours)Readout-preserving reconstructed targetsPeriodically adjusts targets to the student while preserving the teacher's original readout

These controls address two questions:

  • Megrez vs. Fixed: Does reconstructing and refreshing intermediate targets improve prediction compared with using fixed teacher states?
  • Megrez vs. Free: Does preserving the teacher's readout improve prediction when both methods use the same alignment objective and target refresh rate?

Matched LoopFormer evaluation. We evaluated two cohorts, each with three matched runs per method and 256 calibration updates per run. Each cohort was tested on 1,024 held-out Colossal Clean Crawled Corpus (C4) documents. The replication cohort used new calibration subsets and data orders (Table 2 and Appendix H.3).

The table reports mean differences in next-token negative log-likelihood (NLL) at eight inference calls, in units of 10−310^{-3} nats/token. Negative values favor Megrez. Parentheses indicate how many of the three runs favored Megrez.

CohortMegrez − KL-onlyMegrez − FixedMegrez − Free
Original−0.040 (1/3)−0.156 (2/3)−0.092 (3/3)
Replication−0.091 (2/3)−0.217 (2/3)−0.183 (2/3)

What do the results establish? Three findings are particularly relevant.

  1. Reconstruction improves over fixed targets. Megrez achieved lower mean NLL than Fixed in both cohorts, winning in two of three runs in each. The paired-document 95% intervals for Megrez minus Fixed were [−0.275, −0.031] and [−0.313, −0.120], respectively, in the table's units. Both favor Megrez, although these intervals reflect document-level variation for the fitted models rather than uncertainty across training runs.

  2. Preserving the readout provides an additional benefit. Megrez also achieved lower mean NLL than Free in both cohorts. Since both methods use the same reconstruction objective and refresh rate, this comparison directly evaluates the value of the readout constraint.

  3. The improvement extends to new calibration conditions. Megrez maintained its mean advantage over Fixed and Free when we changed the calibration subsets and data orders, suggesting that the eight-call result is not specific to the original training setup.

The original three-seed Ouro experiment showed nearly identical C4 perplexity for Megrez and KL-only (29.5358 vs. 29.5221, Table 16A). Results at four inference calls and in other model comparisons are also less consistent (Table 2, Table 16B, and Appendix H.3). We therefore make a specific rather than universal predictive claim.

Conclusion. The matched experiments identify eight-call LoopFormer as a setting where Megrez's readout-preserving target reconstruction improves average predictive performance over both fixed and unconstrained targets, including under new calibration conditions. This empirical benefit complements the method's central theoretical contribution: for a fixed student, Megrez exactly solves for the closest feasible intermediate targets while preserving the teacher's native readout. The theoretical result does not depend on obtaining a predictive gain in every training run.

W2 / Q2. Why preserve the teacher's native readout?

Megrez makes the teacher's intermediate targets easier for the ternary student to learn while preserving the teacher's original predictions. Our experiments show why this matters: preserving the readout avoids conflicting supervision and can improve the student's predictions, both before and after calibration.

1. Why is readout preservation necessary?

Free Alignment (Free) and Megrez use the same orthogonal alignment objective, target refresh rate, and Kullback–Leibler (KL) distillation (§4.3). Both adjust the teacher's intermediate targets to better match the student. They differ in one important way:

  • Free moves the targets closer to the student without preserving the teacher's readout. As a result, the reconstructed targets may produce different logits from those used in KL distillation, potentially giving the student conflicting supervision.
  • Megrez (ours) also moves the targets closer to the student, but ensures they still produce the teacher's original logits. Proposition 1 establishes an exact solution to this constrained reconstruction problem for a fixed student.

We compared both methods at Ouro-1.4B's initial snapshot, using identical student weights and averaging over 32 windows (Figure 4 and Table 12).

MetricFreeMegrez
Target-error reduction (higher is better)84.6%20.2%
Relative teacher-logit drift (lower is better)45.01%0.048%

These results demonstrate the value of Megrez's constraint. Free achieves greater target alignment, but its reconstructed targets substantially change the teacher's logits. Megrez still reduces the squared target error by 20.2%, while keeping logit drift at just 0.048%, compared with 45.01% for Free.

Megrez brings the targets closer to the student without sacrificing consistency with the teacher's original predictions. This is the property our readout constraint is designed to guarantee.

2. Does this constraint improve prediction?

We tested the effect directly using a frozen Ouro student. After its second inference call, we replaced selected intermediate states with either Free or Megrez targets, then continued the remaining calls without updating any model weights.

Compared with Free, Megrez reduced next-token negative log-likelihood (NLL; lower is better) by:

  • 0.0212 nats/token on C4
  • 0.0151 nats/token on WikiText-2

Each dataset contributed 32 evaluation windows (Appendix H.4).

Both datasets favor Megrez under identical student weights. This is important because the comparison isolates the effect of the reconstructed targets from further weight training. It shows that preserving the teacher's readout can improve subsequent predictions, rather than merely satisfying a mathematical constraint.

3. Does the benefit remain after calibration?

We further evaluated Megrez and Free after calibration, using two separate protocols.

EvaluationMegrez exact matchFree exact matchMegrez − Free NLL (nats/answer)
NIAH Single, 128 development prompts, 32 updates118/128107/128−0.40211
bAbI QA4, 1,000 official test questions, 256 updates, four calls331/1,000335/1,000−0.03068

The NIAH results provide clear evidence of a predictive benefit after calibration. Megrez answered 11 more questions correctly than Free, improving exact-match accuracy by 8.59 percentage points. It also reduced answer NLL, with a paired-question 95% interval of [−0.53561, −0.28873] nats/answer.

On bAbI QA4, Megrez again achieved lower answer NLL, although Free answered four more questions correctly. These findings show that the readout constraint can improve both prediction quality and task accuracy, as demonstrated on NIAH, while its effect varies across tasks.

We also acknowledge the limits of these results: Fixed and KL-only achieved higher exact match on NIAH, and Free achieved lower C4 NLL in both new training blocks at the development-selected weight (Comment 3 and Appendix H.6). We therefore do not claim that readout preservation improves accuracy in every setting.

Readout preservation is a central part of Megrez, not an arbitrary restriction. Unconstrained alignment can make intermediate targets easier to match but change the teacher's original predictions, creating a potential conflict with output distillation. Megrez addresses this problem with an exact constrained reconstruction.

Evidence supports:

  1. Better target consistency: Megrez reduces intermediate target error while keeping teacher-logit drift at only 0.048%.
  2. Better predictions without additional training: Megrez outperforms Free in the frozen-student intervention on both C4 and WikiText-2.
  3. Measurable gains after calibration: On NIAH, Megrez improves exact-match accuracy over Free by 8.59 percentage points and achieves lower answer NLL.

Q3. Why does better state alignment not always improve prediction?

Better state alignment does not guarantee better predictions because training depends on more than the distance between states. Our experiments identify two factors: gradient agreement across recurrent calls and the weight assigned to intermediate-state supervision.

In Ouro, the same ternary weights are reused across recurrent calls. Each call produces a training gradient, but these gradients may point in conflicting directions. Thus, bringing intermediate targets closer to the student does not necessarily make the shared weights easier to optimize.

1. Do gradients from different recurrent calls conflict?

We examined 48 pairs of call-wise gradients across eight windows at the same Ouro initialization. A negative cosine indicates opposing gradient directions.

Target methodOpposing gradient pairs
Fixed3/48
Free23/48
Megrez (ours)2/48

Megrez shows substantially less gradient conflict than Free: only 2 opposing pairs versus 23. Free achieves greater geometric alignment but produces more conflicting training directions across calls.

Megrez's readout-preserving targets are associated with more consistent cross-call gradients at this snapshot. Fixed also shows little conflict (3/48), so this diagnostic suggests a possible mechanism rather than proving a causal effect on final predictions or better alignment with the output-KL gradient.

2. Does the auxiliary-loss weight affect prediction?

Calibration combines KL distillation, which matches the teacher's final predictions, with an auxiliary loss that matches intermediate targets. If the auxiliary weight is too small, reconstructed targets may have limited influence on training.

We conducted a matched ten-configuration Ouro C4 screen, varying the initial auxiliary weight while keeping initialization and calibration order fixed. Each run used the first 32 updates of the original 256-update schedule. Selected results follow.

Initial weightFixed NLL ↓Free NLL ↓Megrez NLL ↓Megrez auxiliary/KL gradient ratio
0.013.4523273.4500013.4521360.88%
0.033.4152703.4570833.4485562.31%
0.103.4060693.4375053.4012685.05%

The KL-only reference NLL was 3.456336. Gradient ratios are the maximum sampled weighted auxiliary/KL gradient-norm ratios before optimizer preconditioning.

At an initial weight of 0.10, Megrez achieves the lowest NLL among all four methods. Increasing its weight from 0.03 to 0.10 reduces development NLL by 0.04729 nats/token, while the maximum sampled auxiliary/KL gradient ratio increases from 2.31% to 5.05%.

This directly addresses the reviewer's concern about weak auxiliary supervision. Even a relatively small auxiliary-gradient contribution can influence predictive performance. The original weight of 0.03 is not uniformly optimal, and Megrez's benefit depends partly on how reconstructed targets are weighted during calibration.

3. Does the improvement extend to held-out data?

We evaluated the original checkpoint fitted at an initial auxiliary weight of 0.10 on 128 C4 windows excluded from weight selection.

ComparisonMegrez NLL reduction (nats/token)
vs KL-only0.03321
vs Fixed0.00206
vs Free0.02435

Megrez retains lower NLL than all three controls on held-out data. This shows that the improvement in this fitted block extends beyond the development windows used for weight selection.

Two additional independently trained blocks did not reproduce the advantage over Free, and comparisons against Fixed had mixed signs (Appendix H.6). The gain therefore remains sensitive to training conditions.

Conclusion. Megrez's contribution goes beyond reducing intermediate-state error. It reconstructs targets closer to the ternary student while preserving the teacher's original readout, with an exact solution for the fixed-state problem.

Our experiments support three findings:

  1. Better gradient agreement: Megrez shows substantially less cross-call gradient conflict than Free at the examined initialization.
  2. Effective calibration: With an initial auxiliary weight of 0.10, Megrez achieves the lowest development NLL among the matched methods.
  3. Held-out predictive gains: Megrez maintains its advantage over all three controls on held-out C4 windows for the original fitted block.

Together, these results explain why geometric alignment alone is insufficient and show that Megrez's readout-preserving reconstruction can improve prediction under suitable calibration conditions. The gradient analysis suggests a mechanism, not a causal guarantee. The theoretical contribution remains valid regardless of variation in downstream training outcomes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.