acceptodds
Under review as a conference paper at ICLR 2027

Latent Depth Recursion Reveals Post-Training Dynamics in Multimodal LLMs

Abstract

Recent work has shown that frozen large language models (LLMs) can correct their own predictions by re-executing a block of their own layers. Prior methods exploit this recursive structure at inference time, either by searching for the right block to repeat or by training a router to select one. All of these evaluations target pretrained or instruction-tuned LLMs. Every deployed model in practice, however, undergoes a longer post-training pipeline that adds multimodal adaptation and reinforcement learning (RL). In this work, we investigate whether the recursive structure that these methods rely on survives this pipeline. To this end, we reframe the recursive capability as a measurable budget: the fraction of instances that the default forward pass gets wrong but that some recursion route gets right. We compute this budget by exhaustively evaluating every route in a small, closed recursion family. Tracking it across the training stages of several LLM and multimodal LLM (MLLM) families, we find that the budget is large in pretrained models and shrinks consistently as post-training proceeds. Both supervised fine-tuning (SFT) and RL absorb the recursive structure into the default forward pass, and their updates align with where recursion was previously helpful. As a result, little budget remains for test-time routing to exploit, which explains why existing methods fail on fully post-trained models. Instead of recovering recursion at test time, we propose recursion-aware post-training, which deliberately spends the remaining budget by adding a fixed per-category recursion route to the training forward pass. At the same parameter count, our scheme outperforms standard GSPO post-training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Latent depth Recursion, Post-training, MLLMs

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LGdCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether the “latent recursion” effect, where re-executing a contiguous block of layers in a frozen LLM fixes wrong answers, survives a full post-training pipeline. The authors enumerate a closed family of L(L+1)/2+1 routes, each repeating one block once, on every instance. From this they define a recursion budget B (the fraction of instances the default pass gets wrong but some route gets right, equal to the instance-oracle gap), a fix density ρ_fix, and transitions between “always-correct”, “in-budget” and “out-of-reach” categories across checkpoints. Across 21 checkpoints from 8 MLLM families, B is large, while the best fixed route gains at most +2.7 and transfers poorly across split halves. Along released Qwen3 → InternVL3.5 intermediate checkpoints, B and ρ_fix generally fall, most sharply at instruction tuning and multimodal adaptation. RL stages never turn an out-of-reach instance into an always-correct one, whereas SFT stages do. For newly learned instances, weight-update norms, logit-lens shifts and hidden-state shifts from post-training align with those of the best recursion route at the earlier checkpoint, more than for a control set. Three test-time methods (POLAR, Dr.LLM, TFLT) give no gain on InternVL3.5-2B. As an alternative, the paper proposes recursion-aware GSPO, which trains through a fixed per-category route chosen on a 2k-instance subset. This method beats standard GSPO in 5 of 6 categories on InternVL3.5-2B and in 14 of 18 cells across three more backbones, supported by compute, route-choice and cross-category ablations.

Critical strengths
  1. Enumerating a closed route family instead of searching or sampling makes B, ρ_fix and the transition matrices exactly comparable across checkpoints. This avoids the search noise that limits prior work and gives a well-defined upper bound for any route selector.

  2. The matched-cardinality comparisons (temperature sampling, Monte-Carlo routes), split-half transfer, and reproductions of three routers from official code are informative. The finding that test-time routing fails on fully post-trained MLLMs is useful.

  3. Recursion-aware GSPO adds no parameters at 1.06–1.23× step time. It wins in most backbone–category cells over three seeds, and the inference-only, two-epoch, and random-route ablations suggest the gain comes from route placement rather than extra compute.

Critical weaknesses
  1. The out-of-reach sets are only 0.7–4.7% of small benchmarks and are pooled across scales, so a zero count may reflect a few dozen instances. The result is also expected under the generic sharpening view and does not specifically implicate recursion.

  2. Both the target set and the “best route” are selected to move toward the correct answer, so high logit-space cosine is nearly guaranteed by construction. A random-correct-route or sampling-based null is missing, the control set’s best route is undefined, and the representation and weight-space signals are small or inconsistent across scales.

  3. Section 3 argues that useful routes are instance-specific and that fixed routes don’t transfer, yet the method trains through a fixed per-category route chosen on a few hundred instances. That selection is noisy (Top-2 underperforms GSPO; a random route costs −7.56), and the claimed “re-entry-stable representation” mechanism is never measured.

  4. Dr.LLM collapsing to ~20% while never choosing the default route suggests an adaptation failure rather than low ρ_fix. The routers are never tested on earlier MLLM checkpoints, so the causal claim that post-training causes routing to fail is untested.

Questions for authors
  1. Please give absolute instance counts, per scale, for the out-of-reach → always-correct cells under RL and SFT, plus a significance estimate. How large is the out-of-reach set per checkpoint before pooling?

  2. For section 5, what is the cosine similarity between Δf and the shift of a random correct route, or of a correct temperature sample, on the target set? How exactly is the “best recursion route” defined for control-set instances?

  3. Do POLAR/Dr.LLM/TFLT recover gains on earlier InternVL3.5 checkpoints (Pretrained/Instruct) in your multimodal adaptation? Without this, the failure on the final checkpoint cannot be attributed to post-training rather than to the adaptation.

4 comments
AuthorsOpenReview ↗

Part 1/2: Response to Reviewer LGdC

We thank the reviewer for a careful review, and for finding the measurement, the router reproductions, and the training method useful. We address each question below.

W1: The out-of-reach sets are only 0.7–4.7% ... so a zero count may reflect a few dozen instances. The result is also expected under the generic sharpening view and does not specifically implicate recursion.

Table R1. Out-of-reach instances and their flow to always-correct (pooled over scales)

Benchmark and transitionPhaseOut-of-reach at the earlier checkpointFlow to always-correct95% upper bound if zero
MMStar, InternVL3.5 Pretrain → InstructSFT22527 (12.0%)
MMStar, InternVL3.5 Instruct → MPORL26201.1%
MMStar, InternVL3.5 MPO → GSPORL27101.1%
MMStar, InternVL3 Pretrain → InstructSFT15013 (8.7%)
MMStar, InternVL3 Instruct → MPORL14002.1%
ARC, Qwen3-Base → Qwen3LLM post-training6004.9%
ARC, Qwen3 → InternVL3.5 PretrainMultimodal continual pretraining5020 (40.0%)
ARC, InternVL3.5 Pretrain → InstructSFT3208.9%
ARC, InternVL3.5 Instruct → MPORL12802.3%
ARC, InternVL3.5 MPO → GSPORL10802.7%

Table R1 gives the counts for every transition on both benchmarks. Each RL transition starts from 108 to 271 out-of-reach instances. For a zero count out of n, the table reports the exact one-sided 95% upper bound. On MMStar, the SFT rates are far above the RL bounds: 12.0% vs. 1.1% for InternVL3.5, and 8.7% vs. 2.1% for InternVL3.

We agree that the zero flow under RL is what the sharpening view predicts [1,2]. The link itself is our own finding, though. Recursion and best-of-N sampling reach largely the same instances (Jaccard 0.86 to 0.97, Table R2 in our reply to r217), so the recursion budget is the same headroom that sharpening consumes. What is specific to recursion is location in depth. For each instance and task, recursion shows which depth band carries the headroom, which sampling cannot (Figs. 14 and 15).

W2: ... A random-correct-route or sampling-based null is missing, ... and the representation and weight-space signals are small ....

Table R2. Alignment between the post-training shift and each intervention. InternVL3.5-2B, MPO → GSPO, MMStar, mean over layers 21 to 27, mean ± standard error.

Set and interventionLogit cosineLogit cosine, answer options removedHidden-state cosine
Target, best recursion route (original)+0.66 ± 0.03+0.14 ± 0.03+0.04 ± 0.02
Control, best recursion route (original)−0.01 ± 0.02+0.07 ± 0.01−0.03 ± 0.01
Target, non-recursive fix (best-of-N, N = 407)+0.75 ± 0.02+0.11 ± 0.04n/a
Target, random correct route+0.59 ± 0.03+0.09 ± 0.03−0.01 ± 0.02
Target, routes shuffled across instances−0.05 ± 0.06+0.07 ± 0.03−0.03 ± 0.02
Control, route matched in length and position−0.17 ± 0.02+0.01 ± 0.01−0.05 ± 0.01

For both sets, the best recursion route is the correct non-default route with the highest average log-likelihood of the ground-truth answer tokens. Table R2 tests the alignment against the two nulls the reviewer suggested, plus two more:

  • A random correct route tests whether any correct route aligns.
  • Routes shuffled across instances test whether any recursion aligns.
  • A control route matched in length and position tests whether the target-versus-control gap comes from different routes.
  • Best-of-N sampling (N = 407) serves as the non-recursive fix. Sampling does not change the hidden states at a fixed position, so it has no hidden-state shift.

The target-versus-control gap does not come from the route choice. With routes matched in length and position, control alignment drops further, to −0.17. The alignment also requires the route to fix the instance, since shuffled routes show none (−0.05). A random correct route gives 0.59, and best-of-N sampling gives 0.75, more than the best recursion route (0.66). The higher value for sampling is expected, since RL directly reinforces correct sampled rollouts. Most of the logit alignment therefore reflects shared correction on the budget that recursion and sampling share (Reviewer r217 Table R2).

We also tested logit alignment without answer options so it becomes free-form answer now. The best recursion route on Target is 0.14, the highest value in the table and clearly above both control rows (0.07 and 0.01). As the reviewer noted, the hidden-state alignment is small but what matter here is the relative difference. Target with best recursion route outperform all null settings (0.04 vs. -0.03 & -0.05) and this pattern prevails across all pattern we tested (1B,2B,4B, and 8B). We will update Sec 5. according to this results.

AuthorsOpenReview ↗

Part 2/2: Response to Reviewer LGdC

W3: Section 3 argues that useful routes are instance-specific and that fixed routes don’t transfer, yet the method trains through a fixed per-category route chosen on a few hundred instances. That selection is noisy (Top-2 underperforms GSPO; a random route costs −7.56), and the claimed “re-entry-stable representation” mechanism is never measured.

We agree that a fixed route cannot recover an instance-specific budget, and we no longer describe the method that way. However, Sec. 3 and Sec. 6 test different things. Sec. 3 applies a fixed route to frozen weights at test time, while Sec. 6 trains through the route so the weights adapt to it. The two results are consistent. Applied at test time without training, the selected route stays below the baseline in five of six categories (Table 2, Rec. Baseline), as Sec. 3 predicts. Trained through, the same route beats standard GSPO in five of six categories. The method therefore uses the route as an inductive bias during training rather than applying it to frozen weights.

Route sensitivity. We agree that the ranking below the top-1 is noisy. The original random route r6,12 is seven layers long, while the top-1 route is one layer. Repeating a random long block can be disruptive, since few routes are useful (Fig. 14) and post-trained layers have more distinct roles (Fig. 9). We therefore add random routes matched in length to the top-1 route.

Table R3. STEM route choice with length-matched random routes (InternVL3.5-2B)

RouteLayersChange vs standard GSPO
Top-1, r19,191+1.88
Top-2, r1,11−0.30
Top-3, r10,145+0.58
Random, r6,127−7.56
Random single layer, r26,26 (new)1+0.48
Random single layer, r7,7 (new)1+0.14

Most of the −7.56 loss comes from the random route's length. Random single-layer routes stay close to standard GSPO (+0.48 and +0.14). At matched length, the top-1 route beats them by 1.4 to 1.7 points, so the top of the depth map carries real information about where to loop. The top-2 route was chosen from only 333 STEM instances, and we expect a larger selection set to make the ranking below the top-1 more reliable.

Re-entry mechanism. We measured how similar the first-pass and second-pass outputs of the repeated block are, using linear CKA as in Appendix C.2. Recursion-aware GSPO raises this similarity from 0.728 to 0.749. With the same route applied, standard GSPO lowers it to 0.710. The two trainings move it in opposite directions, which supports the re-entry mechanism.

W4: Dr.LLM collapsing to ~20% while never choosing the default route suggests an adaptation failure rather than low ρ_fix. The routers are never tested on earlier MLLM checkpoints, so the causal claim that post-training causes routing to fail is untested.

Table R4. Test-time routing along the checkpoints (ARC-Challenge)

CheckpointStageFix densityTFLT gainDr.LLM gainPOLAR gain
Qwen3-1.7B-BaseLLM Pretrain0.321+1.8+2.5+0.1
Qwen3-1.7BLLM Instruct0.149+0.6−0.5−0.4
InternVL3.5-2B-PretrainedMLLM Pretrain0.114−0.1−0.3−0.2
InternVL3.5-2BMLLM GSPO0.087−0.3−1.2−0.6

We agree that Table 1 alone does not test this link. Table R4 runs the three test-time methods on earlier checkpoints of the same family. We use ARC-Challenge, where fix density is known for every stage.

The methods work before post-training and stop working after it. On Qwen3-1.7B-Base, TFLT and Dr.LLM gain +1.8 and +2.5 points, or +1.5 on average across all three methods. After LLM post-training, no method gains more than +0.6. On the final model, all three fall below the default route (−0.3 to −1.2). The average gain falls at every stage, tracking the fall in fix density from 0.321 to 0.087. We have added this table to Sec. 6.

Reference

[1] Yue, Yang, et al. "Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?." Advances in Neural Information Processing Systems 38 (2026): 57654-57689.

[2] Matsutani, Kohsei, et al. "Rl squeezes, sft expands: A comparative study of reasoning llms." International Conference on Learning Representations. Vol. 2026. 2026.

[3] Yang, An, et al. "Qwen3 technical report." arXiv preprint arXiv:2505.09388 (2025).

Reviewer_LGdCOpenReview ↗

Response to authors

I thank the authors for the detailed response and the new experiments. W2 and W3 are partly resolved. My remaining concerns are below.

W2: I appreciate the nulls in Table R2, but they confirm my concern rather than resolve it. Within the target set, the best recursion route (0.66) cannot be told apart from a random correct route (0.59), and best-of-N (0.75) aligns more. So the logit alignment mostly reflects correction toward the right answer, not anything specific to recursion.

The two remaining signals are weak:

With answer options removed, 0.14 ± 0.03 overlaps with 0.11 ± 0.04 (best-of-N) and 0.09 ± 0.03 (random correct). The hidden-state value is 0.04 ± 0.02, against −0.01 ± 0.02 for a random correct route. The response says the pattern holds at 1B to 8B, but no numbers are shown.

The target-vs-control comparison in Fig. 5 is not a matched contrast either: control instances are already correct, so there is no correction direction for them to align with.

The revised abstract now says the alignment holds “beyond what shared error correction explains”. Table R2 does not support that. Sec. 5 and the conclusion also still say the alignment “confirms” absorption and is “not coincidental”.

W3: The distinction between train-through and test-time use is convincing, and the length-matched random routes show that most of the −7.56 came from route length. However, only top-1 stands out. Top-2 does worse than both random single-layer routes. With two random draws in one category and no seed variance, this supports “the top-1 route is good in STEM”, not “the utility map ranks routes reliably”.

AuthorsOpenReview ↗

Response to Reviewer LGdC

We thank the reviewer for the careful follow-up. We respond to the remaining points below.

W2. Alignment beyond shared error correction.

We agree that the logit alignment in Table R2 largely reflects shared error correction, and the revised Section 5 states this. However, the 2B MPO → GSPO hidden-state result (0.04 / −0.01) comes from one of the weakest settings. To show this, we extended the hidden-state analysis with the same nulls to all four scales and three transitions.

Table R5. Hidden-state alignment, mean over layers 21 to 27 (best recursion route / random correct route)

ScalePretrain → InstructInstruct → MPOMPO → GSPO
1B0.15 / 0.130.04 / 0.010.07 / 0.03
2B0.11 / 0.10−0.01 / −0.010.04 / −0.01
4B0.23 / 0.050.12 / 0.010.24 / −0.01
8B0.31 / 0.220.10 / 0.020.16 / 0.03

At 4B and 8B, the best route reaches 0.10 to 0.31 and stays above the random correct route in all six panels, for example 0.24 against −0.01 at 4B MPO → GSPO. The random correct route fixes the same instance at the same checkpoint, so shared error correction alone would predict the same alignment. The gap therefore points to shared structure beyond correction. Best-of-N has no hidden-state counterpart, since sampling does not change the forward pass. The same separation appears in logits with option tokens removed (about 0.6 against 0.1 to 0.2 at 4B MPO → GSPO). The full curves are in the revised Appendix D.2 (Figs. 12 to 14).

We agree that target versus control is not a matched contrast. Fig. 1 and Fig. 5 now compare the best route with a random correct route and a shuffled route on the same target instances. We also updated the abstract and introduction accordingly.

W3. Reliability of route selection.

We agree that the ablation alone supports "the top-1 route is good in STEM". The STEM ranking in the paper is based on only 333 instances, which is the STEM share of the 2k selection subset. To test how stable this ranking is, we repeat the selection with larger STEM sets in Table R6.

Table R6. STEM route ranking by selection-set size (InternVL3.5-2B)

InstancesTop-1Top-2Top-3Top-10 overlap with N = 2000
333r19,19r1,1r10,140.5
500r19,19r10,14r26,260.7
1000r19,19r10,14r16,190.9
1500r19,19r10,14r16,191
2000r19,19r10,14r16,191 (reference)

The top-1 route is r19,19 at every size. The rest of the ranking converges as the set grows. From 1000 instances on, the top 3 is identical, and the top-10 overlap with the 2,000-instance ranking reaches 1 at 1,500. The second-ranked route at 333 instances, r1,1, drops out of the top 5 from 500 instances on. Its weak training result (−0.30 against standard GSPO) therefore reflects noise from the small selection set rather than the ranking itself.

Once the ranking is stable, the second-ranked route is r10,14. In our ablation this route was ranked third and already improved over standard GSPO (+0.58). So both of the top two routes in the stable ranking improve over standard GSPO, and the top-1 route gives the largest gain (+1.88).

We thank the reviewer again and hope the extended results and the narrower claims address the remaining concerns.

Reviewer_r217OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether a multimodal LLM that has gone through full post-training can still fix its own mistakes by re-running a contiguous block of its layers. The authors define a "recursion budget": the fraction of instances where the default forward pass is wrong but at least one route in a small family is right, evaluating every route exhaustively.

Across 21 checkpoints from 8 MLLM families, an instance-level oracle beats the default pass by 23.5–60.2 points, while a task-level oracle gains only -0.5 to 2.7 points, so the useful route varies by instance. Along the public Qwen3 and InternVL3.5 checkpoints, mostly falls with post-training, and during RL no instance moves from out-of-reach to correct under the default pass. On instances that post-training newly gets right, the changes in weights, logits and hidden states point in a direction similar to the previously useful recursion routes, from which the authors conclude that post-training absorbs the recursive structure into the default pass.

Finally, they fix one recursion route per task category and use it inside GSPO training: on InternVL3.5-2B it beats standard GSPO in 5 of 6 categories, and across three further backbones in 14 of 18 categories.

Critical strengths
  1. The question is well chosen and the measurement is clean. Prior recursion and routing work mostly evaluates pretrained or instruction-tuned models. This paper looks at models after the full post-training pipeline. Exhaustively enumerating a small, closed route family avoids any dependence on search or a learned router, so numbers are directly comparable across checkpoints.

  2. The coverage is broad. Eight families, 21 checkpoints, plus the public intermediate checkpoints of Qwen3 and InternVL3.5, make it hard to explain the "budget shrinks with post-training" trend away as a single-model quirk. I also appreciate that Appendix B.2 includes a temperature-sampling control with matched candidates. It is not a comfortable result for the authors, but it belongs in the paper.

Critical weaknesses
  1. The mechanistic claims lack controls that separate "recursion" from more generic effects.
  • BB in Eq. (3) is an absolute fraction, so it has to fall as default accuracy rises. B/(1−acc)B/(1-\mathrm{acc}) should be reported too. From Appendix Fig. 6, this ratio goes from about 89% to about 94% on InternVL3.5-8B / MMStar, but from Fig. 3, it goes from about 97% down to about 89% on ARC. The two benchmarks move in opposite directions, so BB alone cannot tell us whether the share of remaining errors that recursion can fix has changed.

  • In Appendix B.2, temperature sampling with the same number of candidates gets close to the oracle, so a good part of BB may simply be "try enough times and one attempt is right" rather than anything specific to recursion. The budget and fix density of temperature sampling, or of a non-recursive perturbation family, should be reported at each training stage.

  • The alignment analysis in Sec. 5 has a selection problem. Target instances are chosen because post-training gets them right, and the best recursion route is chosen by whether it gets the target answer right, so both are selected to fix the same instances. Similar directions near the output layers are then unsurprising and do not show that post-training "absorbs the recursive mechanism." A null control is needed, for example a non-recursive intervention that also fixes these instances, or routes shuffled across instances. The hidden-state alignment in the right panel of Fig. 5 is also small, with overlapping confidence intervals between target and control at several layers.

  • Table 1 is used to argue that the shrinking budget "explains" why existing routers fail, but this is never tested directly: the three methods run only on two final post-trained 2B models (InternVL3.5-2B and Qwen3-VL-2B), only on STEM tasks, with no evidence that they work on an earlier checkpoint of the same family and stop working after post-training.

  1. The evidence for the method is narrower than claimed, and the story is in tension with Sec. 3.
  • In Appendix Table 16, recursive GSPO in the single-model, all-task setting scores 53.33, below standard GSPO at 54.07, and cross-category transfer in Fig. 15 is weak (14 of 30 off-diagonal cells negative, versus 25 of 30 positive for standard GSPO). In practice this means training six specialist models and knowing the task category at inference, yet the abstract and conclusion read like a general training gain.
  • Sec. 3 itself concludes that fixed routes gain little (task-level oracle at most +2.7) while the instance-level budget is 23–60 points. The method uses exactly a fixed per-category route yet is described as "spending the remaining budget," recovering only 1–3 points, and Sec. 6 admits the mechanism looks more like an inductive bias. Table 17(b) also shows high sensitivity to the route (on STEM, top-1 is +1.88, top-2 is −0.30, and a random route is −7.56).
  • There is no "standard GSPO model + the same route at inference" control and no inference-FLOPs-matched comparison (recursive GSPO runs extra layers at inference: 1–3 on the 2B model, and up to about sixteen on some other backbones). On Spatial & Action the Rec. Baseline is already +2.16, close to the +2.39 of recursive GSPO over standard GSPO.
Questions for authors
  1. Can you report B/(1−acc)B/(1-\mathrm{acc}) per training stage, together with the budget and fix density of temperature sampling or a non-recursive perturbation family? If the trend is the same, how do you define what is specific to the recursion budget?

  2. How is the "best recursion route" for the control group in Sec. 5 defined? Does the target/control alignment gap remain with a non-recursive intervention, or with routes shuffled across instances?

  3. Can you report the "standard GSPO model + the same route at inference" result, together with training and inference FLOPs or latency?

  4. Can you run the same three routing methods on an earlier checkpoint of the same MLLM family to show they work there and only fail after post-training? Also, what is the absolute number of instances in the OOR row of Fig. 7?

4 comments
AuthorsOpenReview ↗

Part 1/4: Response to Reviewer r217

We thank the reviewer for a detailed and constructive review, and for noting that the temperature-sampling control belongs in the paper. We answer each question with new results.

W1-1: (B) in Eq. (3) is an absolute fraction, so it has to fall as default accuracy rises. (B/(1-\mathrm{acc})) should be reported too. ... so (B) alone cannot tell us whether the share of remaining errors that recursion can fix has changed.

Table R1. Budget per stage. ARC: Qwen3-1.7B to InternVL3.5-2B. MMStar: InternVL3.5-8B.

StageARC B (%)ARC B/(1−acc) (%)MMStar B (%)MMStar B/(1−acc) (%)
LLM Pretrain37.596.9
LLM Instruct31.396.9
MLLM Pretrain17.496.138.189.4
MLLM Instruct15.584.634.296.3
MLLM MPO15.686.233.195.9
MLLM GSPO15.688.631.093.7

We agree that B is an absolute fraction and must fall as accuracy rises. Table R1 reports B/(1−acc) per stage. The two benchmarks move differently because the ratio depends on whether a stage extends what the model can solve. SFT can extend it, while RL mainly sharpens what the model already reaches [1,2].

  • MMStar. MLLM SFT trains on multimodal data and extends the model's reach. Of the instances out of reach before SFT, 57.8% enter the budget and 12.0% become always-correct (Fig. 7, pooled over scales). The ratio therefore rises from 89.4 to 96.3. RL then lowers it to 93.7.
  • ARC. MLLM SFT trains mainly on multimodal data. It therefore does not extend reach on ARC, a text task the base LLM could already solve. No out-of-reach instance becomes correct, and 10.3% of in-budget instances fall out of reach (Fig. 8, pooled over scales). The ratio therefore falls from 96.1 to 84.6. Under RL it recovers to 88.6. 94.5% of the instances that RL returns to the budget were in budget before MLLM SFT, so RL restores earlier reach rather than adding new reach.

The ratio therefore depends on the training data, the objective, and the benchmark together, so it should be read step by step rather than as one trend. On both benchmarks, RL never makes an out-of-reach instance correct (Figs. 8 and 9), as the sharpening view predicts. Our general claim is that the model benefits less from recursion after post-training. It also rests on three independent measures: (1) fix density (Fig. 3, right), (2) fixed-route transfer (Fig. 4), and (3) inter-layer CKA (Fig. 10)

W1-2: In Appendix B.2, temperature sampling with the same number of candidates gets close to the oracle, so a good part of (B) may simply be "try enough times and one attempt is right" rather than anything specific to recursion. The budget and fix density of temperature sampling, or of a non-recursive perturbation family, should be reported at each training stage.

Table R2. Recursion and sampling per stage (ARC-Challenge, N = 407).

StageOracle gain, recursionOracle gain, samplingFix density, recursionFix density, samplingOverlap of fixable sets (Jaccard)
LLM Pretrain+37.5+38.70.3210.2770.97
LLM Instruct+31.3+27.40.1490.2040.86
MLLM Pretrain+17.4+18.00.1140.1930.96
MLLM Instruct+15.5+18.10.1030.1760.86
MLLM MPO+15.6+17.70.0980.1640.87
MLLM GSPO+15.6+16.90.0870.1420.88

We compare recursion with temperature sampling at the same number of candidates. Both run on ARC-Challenge along the Qwen3-1.7B to InternVL3.5-2B checkpoints. Both budgets shrink with post-training, and the two fixable sets overlap strongly at every stage (Jaccard 0.86 to 0.97). Recursion and sampling therefore reach largely the same instances: those the model can answer correctly by some perturbation but not by default (Appendix B.2). Fix density also falls for both, which supports this view.

This again agrees with the view that post-training sharpens the output distribution [1,2]. RL makes this explicit. GSPO samples several rollouts per prompt and reinforces the correct ones, which turns headroom reachable by sampling into default answers. The headroom that remains after post-training become harder to reach by recursion.

What is specific to the recursion budget? Its size and trend are shared with sampling. What is specific to recursion is location in depth. For each instance and task, recursion shows which depth band carries the headroom, which sampling cannot. This location depends on the task category (Figs. 14 and 15), and it is what recursion-aware post-training exploits.

AuthorsOpenReview ↗

Part 2/4: Response to Reviewer r217

W1-3: The alignment analysis in Sec. 5 has a selection problem. ... A null control is needed ...

Table R3. Alignment between the post-training shift and each intervention. InternVL3.5-2B, MPO → GSPO, MMStar, mean over layers 21 to 27, mean ± standard error.

Set and interventionLogit cosineLogit cosine, answer options removedHidden-state cosine
Target, best recursion route (original)+0.66 ± 0.03+0.14 ± 0.03+0.04 ± 0.02
Control, best recursion route (original)−0.01 ± 0.02+0.07 ± 0.01−0.03 ± 0.01
Target, non-recursive fix (best-of-N, N = 407)+0.75 ± 0.02+0.11 ± 0.04n/a
Target, random correct route+0.59 ± 0.03+0.09 ± 0.03−0.01 ± 0.02
Target, routes shuffled across instances−0.05 ± 0.06+0.07 ± 0.03−0.03 ± 0.02
Control, route matched in length and position−0.17 ± 0.02+0.01 ± 0.01−0.05 ± 0.01

For both sets, the best recursion route is the correct non-default route with the highest average log-likelihood of the ground-truth answer tokens. Table R3 tests the alignment against the two nulls the reviewer suggested, plus two more:

  • A random correct route tests whether any correct route aligns.
  • Routes shuffled across instances test whether any recursion aligns.
  • A control route matched in length and position tests whether the target-versus-control gap comes from different routes.
  • Best-of-N sampling (N = 407) serves as the non-recursive fix. Sampling does not change the hidden states at a fixed position, so it has no hidden-state shift.

The target-versus-control gap does not come from the route choice. With routes matched in length and position, control alignment drops further, to −0.17. The alignment also requires the route to fix the instance, since shuffled routes show none (−0.05). A random correct route gives 0.59, and best-of-N sampling gives 0.75, more than the best recursion route (0.66). The higher value for sampling is expected, since RL directly reinforces correct sampled rollouts. Most of the logit alignment therefore reflects shared correction on the budget that recursion and sampling share (Table R2).

We also tested logit alignment without answer options so it becomes free-form answer now. The best recursion route on Target is 0.14, the highest value in the table and clearly above both control rows (0.07 and 0.01). As the reviewer noted, the hidden-state alignment is small but what matter here is the relative difference. Target with best recursion route outperform all null settings (0.04 vs. -0.03 & -0.05) and this pattern prevails across all pattern we tested (1B,2B,4B, and 8B). We will update Sec 5. according to this results.

W1-4: Table 1 is used to argue ... why existing routers fail, but this is never tested directly ... with no evidence that they work on an earlier checkpoint of the same family and stop working after post-training.

Table R4. Learned routers along the checkpoints (ARC-Challenge)

CheckpointStageFix densityTFLT gainDr.LLM gainPOLAR gain
Qwen3-1.7B-BaseLLM Pretrain0.321+1.8+2.5+0.1
Qwen3-1.7BLLM Instruct0.149+0.6-0.5-0.4
InternVL3.5-2B-PretrainedMLLM Pretrain0.114-0.1-0.3-0.2
InternVL3.5-2BMLLM GSPO0.087-0.3-1.2-0.6

We agree that Table 1 alone does not test this link. Table R4 runs the three test-time methods on earlier checkpoints of the same family. We use ARC-Challenge, where fix density is known for every stage.

The methods work before post-training and stop working after it. On Qwen3-1.7B-Base, TFLT and Dr.LLM gain +1.8 and +2.5 points, or +1.5 on average across all three methods. After LLM post-training, no method gains more than +0.6. On the final model, all three fall below the default route (−0.3 to −1.2). The average gain falls at every stage, tracking the fall in fix density from 0.321 to 0.087. We have added this table to Sec. 6.

W2-1: In Appendix Table 16, recursive GSPO in the single-model, all-task setting scores 53.33, below standard GSPO at 54.07, and cross-category transfer in Fig. 15 is weak ...

The single-model result reflects the per-category depth structure. Useful routes sit at different depths for different categories, and no single route improves the average (Fig. 15). This is consistent with layers becoming more distinct during post-training (Fig. 10). The average route r14,15 therefore helps categories with mid-network routes (Spatial & Action, +1.86) but hurts those with early routes (Grounding, Counting & Search, −4.25).

Thus, Fig. 6 shows this resulting trade-off. Recursion-aware GSPO specializes the model to its training category. Its in-category gain averages +2.9, against +1.5 for standard GSPO. In exchange, it transfers less to other categories with different optimal routes. We modified the abstract and limitation so that it does not read like a general training gain.

AuthorsOpenReview ↗

Part 3/4: Response to Reviewer r217

W2-2: Sec. 3 itself concludes that fixed routes gain little (task-level oracle at most +2.7) ... The method uses exactly a fixed per-category route yet is described as "spending the remaining budget," recovering only 1–3 points, and Sec. 6 admits the mechanism looks more like an inductive bias. Table 17(b) also shows high sensitivity to the route ...

We agree that “spending the remaining budget” was the wrong description, since a fixed route cannot recover the whole instance-specific budget. We have revised it in the main text. Actually, Sec. 3 and Sec. 6 test different things. Sec. 3 applies a fixed route to frozen weights at test time, while Sec. 6 trains through the route so the weights adapt to it. The method is therefore an inductive bias, as the reviewer notes, and its gains should be compared with standard GSPO rather than with the instance-level budget.

Route sensitivity. The original random route r6,12 is seven layers long, while the top-1 route is one layer. Repeating a random long block can be disruptive, since few routes are useful (Fig. 14) and post-trained layers have more distinct roles (Fig. 9). We therefore add random routes matched in length to the top-1 route.

Table R5. STEM route choice with length-matched random routes (InternVL3.5-2B)

RouteLayersChange vs standard GSPO
Top-1, r19,191+1.88
Top-2, r1,11−0.30
Top-3, r10,145+0.58
Random, r6,127−7.56
Random single layer, r26,26 (new)1+0.48
Random single layer, r7,7 (new)1+0.14

Most of the −7.56 loss comes from the random route's length. Random single-layer routes stay close to standard GSPO (+0.48 and +0.14). At matched length, the top-1 route beats them by 1.4 to 1.7 points, so the top of the depth map carries real information about where to loop. The top-2 route was chosen from only 333 STEM instances, and we expect a larger selection set to make the ranking below the top-1 more reliable.

W2-3: There is no "standard GSPO model + the same route at inference" control and no inference-FLOPs-matched comparison (recursive GSPO runs extra layers at inference: 1–3 on the 2B model, and up to about sixteen on some other backbones). ...

Table R6. Adding the route at inference to standard GSPO (InternVL3.5-2B, category averages)

CategoryRouteBaselineRec. BaselineGSPOGSPO + route (new)Rec. GSPO
STEM(19,19)38.2337.9439.0139.6840.89
Chart & OCR(17,18)55.5755.1356.2356.0056.92
Spatial & Action(13,15)47.4849.6449.1849.9651.57
Knowledge & Recognition(14,15)59.0158.0959.2258.8458.55
Grounding, Counting & Search(2,2)51.1950.5553.4053.0854.52
Captioning & IF(2,3)60.0959.3163.5863.2066.70

Adding the route at inference to standard GSPO lowers accuracy in four of six categories. It improves only STEM and Spatial & Action, and stays below recursion-aware GSPO in both. Even on Spatial & Action, where the route alone helps the base model (+2.16), training through the route adds 1.61 points beyond applying it at inference (51.57 vs 49.96). The weights therefore need to adapt to the route during training.

Table R7. Repeated layers and extra decoder compute per token

CategoryInternVL3.5-2B (L = 28)InternVL3.5-4B (L = 36)Qwen2.5-VL-3B (L = 36)Qwen3-VL-2B (L = 28)
STEM1 (+3.6%)16 (+44.4%)9 (+25.0%)14 (+50.0%)
Chart & OCR2 (+7.1%)4 (+11.1%)5 (+13.9%)1 (+3.6%)
Spatial & Action3 (+10.7%)6 (+16.7%)3 (+8.3%)4 (+14.3%)
Knowledge & Recognition2 (+7.1%)4 (+11.1%)1 (+2.8%)3 (+10.7%)
Grounding, Counting & Search1 (+3.6%)7 (+19.4%)7 (+19.4%)1 (+3.6%)
Captioning & IF2 (+7.1%)2 (+5.6%)4 (+11.1%)5 (+17.9%)

Compute. Training costs 1.06 to 1.23 times the step time of standard GSPO on InternVL3.5-2B. Table R7 gives the inference cost of each selected route. Across the 24 cells, the route adds 2.8% to 50% of decoder compute per token, with a median of about 11%. The largest gains come from short routes. On Captioning & IF, recursion-aware GSPO gains +3.1 to +4.6 points on three backbones at 5.6% to 17.9% extra compute. The two most expensive routes, STEM on InternVL3.5-4B and Qwen3-VL-2B, give smaller gains (+0.59 and +0.83). Moreover, every route costs less than one extra sample, so a sampling or voting baseline cannot match its compute.

AuthorsOpenReview ↗

Part 4/4: Response to Reviewer r217

Q4: what is the absolute number of instances in the OOR row of Fig. 7?

Table R8. Out-of-reach instances and their flow to always-correct (pooled over scales)

Benchmark and transitionPhaseOut-of-reach at the earlier checkpointFlow to always-correct95% upper bound if zero
MMStar, InternVL3.5 Pretrain → InstructSFT22527 (12.0%)
MMStar, InternVL3.5 Instruct → MPORL26201.1%
MMStar, InternVL3.5 MPO → GSPORL27101.1%
MMStar, InternVL3 Pretrain → InstructSFT15013 (8.7%)
MMStar, InternVL3 Instruct → MPORL14002.1%
ARC, Qwen3-Base → Qwen3LLM post-training6004.9%
ARC, Qwen3 → InternVL3.5 PretrainMultimodal continual pretraining5020 (40.0%)
ARC, InternVL3.5 Pretrain → InstructSFT3208.9%
ARC, InternVL3.5 Instruct → MPORL12802.3%
ARC, InternVL3.5 MPO → GSPORL10802.7%

Table R8 gives the counts for every transition on both benchmarks. Each RL transition starts from 108 to 271 out-of-reach instances, not a few dozen. For a zero count out of n, the table reports the exact one-sided 95% upper bound. On MMStar, the SFT rates are far above the RL bounds: 12.0% vs. 1.1% for InternVL3.5, and 8.7% vs. 2.1% for InternVL3.

On ARC, both Pretrain-to-Instruct steps also show zero flow. For the sizes we study, Qwen3's post-training is strong-to-weak distillation that ends with an on-policy stage on the model's own samples [3], which may behave like RL here. MLLM SFT trains mainly on multimodal data, so it does not extend reach on ARC, a text task the base LLM could already solve (Appendix C.1).

Reference

[1] Yue, Yang, et al. "Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?." Advances in Neural Information Processing Systems 38 (2026): 57654-57689.

[2] Matsutani, Kohsei, et al. "Rl squeezes, sft expands: A comparative study of reasoning llms." International Conference on Learning Representations. Vol. 2026. 2026.

[3] Yang, An, et al. "Qwen3 technical report." arXiv preprint arXiv:2505.09388 (2025).

Reviewer_fKoXOpenReview ↗
Rating
3(+2)
Confidence
2(−1)
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper examines how post-training affects the recursion budget of multimodal large language models (MLLMs), defined by the authors as the performance gain relative to the non-recursive forward pass achievable through oracle inference-time depth recursion: selecting, for each sample, a contiguous block of layers to repeat once during the forward pass. Across several MLLMs, experiments on text and multimodal QA benchmarks show that post-training reduces the recursion budget. Analyses in weight, representation, and logit space show correlations between changes due to recursion and due to post-training, suggesting that post-training may absorb some of the effects of recursion into the non-recursive forward pass, which the authors interpret as an explanation of why the recursion budget drops with post-training. With the goal of enabling post-training to also benefit from recursion, the authors propose recursion-aware post-training, in which a task-dependent recursion path is used during post-training. Experiments show that this approach outperforms standard post-training and applying the same recursion routes at inference time without further training on five of six task categories of a visual reasoning dataset.

Critical strengths

None.

Critical weaknesses
  • It is hard to tell if the core object of study—the recursion budget—is a meaningful phenomenon. The recursion budget is defined through an oracle that exhaustively evaluates hundreds of possible forward passes and selects the best route given ground-truth answers. It is unclear whether the resulting gains stem from meaningful latent recursive structure or simply the exploration of many alternatives, especially considering the main evaluations are multiple-choice questions. The authors’ own controls are consistent with the latter, as they show that best-of-N temperature sampling reaches comparable gains, including on free-form answer benchmarks. While recursion may define more constrained perturbations than stochastic sampling, this is not demonstrated and, even if it exists, it remains unclear whether this additional structure would be meaningfully related to the successful predictions or whether the gains simply arise from exploration. This concern about the existence of useful recursive structure is compounded by some of the other experiments: Fixed routes yield marginal gains, selected routes do not transfer across subsets of data, and the evaluated learned routers fail to improve performance meaningfully. While the authors identify some task-dependent patterns, these do not explain the large oracle gains. The proposed recursion-aware post-training does not address this concern, as it adapts the model weights to selected routes, demonstrating that training with recursion can be beneficial rather than that the original model possesses exploitable recursive structure. Moreover, the method is likely to be impractical because of its reliance on predefined task categories, requiring separate category-specific models and knowledge of the input's task category at inference time, with no demonstrated mechanism to handle unseen task categories.

  • The second main object of study: the interpretation of the post-training dynamics, is not well supported by the experiments. First, the authors’ usage of the post-training term is questionable. Much of the observed reduction in recursion budget occurs between the Qwen-3 and the InternVL-3.5 checkpoints. The difference between these is significant, with the latter undergoing extensive multimodal (including further text) continual pretraining. The difference in the recursion budget might therefore simply reflect the strengthening of the model due to large scale further training. Second, the analyses relating post-training to recursive computation, and that are meant to support the authors’ hypothesis of post-training incorporating recursive structure into the standard forward pass, do not adequately distinguish shared computation from shared error correction. Target examples are selected so that recursion and the later further trained checkpoint both correct the earlier checkpoint’s error, whereas controls are already correct. Because both interventions correct the same initial error, their output changes may align simply because they favor the same correct answer. Comparing these examples with already-correct controls does not distinguish this effect from the authors’ proposed explanation of shared recursive computation. The additional measurements do not resolve this ambiguity: correlations between layer-update magnitudes and recursion frequencies require controls for differences in route length and depth dynamics, while the hidden-state similarities are very small, and have no demonstrated causal effect on the behavioral improvements.

Questions for authors

Beyond clarifying potential misunderstandings in my critique above, I do not have specific questions whose answers would materially affect my assessment.

Edit: Updated my rating to Weak Accept.

7 comments
AuthorsOpenReview ↗

Part 1/3: Response to Reviewer fKoX

We thank the reviewer for a careful and critical review. We agree with several of its central points and have narrowed the paper's claims. We respond to each concern below.

W1-1: It is hard to tell if the core object of study—the recursion budget—is a meaningful phenomenon. ... It is unclear whether the resulting gains stem from meaningful latent recursive structure or simply the exploration of many alternatives, especially considering the main evaluations are multiple-choice questions. ... it remains unclear whether this additional structure would be meaningfully related to the successful predictions or whether the gains simply arise from exploration.

We agree that the size of B alone does not show recursive structure. Our matched-cardinality control already shows that best-of-N sampling reaches a comparable oracle (Appendix B.2), and Table R1 extends this to every training stage. We also do not claim that recursion is a more constrained perturbation than sampling, since routes change the answer two to three times as often (Appendix B.2). We still believe the budget is meaningful, for three reasons.

Not an artifact of multiple choice. The budget is also present on free-form benchmarks where chance is near zero: +12.6 and +15.8 points on ChartQA-lite, and +26.6 and +31.5 on MIA-Bench (Appendix B, Table 7). There, a perturbation cannot guess the answer. The gains therefore reflect answers the model can actually produce but does not produce by default.

Table R1. Recursion and sampling per stage (ARC-Challenge, N = 407).

StageOracle gain, recursionOracle gain, samplingFix density, recursionFix density, samplingOverlap of fixable sets (Jaccard)
LLM Pretrain+37.5+38.70.3210.2770.97
LLM Instruct+31.3+27.40.1490.2040.86
MLLM Pretrain+17.4+18.00.1140.1930.96
MLLM Instruct+15.5+18.10.1030.1760.86
MLLM MPO+15.6+17.70.0980.1640.87
MLLM GSPO+15.6+16.90.0870.1420.88

The same headroom as sampling. We compare recursion with temperature sampling at the same number of candidates along the Qwen3-1.7B to InternVL3.5-2B checkpoints. The two fixable sets overlap strongly at every stage (Jaccard 0.86 to 0.97), so recursion and sampling reach largely the same instances. Both budgets shrink with post-training, as the sharpening view predicts [1,2]. We see this link as a finding in itself: the recursion budget is the headroom that post-training consumes.

What is specific to recursion. Recursion shows which depth band carries the headroom for each instance and task, which sampling cannot. This location depends on the task category (Appendix, Figs. 14 and 15). It is also related to successful predictions. Training through the top-1 route from this map beats length-matched random routes by 1.4 to 1.7 points (Table R5 in our reply to Reviewer r217).

W1-2: This concern about the existence of useful recursive structure is compounded by some of the other experiments: Fixed routes yield marginal gains, selected routes do not transfer across subsets of data, and the evaluated learned routers fail to improve performance meaningfully. While the authors identify some task-dependent patterns, these do not explain the large oracle gains.

We agree with these observations. They are our own findings, and they are why we conclude that the headroom is hard to use at test time after post-training. Post-training consumes the same headroom that sampling reaches (Table R1), so little is left for a test-time router to find on the final model. To test this directly, we ran the learned routers on earlier checkpoints.

Table R2. Learned routers along the checkpoints (ARC-Challenge)

CheckpointStageFix densityTFLT gainDr.LLM gainPOLAR gain
Qwen3-1.7B-BaseLLM Pretrain0.321+1.8+2.5+0.1
Qwen3-1.7BLLM Instruct0.149+0.6-0.5-0.4
InternVL3.5-2B-PretrainedMLLM Pretrain0.114-0.1-0.3-0.2
InternVL3.5-2BMLLM GSPO0.087-0.3-1.2-0.6

Both routers gain +1.5 points in average before post-training but at most -0.3 after it, tracking the fall in fix density. The recursive signal that test-time routing exploits therefore exists in earlier checkpoints and fades with post-training.

We also agree that the task-dependent patterns do not explain the size of the oracle gain, which recursion shares with sampling (Table R2). They explain why fixed routes fail on average. Useful routes sit at different depths for different tasks (Figs. 14 and 15), and post-trained layers grow more distinct (Fig. 10), so no single route helps every task. This is why we train through a task-specific route instead (Sec. 6).

AuthorsOpenReview ↗

Part 2/3: Response to Reviewer fKoX

W1-3: The proposed recursion-aware post-training does not address this concern, ... Moreover, the method is likely to be impractical because of its reliance on predefined task categories, ...

We agree that the method shows that training through a route helps, not that the frozen model's routes can be used directly at test time. The frozen model still decides where the loop goes, since the route comes from its per-category depth map.

Does the route choice matter? The original random route in Table 17, r6,12, is seven layers long, while the top-1 route is one layer. Repeating a random long block can be disruptive, since few routes are useful (Fig. 15) and post-trained layers have more distinct roles (Fig. 10). We therefore add random routes matched in length to the top-1 route.

Table R3. STEM route choice with length-matched random routes (InternVL3.5-2B)

RouteLayersChange vs standard GSPO
Top-1, r19,191+1.88
Top-2, r1,11−0.30
Top-3, r10,145+0.58
Random, r6,127−7.56
Random single layer, r26,26 (new)1+0.48
Random single layer, r7,7 (new)1+0.14

Most of the −7.56 loss comes from the random route's length. Random single-layer routes stay close to standard GSPO (+0.48 and +0.14). At matched length, the top-1 route beats them by 1.4 to 1.7 points, so the top of the depth map carries real information about where to loop. The top-2 route was chosen from only 333 STEM instances, and we expect a larger selection set to make the ranking below the top-1 more reliable.

We also agree that the method needs one model per category and the category at inference, and does not handle unseen categories. We now state this in the abstract and the limitations (Sec. 8). Per-category routes follow from the depth structure of post-trained models. Useful routes sit at different depths for different tasks (Figs. 14 and 15), in line with prior layer-role analyses [3,4], so no single route serves every task.

Fig. 6 shows the consequence. Training through a category's route raises in-category accuracy by +2.9 on average, against +1.5 for standard GSPO, but transfers less to categories with different optimal routes. Models trained to be recursive from scratch may avoid this trade-off, as we discuss in Sec. 8. We see the method as a complementary direction for post-training.

W2-1: ... the authors’ usage of the post-training term is questionable. ... The difference in the recursion budget might therefore simply reflect the strengthening of the model due to large scale further training.

We agree on the terminology. The step from Qwen3 to InternVL3.5-Pretrained is multimodal continual pretraining on mixed text and image data, as Appendix C.1 notes, not post-training. We now name three phases: LLM post-training (Qwen3-Base to Qwen3), multimodal continual pretraining (Qwen3 to InternVL3.5-Pretrained), and MLLM post-training (SFT, MPO, GSPO). We report every measure per phase. We also agree that part of the drop in B reflects a stronger model. Table R4 normalizes B by the error rate.

Table R4. Budget per stage. ARC: Qwen3-1.7B to InternVL3.5-2B. MMStar: InternVL3.5-8B.

StageARC B (%)ARC B/(1−acc) (%)MMStar B (%)MMStar B/(1−acc) (%)
LLM Pretrain37.596.9
LLM Instruct31.396.9
MLLM Pretrain17.496.138.189.4
MLLM Instruct15.584.634.296.3
MLLM MPO15.686.233.195.9
MLLM GSPO15.688.631.093.7

Across continual pretraining, B/(1−acc) barely moves (96.9 to 96.1). The drop in B there (31.3 to 17.4) therefore comes from higher accuracy, as the reviewer suggests. Within the post-training phases, the ratio follows the training objective rather than accuracy alone. SFT can extend what the model can solve, while RL mainly sharpens what it already reaches in line with previous findings [1,2].

  • MMStar. MLLM SFT extends the model's reach. Of the instances out of reach before SFT, 57.8% enter the budget and 12.0% become always-correct (Fig. 7). The ratio rises from 89.4 to 96.3, and RL then lowers it to 93.7.
  • ARC. MLLM SFT trains mainly on multimodal data and does not extend reach on this text task. No out-of-reach instance becomes correct, and 10.3% of in-budget instances fall out of reach (Fig. 8). The ratio therefore falls from 96.1 to 84.6. RL brings it back to 88.6, and 94.5% of the instances RL returns to the budget were in budget before MLLM SFT.

A general strengthening account would not predict these opposite moves under SFT and RL. Strengthening and losing recursive headroom may also be two sides of one process. Post-training turns the headroom that recursion and sampling share into default answers (Table R1). Three independent measures also support the same view: fix density (Fig. 3, right), fixed-route transfer (Fig. 4), and inter-layer CKA (Fig. 10).

AuthorsOpenReview ↗

Part 3/3: Response to Reviewer fKoX

W2-2: Second, the analyses relating post-training to recursive computation, ... do not adequately distinguish shared computation from shared error correction. ... Because both interventions correct the same initial error, their output changes may align simply because they favor the same correct answer. ...

Table R5. Alignment between the post-training shift and each intervention. InternVL3.5-2B, MPO → GSPO, MMStar, mean over layers 21 to 27, mean ± standard error.

Set and interventionLogit cosineLogit cosine, answer options removedHidden-state cosine
Target, best recursion route (original)+0.66 ± 0.03+0.14 ± 0.03+0.04 ± 0.02
Control, best recursion route (original)−0.01 ± 0.02+0.07 ± 0.01−0.03 ± 0.01
Target, non-recursive fix (best-of-N, N = 407)+0.75 ± 0.02+0.11 ± 0.04n/a
Target, random correct route+0.59 ± 0.03+0.09 ± 0.03−0.01 ± 0.02
Target, routes shuffled across instances−0.05 ± 0.06+0.07 ± 0.03−0.03 ± 0.02
Control, route matched in length and position−0.17 ± 0.02+0.01 ± 0.01−0.05 ± 0.01

We agree that our original target-versus-control comparison could not separate shared computation from shared error correction. Table R5 adds the nulls needed to separate them. For both sets, the best recursion route is the correct non-default route with the highest average log-likelihood of the ground-truth answer tokens. The nulls test different explanations:

  • A random correct route tests whether any correct route aligns.
  • Routes shuffled across instances test whether any recursion aligns.
  • A control route matched in length and position tests whether the target-versus-control gap comes from route length.
  • Best-of-N sampling (N = 407) is a non-recursive fix. It has no hidden-state shift, since sampling does not change the hidden states at a fixed position.

The target-versus-control gap does not come from the route choice. With routes matched in length and position, control alignment drops further, to −0.17. The alignment also requires the route to fix the instance, since shuffled routes show none (−0.05). A random correct route gives 0.59, and best-of-N sampling gives 0.75, more than the best recursion route (0.66). The higher value for sampling is expected, since RL directly reinforces correct sampled rollouts. Most of the logit alignment therefore reflects shared correction on the budget that recursion and sampling share.

We also tested logit alignment without answer options so it becomes free-form answer now. The best recursion route on Target is 0.14, the highest value in the table and clearly above both control rows (0.07 and 0.01). As the reviewer noted, the hidden-state alignment is small but what matter here is the relative difference. Target with best recursion route outperform all null settings (0.04 vs. -0.03 & -0.05) and this pattern prevails across all pattern we tested (1B,2B,4B, and 8B). We will update Sec 5. according to this results.

Reference

[1] Yue, Yang, et al. "Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?." Advances in Neural Information Processing Systems 38 (2026): 57654-57689.

[2] Matsutani, Kohsei, et al. "Rl squeezes, sft expands: A comparative study of reasoning llms." International Conference on Learning Representations. Vol. 2026. 2026.

[3] Shi, Cheng, Yizhou Yu, and Sibei Yang. "Vision function layer in multimodal llms." Advances in neural information processing systems 38 (2026): 27510-27537.

[4] Yu, Zhuoran, and Yong Jae Lee. "How multimodal llms solve image tasks: A lens on visual grounding, task reasoning, and answer decoding." arXiv preprint arXiv:2508.20279 (2025).

Comments on the paper

1 comment
Area_Chair_PUcrOpenReview ↗

Please Discuss

The authors have provided detailed responses and additional experiments. Given the differing review scores, please take another look at the rebuttal and discuss whether the main concerns have been adequately addressed, particularly regarding the recursion-specific claims and post-training analysis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.