acceptodds
Under review as a conference paper at ICLR 2027

COMPUTATIONAL DEPTH AS AN ADAPTATION VARIABLE FOR PARAMETER EFFICIENT FINE TUNING

Abstract

Parameter efficient fine tuning adapts pretrained language models with a small number of trainable parameters. Most existing methods seek greater adaptation capacity by changing how these parameters modify a fixed pretrained computation graph. We study a complementary possibility in which the amount of pretrained computation itself becomes part of adaptation. We introduce recurrent PEFT by repeatedly executing a frozen pretrained middle stack while training lightweight adapters around the reused computation. Building on residual damped recurrence introduced in prior work, we consider both iteration specific and iteration shared adaptation. This construction increases effective computational depth without adding backbone parameters and enables direct comparison with conventional single pass PEFT under matched trainable parameter budgets. Across Qwen and Llama backbones from 0.6B to 8B parameters, recurrent adaptation remains broadly competitive on general reasoning and shows gains on GSM8K across all evaluated backbones. Minerva reveals a more pronounced backbone-dependent effect, with substantially larger gains on Qwen than on Llama models. We further find that these effects depend on the underlying PEFT parameterization and that the choice of recurrent execution remains consequential after fine-tuning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
PEFT, Looped Transformer

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bCVtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

In this work, the authors combine LoRA fine-tuning with recurrent Transformer computations. The results compare shared versus depth-wise LoRA adaptation for repeated layer groups across different backbones and tasks. The manuscript suggests an advantage of LoRA-adapted recurrent computations for certain combinations of mathematical reasoning tasks and backbones.

Critical strengths

Many PEFT baselines and datasets are considered, demonstrating the advantage of PEFT tuning for recursive Transformer layers.

Critical weaknesses
  1. The combination of LoRA and recurrent Transformers is not particularly new (see, for example, Fu, T., et al. (2026). Think-at-Hard: Dynamic Looped Transformers for Improved Reasoning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research), and the proposed method is rather a modest extension of previous recurrent architectures.
  2. The analysis is limited to a fixed depth and does not quite match the title, which might suggest computational depth K as an adaptive variable rather than a fixed hyper-parameter.
Questions for authors
  1. The results in Table 4 are worse for the standard LoRA-tuned baseline than in Table 1 for the same backbone. Could you explain the reason for this mismatch?
  2. What do you mean by "computational depth as an adaptation variable" if the depth is a fixed hyper-parameter in the proposed setup?
  3. Could you show a table with non-tuned backbone, non-tuned recurrent layers, and tuned recurrent layers comparison?
4 comments

We thank Reviewer bCVt for the time and constructive feedback on our submission. We appreciate your positive assessment of our paper's soundness and our evaluation across diverse PEFT baselines and datasets. Below, we address each of your comments regarding related work, terminology, the baseline numbers in Table 4, and the comparison with untrained recurrent models.

1. Distinction from Think-at-Hard (Fu et al., 2026) and Recurrent LoRA (W1)

Our problem formulation and architecture differ from Think-at-Hard in three key respects:

  1. No routing modules or attention changes: Think-at-Hard trains an auxiliary neural decider for dynamic token-level skipping and modifies cross-depth attention (duo-causal attention). RLR adds zero routing parameters and preserves standard attention via residual damping (γ=1/K\gamma = 1/K).

  2. Strictly matched trainable budgets: Rather than adding trainable capacity with depth, RLR matches the exact trainable parameter budget of single-pass PEFT, isolating computational depth from trainable parameter count.

  3. PEFT parameterization & post-tuning damping: Beyond standard LoRA, we show that recurrent gains depend strongly on the PEFT parameterization across LoRA, DoRA, PiSSA, and MiSS (Table 2) and that residual damping remains necessary after optimization (Table 4).

2. Meaning of "Computational Depth as an Adaptation Variable" (W2, Q2)

Here, "adaptation" refers to model fine-tuning (as in Low-Rank Adaptation), not input-adaptive dynamic halting at inference time. Calling computational depth an adaptation variable means treating recurrent depth KK (and window width ww) as controllable structural design choices during PEFT (analogous to rank rr or target module selection), evaluated across K∈{1,2,4,8}K \in \{1, 2, 4, 8\} and w∈{4,8,16}w \in \{4, 8, 16\} (Tables 6 and 11) under a fixed trainable-parameter budget.

3. Apparent Mismatch Between Table 4 and Table 1 (Q1)

There is no numerical mismatch between Table 4 and Table 1. For Llama-3.2-1B, the Baseline row in Table 4 (PIQA 69.70, ARC-E 35.52, ARC-C 30.80, HS 56.19, WG 54.54, CSQA 19.25, OBQA 35.60, Avg 43.09, GSM8K 14.48) is identical across all columns to the Dense row in Table 1. The lower scores directly beneath it (Avg 42.32, GSM8K 13.27) belong to the undamped Additive recurrent ablation.

4. Non-Tuned Backbone vs. Non-Tuned Recurrent vs. Tuned Recurrent (Q3)

Below we compare the untrained backbone (K=1K=1), untrained damped recurrent model (Untrained RLR, K=2K=2, zero-initialized adapters), tuned single-pass LoRA (K=1K=1), and tuned RLR-I / RLR-S (K=2K=2) on Llama-3.2-1B and Qwen3-4B (The tuned rows represent the optimal checkpoints identified from our expanded learning rate sweep within the range of 2e-5 to 3e-4):

ModelMethodGSM8K StrictGSM8K FlexMinerva MVMinerva EMGeneral Avg
Llama-3.2-1BUntrained Base4.854.855.165.164.464.462.922.9243.8143.81
Untrained RLR3.183.183.343.344.084.082.742.7443.5143.51
LoRA33.0633.0619.4819.489.04\boldsymbol{9.04}4.444.4444.0744.07
RLR-I33.28\boldsymbol{33.28}28.4328.438.948.946.74\boldsymbol{6.74}44.70\boldsymbol{44.70}
RLR-S31.4631.4632.07\boldsymbol{32.07}8.368.365.565.5644.5044.50
Qwen3-4BUntrained Base48.6048.6084.3884.380.640.640.000.0051.2351.23
Untrained RLR41.3241.3283.4083.4029.8029.800.320.3251.6151.61
LoRA86.2886.2885.9785.9747.9047.9034.5434.5454.5854.58
RLR-I86.50\boldsymbol{86.50}86.50\boldsymbol{86.50}48.20\boldsymbol{48.20}38.5438.5457.6257.62
RLR-S86.2886.2886.3586.3547.7847.7840.00\boldsymbol{40.00}58.32\boldsymbol{58.32}
  1. Recurrence without adaptation does not account for the gains: Repeating layers without tuning slightly degrades Llama-3.2-1B across tasks, and on Qwen3-4B lowers GSM8K while partially stabilizing zero-shot Minerva formatting (0.64→29.800.64 \to 29.80).

  2. PEFT on recurrent depth drives consistent improvements: After adaptation, RLR-I and RLR-S substantially outperform both untrained recurrence and tuned single-pass LoRA across GSM8K, Minerva EM, and General reasoning.

I thank the authors for their response. I agree, there is no mismatch between Table 4 and Table 1, that was my oversight. My core concerns are addressed, I am willing to increase my score.

Thank you for taking the time to read our response and for raising your score. We are glad that our explanation resolved your concerns.

Reviewer_5E7hOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a recurrent PEFT method called Recurrent Layer Reuse (RLR). A frozen pretrained decoder is split into a prefix, a middle stack, and a suffix. The middle stack is run KK times with the residual-damped update from Training-Free Looped Transformers (TFLT):

ht+1=ht+1K(M(ht)−ht).h_{t+1} = h_t + \frac{1}{K}\left(M(h_t) - h_t\right).

LoRA-style adapters are trained around the middle stack. RLR-I gives each iteration its own lower-rank adapter, while RLR-S shares one adapter across iterations; both are matched in trainable parameters to single-pass LoRA.

Critical strengths
  1. The paper has a clean framing of the question: Separating trainable capacity from the amount of pretrained computation that is executed is a sensible way to pose the problem. The correctness checks in Appendix B.3 (K = 1 reproduces the base model, and the unrolled model matches the looped forward pass) make the implementation credible.
  2. The recurrence formulation is ablated after training: Table 4 compares additive, naive and residual-damped recurrence at the same budget and shows that the damping choice still matters after optimization, which goes beyond TFLT's training-free result.
Critical weaknesses
  1. Baselines and evaluation look miscalibrated: All runs use a learning rate of 2×10⁻⁵ (Table 8), roughly ten times lower than the usual 1 to 3×10⁻⁴ for LoRA-style adapters, so single-pass LoRA is probably undertrained. Several fine-tuned numbers are also implausibly low. CommonsenseQA stays at 19 to 23%, close to chance for five options, and Llama-3.1-8B after Commonsense170K tuning gets HellaSwag 72.97 and ARC-C 40.70, below what the base model usually scores zero-shot. On Qwen3-4B, dense LoRA gets Minerva EM 0.28 but MathVerify 18.30, which looks like an answer-format failure, so the large RLR gain (18.30 to 47.60 MathVerify) may come from formatting or termination changes rather than better reasoning. Table 3 reports held-out losses of 6.70 (math) and 10.26 (general) nats for base Llama-3.2-1B, and damped looping with zero adapters lowers this to 5.33. That suggests something is off in the loss measurement or the template, and it also means the recurrence changes the model before any adaptation. Two controls are missing: the TFLT model without fine-tuning, and single-pass LoRA with a tuned learning rate. Without them, the improvements cannot be credited to depth as an adaptation variable.

  2. Not compute-matched, and limited novelty: RLR runs 1.14 to 1.5 times as many layer passes during training and inference (Table 6: 32 to 42 vs. 28), so the comparison only holds trainable parameters fixed. It lacks baselines that also spend extra compute or depth with few trainable parameters, such as higher-rank or all-layer LoRA and depth up-scaling or block expansion [1, 6]. In terms of method, RLR combines TFLT's damped loop, used unchanged, with per-depth LoRA as in Relaxed Recursive Transformers [3]. The results are also inconsistent: Llama-3.1-8B gets worse on commonsense and Minerva, PiSSA on Qwen3-0.6B drops on Minerva from 26.48 to 18.22, neither RLR-I nor RLR-S is consistently better, and a wider loop window (w = 16) makes performance collapse. The paper does not explain when depth helps, which it acknowledges, so the contribution reads more like an exploratory report than a method.

  3. Missing related work: Block expansion and depth up-scaling (LLaMA Pro [6], SOLAR [1]) are the closest "add depth, train few parameters" alternatives and should be discussed and compared. Mixture-of-Recursions [7] learns a per-token recursion depth on top of shared layers and is a natural extension of RLR-S. Work on which layers can be removed or reused also bears on the choice of loop window: [4] finds that many deeper layers can be pruned with little loss, and [2] shows that middle layers are similar enough to be skipped, reordered or repeated. Finally, the paper should discuss sparse PEFT under matched trainable budgets on the same Commonsense170K and math protocol. SMT [5] reports that LoRA and DoRA plateau as rank grows, which is a natural motivation for spending a fixed budget on depth instead of rank (Appendix B.5), and it would be another parameterization for Section 5.3.

References:

[1] Scaling Large Language Models with Simple yet Effective Depth Up-Scaling.

[2] Transformer Layers as Painters.

[3] Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA.

[4] The Unreasonable Ineffectiveness of the Deeper Layers.

[5] SMT: Fine-Tuning Large Language Models with Sparse Matrices.

[6] LLaMA Pro: Progressive LLaMA with Block Expansion.

[7] Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation.

Questions for authors
  1. Could the authors re-run single-pass LoRA and RLR with a tuned learning rate (e.g. a sweep from 5×10⁻⁵ to 3×10⁻⁴) and check whether the GSM8K and Minerva gains remain? (Clarification; this could change my rating.)
  2. What does the TFLT model score on Minerva and GSM8K with zero adapters and no fine-tuning? This would separate the effect of recurrence from the effect of adaptation.
  3. Why are CommonsenseQA scores close to chance and Qwen3-4B Minerva EM at 0.28 after fine-tuning? Were the likelihood-scored tasks evaluated with the chat template used in training? Base-model zero-shot scores under the same harness would help.
  4. How does RLR compare with a compute-matched baseline, e.g. a higher-rank LoRA trained for the same FLOPs, or LLaMA-Pro-style block expansion with the same number of trainable parameters?
  5. What are the inference latency and KV-cache cost of the unrolled model? Unrolling copies the window K times with merged adapters, which also increases model memory.
12 comments

We thank Reviewer 5E7h for the thorough examination of our experimental setup, baseline calibration, and related literature. Your feedback prompted a careful audit of our training and evaluation pipeline, through which we resolved two implementation issues and conducted comprehensive learning-rate sweeps, untuned control evaluations, and block-expansion comparisons.

1. Pipeline Audit and Formatting Controls

Prompted by your diagnostic questions on answer termination, baseline scores, and loss calibration, we audited our pipeline and identified two concrete implementation issues alongside an evaluation-parser sensitivity:

  1. Chat-template termination on Qwen base models: Applying Qwen's default chat template to base (non-instruct) checkpoints relied on <|im_end|>, which is untrained in the base model and shares identical initial embeddings with other unused special tokens. This affected clean generation termination and answer formatting, and we therefore changed the termination token to <|endoftext|>.

  2. Padding tokens in batch loss: During batched training, padding tokens were inadvertently included in the cross-entropy loss calculation. While this did not materially affect the overall trends, we applied this fix in the experiments reported below and will update the full results in the final version.

  3. Separating reasoning from symbol formatting on GSM8K: As you mentioned in your review, because apparent gains or drops can stem from formatting or termination shifts rather than reasoning quality, we relaxed punctuation tolerances (such as : and $) in GSM8K Strict when extracting the model's generated final answer, and report it alongside GSM8K Flex, Minerva MathVerify (MV), Minerva Exact Match (EM), and the 7-task General Avg.

  4. Template consistency on likelihood tasks and CommonsenseQA (Q3): The 0-shot likelihood-scored general benchmarks were evaluated using the exact same chat-template construction function used during training.

2. Learning-Rate Sweep Across Backbones and General Benchmarks

Using the corrected pipeline, we re-ran single-pass LoRA, RLR-I, and RLR-S across five learning rates ({2×10−5,5×10−5,1×10−4,2×10−4,3×10−4}\{2\times 10^{-5}, 5\times 10^{-5}, 1\times 10^{-4}, 2\times 10^{-4}, 3\times 10^{-4}\}) under matched trainable-parameter budgets, reporting the per-method Best row at the bottom of each summary table followed by the full 7-task general evaluation breakdown.

Llama-3.2-1B Learning-Rate Sweep:

ModelSetting / LRMethodGSM8K StrictGSM8K FlexMinerva MVMinerva EMGeneral Avg
Llama-3.2-1B2.00E-05LoRA16.2214.255.524.0644.06
2.00E-05RLR-I17.0611.526.364.2444.20
2.00E-05RLR-S17.8916.006.183.8643.86
5.00E-05LoRA20.478.646.942.7443.41
5.00E-05RLR-I23.8821.836.304.3444.70
5.00E-05RLR-S23.1215.017.103.9242.90
1.00E-04LoRA26.9118.888.361.8244.07
1.00E-04RLR-I31.7728.437.605.6643.87
1.00E-04RLR-S27.9017.667.565.5644.50
2.00E-04LoRA30.8619.488.701.5441.96
2.00E-04RLR-I32.5217.368.406.7243.42
2.00E-04RLR-S31.0817.448.145.4242.21
3.00E-04LoRA33.0613.659.044.4440.42
3.00E-04RLR-I33.2818.738.946.7442.02
3.00E-04RLR-S31.4632.078.363.0041.75
BestLoRA33.0619.489.044.4444.07
BestRLR-I33.2828.438.946.7444.70
BestRLR-S31.4632.078.365.5644.50

Qwen3-4B Learning-Rate Sweep:

ModelSetting / LRMethodGSM8K StrictGSM8K FlexMinerva MVMinerva EMGeneral Avg
Qwen3-4B2.00E-05LoRA84.0881.9646.3013.3654.58
2.00E-05RLR-I84.6984.6946.8214.1257.62
2.00E-05RLR-S85.1485.2247.2818.1858.32
5.00E-05LoRA85.1442.3047.9021.5051.61
5.00E-05RLR-I86.3586.2848.1020.2849.62
5.00E-05RLR-S86.2886.3547.7824.5649.93
1.00E-04LoRA86.2885.5247.4632.3651.19
1.00E-04RLR-I86.5086.5048.2029.0848.97
1.00E-04RLR-S85.4485.3747.3826.5651.40
2.00E-04LoRA86.0585.9747.7634.5449.14
2.00E-04RLR-I84.8484.8447.1437.0251.16
2.00E-04RLR-S83.9383.9346.7639.0449.62
3.00E-04LoRA84.1584.2347.1418.8847.41
3.00E-04RLR-I83.7883.8546.8238.5449.89
3.00E-04RLR-S84.2383.7846.6840.0032.05
BestLoRA86.2885.9747.9034.5454.58
BestRLR-I86.5086.5048.2038.5457.62
BestRLR-S86.2886.3547.7840.0058.32

Across both backbones, RLR retains clear advantages after tuning the learning rate (Best rows): on Llama-3.2-1B, RLR outperforms LoRA at every individual learning rate on GSM8K Strict and achieves higher sweep-best GSM8K Flex (32.07% vs. 19.48%), Minerva EM (6.74% vs. 4.44%), and General Avg (44.70% vs. 44.07%). On Qwen3-4B, RLR achieves higher sweep-best performance across all five metrics: GSM8K Strict (86.50%), GSM8K Flex (86.50%), Minerva MV (48.20%), Minerva EM (40.00% vs. 34.54%), and General Avg (58.32% vs. 54.58%).

Reviewer_3KxVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes a parameter-efficient fine-tuning method by fine-tuning the recurrent blocks in pre-trained large models. These recurrent blocks are introduced by a prior paper that repeats intermediate blocks multiple times while scaling down the individual recurrence's feature updates. There are two modes: 1. Assigning a different parameter update block to each iteration and another is to each iteration having a shared parameter offset.

Experiments show marginal-to-strong improvements over multiple benchmarks.

Critical strengths
  1. proposed method is novel and interesting
  2. the experimental design is carefully controlled, with matched trainable-parameter budgets across all comparisons
  3. analysis in Section 5.4 are interesting: presenting observations about initialization stability and effects of damping formulation.
Critical weaknesses
  1. no reported computational cost? (K times overhead?)
  2. any comparision against test-time compute scaling methods?
Questions for authors
  1. though it is common, I wanted to point out that low cosine similarity does not necessarily indicate low functional similarity (in table 5)
  2. Any analysis or hypothesis for why Qwen gains substantially while Llama does no on Minerva?
8 comments

We sincerely thank the Reviewer for the positive assessment of our work and for highlighting the novelty of our recurrent PEFT formulation, our parameter-matched experimental controls, and the initialization and damping analyses in Section 5.4. Below, we address each weakness and question in detail.

1. Computational Cost (W1)

We would like to clarify that RLR does not incur a K×K\times cost across the full network because recurrence is restricted to a middle window of width w=Lrecw = L_{\text{rec}}, giving Leff=L+(K−1)wL_{\text{eff}} = L + (K - 1)w. For our default setting (w=4,K=2w = 4, K = 2), this adds only 11% to 25% layer passes across backbones (14.3% on Qwen3-0.6B, 25.0% on Llama-3.2-1B, 11.1% on Qwen3-4B, and 12.5% on Llama-3.1-8B), while keeping static backbone weight memory at 1.00×1.00\times during recurrent execution.

ConfigurationwwKKLeffL_{\text{eff}}Layer PassesTFLOPs/step
Single-Pass LoRA-1281.00×1.00\times238
RLR (Main)42321.14×1.14\times272 (+14.3%)
RLR (Deeper)44401.43×1.43\times340 (+42.9%)
RLR (Wide)142421.50×1.50\times358 (+50.4%)

2. Test-Time Compute Scaling (W2)

Token-level test-time scaling (e.g., Majority Voting / Best-of-NN) scales output trajectories, which requires at least 2.0×2.0\times decoding FLOPs even at N=2N = 2 and does not apply to deterministic 0-shot likelihood tasks. In contrast, RLR scales per-token latent computation within a single greedy pass at much lower overhead (1.11×1.11\times to 1.25×1.25\times) and is orthogonal to sample-based scaling (they can be directly combined at inference).

Compared to training-free latent test-time looping (Untrained RLR / TFLT), directly comparing against the untuned backbone (Untrained Base) shows that simply repeating frozen layers at test time without adaptation fails to improve GSM8K and only partially stabilizes Minerva generation on Qwen3-4B, whereas adapting the recurrent stack via PEFT (RLR-I / RLR-S) yields consistent gains over both untuned looping and tuned single-pass LoRA:

ModelMethodGSM8K StrictGSM8K FlexMinerva MVMinerva EMGeneral Avg
Llama-3.2-1BUntrained Base4.855.164.462.9243.81
Untrained RLR3.183.344.082.7443.51
LoRA33.0619.489.044.4444.07
RLR-I33.2828.438.946.7444.70
RLR-S31.4632.078.365.5644.50
Qwen3-4BUntrained Base48.6084.380.640.0051.23
Untrained RLR41.3283.4029.800.3251.61
LoRA86.2885.9747.9034.5454.58
RLR-I86.5086.5048.2038.5457.62
RLR-S86.2886.3547.7840.0058.32

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.