acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training

Abstract

LLM post-training typically propagates task gradients through the full depth of the model. Although this end-to-end structure is simple and general, it couples task adaptation to full-depth activation storage, long-range backward dependencies and direct task-gradient access to pretrained representations. We argue that this full-depth backward coupling can be unnecessarily expensive and intrusive, particularly when post-training supervision is much narrower than pre-training. To this end, we propose LoPT: Local-Learning Post-Training, a simple post-training strategy that makes gradient reach an explicit design choice. LoPT places a single gradient boundary at the transformer midpoint where the upper block learns from the task objective and the lower block is updated by a lightweight feature-reconstruction objective to preserve useful representations and maintain interface compatibility. Such design shortens the task-induced backward path while limiting direct interference from narrow task gradients on lower-layer representations. Extensive experiments demonstrate that LoPT achieves competitive performance with lower memory cost, higher training efficiency and better retention of pretrained capabilities.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large language model, Post Training

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_q3ytOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces LoPT, a local-learning approach that treats task-gradient reach as an explicit design choice for LLM post-training. LoPT splits a Transformer at its midpoint, stops the SFT or GRPO task gradient from entering the first half, and instead updates the lower layers with a lightweight feature-reconstruction objective, while the upper layers learn the original task objective. This separates the two backward graphs and reduces peak activation memory without changing the inference architecture. Across the reported SFT experiments, LoPT achieves competitive but setting-dependent quality while reducing peak memory by about 24–36% and improving throughput by about 2–7%.

Critical strengths
  • The paper is clearly written, with intuitive figures and a detailed explanation of the gradient-routing and training procedure, making the method easy to follow.
  • LoPT is simple, inference-free, and compatible with existing techniques such as LoRA, gradient checkpointing, and GRPO, while providing meaningful peak-memory reductions in the tested settings.
Critical weaknesses
  • The novelty is limited. LoPT mainly replaces one full-depth backward pass with two shorter local backward passes plus an auxiliary head, which is close in spirit to the common engineering practice of freezing the lower layers and tuning only the upper ones. This scheduling can reduce peak activation memory, but it does not remove model parameters, parameter gradients, or optimizer states because both halves are still updated.
  • To distinguish LoPT from simply freezing the first half, the method updates the lower layers with a reconstruction objective. However, reconstructing input embeddings does not guarantee preservation of the pretrained representations and may alter or even hurt general capabilities. The reported gains are small or mixed in several settings and are evaluated on a limited set of models and tasks, so the generalization benefit is not yet convincing.
  • The placement of the stop-gradient boundary is also underexplored. The paper simply uses the model midpoint as an architecture-agnostic default, without a theoretical argument or a proper split-location sensitivity study. Since different models and tasks may distribute useful representations differently across layers, the midpoint is unlikely to be universally optimal.
Questions for authors
  • How sensitive is LoPT to the location of the stop-gradient boundary? Please report the quality–memory–throughput trade-off for several split layers and explain whether the best boundary can be predicted from model depth, layer representations, or task characteristics rather than fixed at the midpoint.
  • Would a task-aware local objective, such as local language modeling or distillation from the upper block, provide a better adaptation–preservation trade-off than input-embedding reconstruction? How sensitive is LoPT to the choice of the auxiliary objective across different downstream tasks?
3 comments
AuthorsOpenReview ↗

Response to Reviewer q3yt part 1

Thank you for recognizing the clarity of gradient routing, implementation details, and framework compatibility. We address novelty and memory savings, the role of reconstruction, boundary choice, and task-aware local objectives below.

1. Novelty and why memory decreases even though parameters and optimizer states remain

LoPT investigates decoupling task-gradient reach from lower-block trainability in LLM post-training. During training, k1k_1 receives local reconstruction updates while k2k_2 optimizes the SFT/GRPO task objective. We acknowledge in Appendix B.1 that gradient isolation and reconstruction predate LoPT (e.g., InfoPro, and we will also cite Pathak et al., 2022 on local feature reconstruction in transformers). Relative to multi-block local supervision designed for vision classification from scratch, our contribution is this single-boundary persistent asymmetric objective assignment for LLM post-training, along with its execution schedule and SFT/GRPO quality–cost trade-offs.

Your observation on memory components is exact: because both halves update, weights, gradients, and optimizer states do not vanish. Memory savings come from shortening graph and activation lifetimes. Under SF-local-immediate, LoPT finishes k1k_1's local backward and releases the lower backward graph before constructing k2k_2's task graph from the cached detached boundary activation. Table 38 isolates this effect under a common workload: SF-local-immediate lowers steady allocated memory from 66.12066.120 to 58.67758.677 GiB relative to joint graph construction SF-joint (−11.26%-11.26\%), and delivers +12.09%+12.09\% higher input-token throughput than explicit boundary recomputation for +0.200+0.200 GiB. Combined with Table 18 (−15.3%-15.3\% to −35.9%-35.9\% memory and +3%+3\%–+5%+5\% speed across GC, LoRA, and ZeRO-2/3) and Tables 19–20 (PP-LoPT −53%-53\% memory, +22%+22\% speed vs. DDP E2E), the efficiency gains are directly grounded in this execution order.

2. Reconstruction and preservation of pretrained capabilities

Reconstruction supplies a local input-recoverability objective for the task-isolated first half, reconstructing sg⁡(h0)\operatorname{sg}(h_0) from h1h_1. We separate its mechanistic increment from the full recipe's behavior:

  • Mechanistic controls (Appendix N, 3 seeds): Relative to optimizer-matched Freeze-HPmatch (NLL 0.14038±0.000140.14038\pm0.00014), LoPT lowers validation NLL on all three seeds to 0.13975±0.000110.13975\pm0.00011 (Δ=−0.00063±0.00003\Delta=-0.00063\pm0.00003, p=0.00075p=0.00075) and raises ARC-C by +0.68+0.68 pp (p=0.005p=0.005). Replacing reconstruction gradients with per-tensor norm-matched isotropic noise (Random-matched) reverts NLL to 0.14037±0.000110.14037\pm0.00011 (indistinguishable from freezing), confirming a Bonferroni-significant NLL advantage for genuine reconstruction (p=0.00028p=0.00028).
  • Full-recipe SFT/GRPO performance: Across all 12 SFT runs (Table 7), LoPT cuts peak memory by 22.8%22.8\%–35.8%35.8\% and increases throughput by 2.1%2.1\%–19.1%19.1\%; in GRPO (Table 2, Table 17), it cuts policy-update memory by 21%21\%–24%24\% and step time by 7%7\%–21%21\% while winning 7/7 benchmarks on NuminaMath/Llama-3.1-8B (+1.08+1.08 pp).

At the same time, we fully agree that reconstruction does not automatically guarantee invariant representations and have scoped our preservation claims accordingly below.

AuthorsOpenReview ↗

Response to Reviewer q3yt part 2

3. Midpoint default and Q1: boundary sensitivity and predicting the optimal split

The midpoint s=⌊N/2⌋s=\lfloor N/2 \rfloor provides a uniform depth-based rule without per-architecture boundary tuning. Shifting a two-block boundary trades task-facing depth against reconstruction depth; because peak memory depends on sequence length, auxiliary/output head overhead, and checkpointing, equal layer depth does not guarantee equal memory cost. Systems balance can be profiled from per-stage activations, while quality optima depend on task-adaptation needs and require empirical verification. Table 5 evaluates partition count (k=2k=2 vs. k=4k=4, with k=4k=4 trailing on 6/7 tasks), which we will distinguish clearly from single-boundary placement.

4. Q2: Local language modeling or upper-block distillation

Both alternatives can be formulated with a stop-gradient boundary, but each alters the objective and execution dependencies studied here:

  • Local language modeling attaches a direct token-prediction loss to k1k_1, introducing an auxiliary LM-head projection cost while removing the pure task-agnostic maintenance design.
  • Upper-block distillation with detached same-pass upper targets requires executing k2k_2 forward before k1k_1 local backward, breaking SF-local-immediate's lower-backward-first graph release or necessitating cross-step target caching and extra computation.

Evaluating these alternatives therefore requires jointly accounting for adaptation, retention, and target-generation overhead. We frame input-embedding reconstruction as a clean, well-characterized baseline design and note task-aware local objectives as important follow-up directions.

Thanks for the detailed response. However, my core concerns remain unresolved:

  1. Limited Novelty: Combining stop-gradient boundaries with an auxiliary reconstruction loss remains conceptually incremental over standard partial-layer freezing, especially since it achieves similar downstream performance to optimizer-matched freezing while incurring higher memory and lower throughput.
  2. Questionable Role of Input Reconstruction: Updating lower layers via token-embedding reconstruction forces intermediate states to retain low-level input details rather than supporting high-level task abstraction, while also introducing unnecessary feature drift at the boundary. The newly added 3-seed comparison on a single dataset is insufficient to justify this auxiliary objective over simple freezing.

Therefore, I will maintain my original rating of 2: Weak rejection.

Reviewer_hFGvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LoPT, a method that enables LLM post-training cheaper by stopping the task gradient in the middle of the network. The model is cut in two halves: the second half is trained normally on the target task (SFT or GRPO), while the first half never gets the task gradient. To still update the first half, the authors add a small MLP (then removed at inference) that tries to reconstruct the input embedding of each token from the hidden state at the cut, and this reconstruction loss is the only thing that trains the first half. Experiments are done on Qwen3-4B, Qwen2.5-7B and Llama-3.1-8B, with several SFT datasets and with GRPO on math. Compared to end-to-end training, LoPT uses about 24–36% less peak memory in SFT and 21–24% less in GRPO, gets similar or better benchmark results, and in some SFT settings forgets less on held-out benchmarks.

Critical strengths
  1. Simple and practical. The method is easy to implement: it only needs a detach in the middle of the model and a small extra mlp, and the model at inference is unchanged. The memory saving is significant, and the authors show that the method also works together with LoRA, gradient checkpointing and pipeline parallelism, so it can be used in real training setups.

  2. Good results and less forgetting in some SFT settings. In several SFT runs end-to-end training damages held-out benchmarks a lot. For example, with Llama-3.1-8B trained on Alpaca, GSM8K drops from 69.67 to 46.62, while LoPT stays at 72.10. The authors are also honest about the limits of their study, and the appendix has many implementation details and checks.

Critical weaknesses
  1. The method is very close to freezing the first half, so the novelty is limited. The first half never sees the task and is only trained to reconstruct the input embeddings, which has nothing to do with the task. The pretrained hidden states probably already contain enough information to recover the input token, so the small MLP can do most of the work and the first half has little reason to change. Figure 3 shows exactly this: the first half moves orders of magnitude less than with end-to-end training. But for me this figure only shows that the method does what it is built to do, since without the task gradient the first half must move less. It does not explain why the method works. The authors also write that the contribution is "not the invention of reconstruction or gradient isolation", and reconstruction for gradient-isolated blocks was already used before (InfoPro, cited in the paper). In the end the idea is close to freezing the lower half of the network with a small extra loss.

  2. It is not shown that the reconstruction term is needed. The only comparison is with freezing the first half (Table 4). But freezing removes both the reconstruction loss and any update of the first half, so this comparison cannot say whether reconstruction itself matters. No other local loss is tried, and λ_aux is always 10.0. In the only comparison with several seeds, freezing is even slightly better than LoPT . The gain over freezing in Table 4 comes from single runs on one model. This is a problem because Table 3(a) shows that changing the first half by a relative 2×10⁻⁷ already moves the average by 0.54 pp so a difference of 1–1.6 pp from one run is not convincing. A useful control would be for example to update the first half with random changes of the same size as in LoPT.

  3. Few baselines and ablations. The only real baseline is end-to-end training. Other memory-efficient methods that the paper itself cites, like LoRA, are not compared, and the authors say that these comparisons are "outside the reported study". Freezing, the cheapest option, is compared only on quality, not on memory and speed. Also, LoPT gains most where end-to-end training forgets a lot: in the three-seed study, end-to-end training has the best validation loss but the worst benchmark average. In these cases any method that changes the lower layers less (freezing, a lower learning rate, LoRA) would probably look good too. in addition the cut is always in the middle and its position is not studied. The ablation with four blocks changes the number of blocks, not where the cut is.

Questions for authors
  1. Can you report freeze-k1 in all the main settings, with memory, speed and quality, and with several seeds? How big is the gap between LoPT and freezing then?
  2. Did you try other local losses for the first half, different values of λ_aux, or random updates of the first half with the same size as in LoPT? Without these, how can the gain be attributed to reconstruction?
  3. How does LoPT compare with LoRA with the same memory budget? And does end-to-end training forget less with a smaller or tuned learning rate?
3 comments
AuthorsOpenReview ↗

Response to Reviewer hFGv part 1

Thank you for recognizing the practicality, systems compatibility, and SFT performance of LoPT. Your questions regarding optimizer confounding, random-perturbation controls, noise estimates, and tuned/early-stopped baselines prompted our new controlled experiments under Appendix N (Qwen2.5-7B-Instruct / MetaMathQA 100K, 3 seeds), which we detail below.

1. Relation to freezing and InfoPro

LoPT decouples task-gradient reach from lower-block trainability. Pure freezing halts all updates in k1k_1; LoPT enforces ∇θ1Ltask=0\nabla_{\theta_1}\mathcal L_{\mathrm{task}}=0 at the midpoint boundary while keeping k1k_1 adaptable via feature reconstruction and updating k2k_2 with the SFT/GRPO objective. Compared with InfoPro (Appendix B.1), which pairs reconstruction with local task supervision on vision networks trained from scratch, we study a persistent asymmetric objective assignment for LLM post-training together with its cached-boundary execution and quality–cost trade-offs. Figure 3 illustrates the resulting localized parameter drift under this configuration; the optimizer-matched freezing and random-gradient controls below isolate the causal effect of the reconstruction update directly.

2. Q1: Optimizer-matched freeze-k1 and variance quantification

As you noted, freeze-k1 in Table 3(b) used a different optimizer schedule (lr=2×10−5,β2=0.999\text{lr}=2\times 10^{-5}, \beta_2=0.999, 3%3\% warmup) than LoPT (1.5×10−5,0.981.5\times 10^{-5}, 0.98, no warmup). We reran 3 seeds of Freeze-HPmatch using LoPT's exact upper-block optimizer settings and identical per-seed data orders:

Table H1. NLL is validation answer-token NLL used for checkpoint selection. Macro-flex replaces only GSM8K with post-hoc flexible extraction on the same outputs, leaving the other six tasks unchanged. Scores are three-seed means ± sample s.d.; Freeze-HPmatch throughput reports the two runs without co-located evaluation interference.

MethodValidation NLL ↓Macro-flexPeak allocated memory (GiB)Throughput (tokens/s)
E2E, tuned LR0.13735 ± 0.0000468.28 ± 0.2165.941864 ± 18
Original Freeze0.14127 ± 0.0002267.85 ± 0.2244.202933 ± 2
Freeze-HPmatch0.14038 ± 0.0001468.07 ± 0.0344.202914 / 2924
LoPT0.13975 ± 0.0001167.99 ± 0.1558.671841 ± 2

After matching optimizer settings, LoPT's in-domain NLL and ARC-C gains persist on every seed: validation NLL drops by 0.00061,0.00061,0.000670.00061, 0.00061, 0.00067 (mean 0.00063±0.000030.00063\pm0.00003, p=0.00075p=0.00075), and ARC-C improves by +0.77,+0.60,+0.68+0.77, +0.60, +0.68 pp (mean +0.68+0.68 pp, p=0.005p=0.005).

Why does Appendix N show parity on Macro-flex (67.9967.99 vs. 68.0768.07) whereas Table 4 showed +1.24/+1.64+1.24 / +1.64 pp? In Appendix N, E2E exhibits virtually no forgetting (6-task non-GSM8K mean 65.58 vs. Base 65.03). In contrast, under the original protocol of Table 4, LoPT outperforms freeze-k1 by +1.24+1.24 pp (SFT) and +1.64+1.64 pp (GRPO)—and even excluding GSM8K entirely, the remaining six tasks still improve by +1.23+1.23 and +1.66+1.66 pp. We will remove the overly broad Discussion statement that LoPT generally beats pure freezing on retention and separate these protocol-specific findings clearly.

On the 0.54 pp perturbation check (Table 3(a)) and direct variance quantification: Table 3(a) is a base-model sensitivity diagnostic rather than an error bar. Across our 3 training seeds, the macro s.d. is 0.180.18–0.330.33 pp (Macro-strict) and 0.030.03–0.210.21 pp (Macro-flex). Across 24 repeated evaluations of the fixed seed-1002 checkpoint, evaluation s.d. is only 0.01340.0134 pp (consistent with 0.0150.015 pp across 4 repetitions in Appendix P.2), confirming that evaluator variance is negligible relative to training-seed differences.

AuthorsOpenReview ↗

Response to Reviewer hFGv part 2

3. Q2: Isolating reconstruction via the norm-matched random-gradient control

To test whether arbitrary lower-block updates account for LoPT's gains, we kept the entire pipeline unchanged while replacing each k1k_1 reconstruction gradient—prior to clipping and Adam updates—with an isotropic Gaussian direction scaled to the exact same per-tensor L2L_2 norm (Random-matched).

Random-matched obtains validation NLL 0.14037±0.000110.14037\pm0.00011, indistinguishable from Freeze-HPmatch (0.14038±0.000140.14038\pm0.00014, p=0.39p=0.39). By contrast, genuine reconstruction lowers NLL on all three seeds by 0.00063,0.00059,0.000620.00063, 0.00059, 0.00062 (mean 0.00061±0.000020.00061\pm0.00002, p=0.00028p=0.00028, the single test across all 88 exploratory comparisons surviving Bonferroni correction) and improves ARC-C across all 3 seeds (+0.63+0.63 pp). This confirms that the NLL improvement depends on the reconstruction direction rather than isotropic perturbation of equal gradient norm.

4. Q3: LoRA composability and tuned / NLL-matched E2E comparisons

  • LoRA comparison: Table 18 and Tables 24–25 evaluate LoPT and E2E combined with rank-32 LoRA: LoPT+LoRA cuts memory by 35.9%35.9\% (43.8543.85 vs. 68.3768.37 GB/GPU) and boosts speed by ∼4%\sim 4\%, while beating E2E+LoRA on 7/7 SFT benchmarks (+0.67+0.67 pp) and 5/7 GRPO benchmarks (+0.15+0.15 pp).
  • Tuned E2E and NLL-matched controls: The Appendix N E2E baseline already uses an independent HPO search on seed 11 ({5×10−6,10−5,2×10−5,4×10−5}\{5\times 10^{-6}, 10^{-5}, 2\times 10^{-5}, 4\times 10^{-5}\}), selecting 10−510^{-5}. To compare at equal validation fit, we evaluated E2E@781 and WiSE-FT-inspired Freeze-HPmatch–E2E interpolation (α∈{0.0936,0.0839,0.0857}\alpha \in \{0.0936, 0.0839, 0.0857\} chosen solely by validation NLL, matched within 0.000080.00008):
Table H2: MethodValidation NLL ↓Macro-strictMacro-flex
LoPT0.13975 ± 0.0001163.70 ± 0.2767.99 ± 0.15
E2E@7810.13960 ± 0.0002260.67 ± 0.2368.34 ± 0.07
Freeze–E2E interpolation (WiSE-FT)0.13973 ± 0.0001663.99 ± 0.3368.14 ± 0.11

At matched NLL, LoPT outperforms E2E@781 on WinoGrande by +2.16+2.16 pp (p=0.019p=0.019). Furthermore, while post-hoc WiSE-FT achieves similar aggregate scores (though −0.54-0.54 pp below LoPT on ARC-C), it requires running both Freeze-HPmatch and full E2E—including E2E's 65.9465.94 GiB peak memory—whereas LoPT achieves this operating point in a single run with 11.0%11.0\% lower peak memory.

5. Midpoint placement and the four-block ablation

The midpoint serves as an architecture-agnostic two-block default. Shifting a single two-block boundary alters both task-gradient depth and reconstruction-block cost; Table 5 compares partition counts (k=2k=2 vs. k=4k=4, where k=4k=4 drops 6/7 benchmarks). We will explicitly distinguish these two variables in the revised text.

I thank the authors for the rebuttal and for the new experiments. The optimizer-matched freezing baseline and the random update control are exactly the kind of experiments I asked for, and they help to understand the method. However, in my opinion the new results confirm my main concern rather than resolve it.

Q1.

In the only controlled multi-seed comparison, LoPT reaches the same downstream quality as optimizer-matched freezing (67.99 vs 68.07 Macro-flex), while freezing uses less memory and is faster. In this setting LoPT is also not faster than E2E. The advantage of LoPT is limited to a small improvement in validation NLL and on ARC-C, and the larger gains over freezing in Table 4 come from single runs and are not reproduced here.

Q2.

The control with random updates of the same size shows that the reconstruction direction improves validation NLL, which I appreciate. However only NLL and ARC-C are reported for this control, not the downstream average. The other two requests are not addressed experimentally: no other value of λ_aux is tested, and alternative local objectives are only discussed conceptually (in Reviewer's q3yt response) and left as future work.

Q3.

The LoRA results compare LoPT+LoRA with E2E+LoRA, which shows composability but not how LoPT compares with LoRA at the same memory budget. The LISA comparison added for another reviewer does not favour LoPT either, since LISA uses about half the memory with similar Macro-flex. The tuned and early-stopped E2E baselines are only evaluated in the low-forgetting setting where E2E barely forgets and with flexible GSM8K extraction matches LoPT. My question was about the high-forgetting settings of the main tables (e.g. Llama-3.1-8B on Alpaca), where E2E uses a fixed learning rate and where the retention advantage of LoPT is claimed.

For these reasons, I keep my score.

Reviewer_vLN8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LoPT places a single gradient boundary at the transformer midpoint. The second-half block follows the task objective, while the first-half block remains trainable through a lightweight feature-reconstruction objective. Results show competitive quality with lower peak memory and reduced held-out capability degradation in several SFT settings.

Critical strengths
  1. The paper is clearly written and easy to follow.
  2. The proposed method is simple and practical.
  3. The experiments cover multiple models and post-training settings.
Critical weaknesses
  1. Comparisons with related methods are missing.
    LISA and BAdam are cited but not evaluated, leaving LoPT’s quality–efficiency advantage over relevant selective and blockwise optimization methods unclear.

  2. The benefit of the reconstruction objective is insufficiently explained.
    Reconstructing input embeddings introduces no additional supervision beyond the training inputs. Yet the paper does not sufficiently explain why making inputs easier to reconstruct should shape lower-layer representations in ways that improve downstream adaptation or preserve pretrained capabilities.

  3. Comparisons with explicit preservation objectives are missing.
    The paper reports reduced held-out degradation relative to E2E, but does not compare against explicit preservation objectives, such as regularization toward pretrained weights or reference-model KL regularization. Comparing downstream performance, capability retention, and training cost would clarify whether LoPT offers a better adaptation–preservation trade-off.

  4. Sensitivity to the split location remains unclear.
    Table 5 compares two-block and four-block designs, but does not isolate the effect of boundary location. Although the midpoint is presented as a simple default, a split-location sweep with two blocks would clarify how the central design variable affects quality and efficiency.

Questions for authors

See weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Review vLN8 part 1

Thank you for recognizing the clarity, practicality, and multi-model evaluation of LoPT. Following your four suggestions, we ran new controlled baselines under the three-seed Appendix N protocol (Qwen2.5-7B-Instruct on MetaMathQA 100K, 1 epoch, seeds 1001–1003). We respond point by point below.

1. Comparison with LISA, BAdam, and related methods

We added a direct three-seed LISA comparison with an independent learning-rate search. Our LISA implementation keeps the embeddings, final norm, and LM head trainable while randomly resampling 2 decoder layers every 20 optimizer steps and discarding deactivated optimizer states. Using the same four-point LR grid and HPO protocol as E2E, validation answer-token NLL selected 10−510^{-5}. E2E and LoPT entries are taken from Appendix N.

Table V1. NLL is validation answer-token NLL used for checkpoint selection. Task scores are percentages (mean ± sample s.d.). Macro-strict uses the original GSM8K strict-match metric; Macro-flex replaces only GSM8K with post-hoc flexible extraction on the same generated outputs, keeping the other six tasks unchanged.

MethodValidation NLL ↓Macro-strictMacro-flexPeak allocated memory (GiB) ↓
E2E0.13735 ± 0.0000460.75 ± 0.1868.28 ± 0.2165.94
LISA0.13982 ± 0.0000659.93 ± 0.2668.17 ± 0.0231.99
LoPT0.13975 ± 0.0001163.70 ± 0.2767.99 ± 0.1558.67

LoPT matches LISA closely in validation NLL (0.139750.13975 vs. 0.139820.13982) and Macro-flex (67.9967.99 vs. 68.1768.17), while outperforming LISA by +3.00+3.00 pp on WinoGrande (p=0.006p=0.006) and +3.77+3.77 pp on Macro-strict (p=0.006p=0.006) and reducing peak memory by 11.0%11.0\% relative to E2E.

Conceptually, LoPT and parameter-selective methods such as LISA, LoRA, and BAdam operate along orthogonal design axes. LISA selects which layers participate in task optimization over time; LoPT controls task-gradient reach via a persistent midpoint detach() boundary and asymmetric objective assignment, keeping all layers trainable while cutting full-depth backward coupling. Consequently, LoPT composes directly with standard training stacks: Table 18 shows additional memory/speed gains when combined with Gradient Checkpointing (−34.6%-34.6\% mem, +5%+5\% speed), LoRA (−35.9%-35.9\% mem, +4%+4\% speed), DeepSpeed ZeRO-2 (−15.3%-15.3\% mem, +5%+5\% speed), and ZeRO-3 (−29.0%-29.0\% mem, +3%+3\% speed). Moreover, a persistent midpoint boundary enables clean pipeline-parallel decoupling (PP-LoPT vs. DDP E2E: −53%-53\% per-GPU peak memory and +22%+22\% throughput in Tables 19–20), whereas random layer sampling must still propagate task gradients through early layers whenever shallow layers are sampled.

AuthorsOpenReview ↗

Response to Review vLN8 part 2

2. Why input-embedding reconstruction affects lower-block adaptation

Reconstruction requires no extra task labels, yet supplies a distinct self-supervised signal to the first-half block k1k_1:

\qquad \mathcal L_{\mathrm{aux}} =\frac{1}{BLd}\sum_{b,\ell} \left\|g_\phi(h_{1,b,\ell})-\operatorname{sg}(h_{0,b,\ell})\right\|^2,$$ where $h_0$ is the input embedding and $h_1$ is the boundary activation. Reconstructing $\operatorname{sg}(h_0)$ through the lightweight bottleneck MLP (discarded at inference) updates $\theta_1$ via input recoverability without exposing lower layers to narrow task gradients. Two new controls isolate the empirical role of this update: 1. **Optimizer-matched freezing (`Freeze-HPmatch`):** Matching LoPT's upper-block LR, AdamW $\beta_2$, and warmup yields validation NLL $0.14038\pm0.00014$ versus LoPT's $0.13975\pm0.00011$—a consistent reduction on all three seeds ($0.00061, 0.00061, 0.00067$; mean $0.00063\pm0.00003$, $p=0.00075$), alongside a $+0.68$ pp gain on ARC-C ($p=0.005$). 2. **Norm-matched random-gradient control (`Random-matched`):** Keeping the rest of LoPT identical, we replace each $k_1$ reconstruction gradient prior to clipping and Adam updates with an isotropic Gaussian vector rescaled to the exact same per-tensor $L_2$ norm. `Random-matched` obtains NLL $0.14037\pm0.00011$, indistinguishable from `Freeze-HPmatch` ($p=0.39$), whereas genuine reconstruction improves validation NLL over `Random-matched` by **$0.00061\pm0.00002$** ($p=0.00028$—the sole comparison surviving strict 88-test Bonferroni correction) and improves ARC-C across all 3 seeds ($+0.63$ pp). Thus, the in-domain improvement depends on the reconstruction direction itself rather than generic lower-block perturbation. ### 3. Explicit preservation objectives and the adaptation–preservation trade-off Weight regularization and reference-model KL constrain parameter or output drift from an initialization, whereas LoPT controls which layers receive task gradients. To test whether differences in validation fit explain downstream outcomes, we added two NLL-matched controls: early-stopped `E2E@781` and [WiSE-FT](https://arxiv.org/abs/2109.01903)-style interpolation between `Freeze-HPmatch` and `E2E`: $$\theta_\alpha=(1-\alpha)\theta_{\mathrm{Freeze\text{-}HPmatch}}+\alpha\theta_{\mathrm{E2E}}.$$ Per-seed $\alpha$ ($0.0936, 0.0839, 0.0857$) was chosen strictly on validation NLL without looking at downstream scores, matching LoPT's NLL within $0.00008$. | Table V2: Method | Validation NLL ↓ | Macro-strict | Macro-flex | |---|---:|---:|---:| | E2E@781 | 0.13960 ± 0.00022 | 60.67 ± 0.23 | 68.34 ± 0.07 | | Freeze–E2E interpolation (WiSE-FT) | 0.13973 ± 0.00016 | 63.99 ± 0.33 | 68.14 ± 0.11 | | **LoPT** | **0.13975 ± 0.00011** | **63.70 ± 0.27** | **67.99 ± 0.15** | Two points stand out. First, **weight interpolation requires training both endpoints from scratch—incurring E2E's full 65.94 GiB peak memory—plus validation tuning of $\alpha$, whereas LoPT reaches essentially the same fit–retention frontier in a single training run with $11.0\%$ lower peak memory** (and $+0.54$ pp higher ARC-C). Second, at matched NLL, LoPT outperforms `E2E@781` on WinoGrande by $+2.16$ pp ($p=0.019$). Note also that `Appendix N` is a low-forgetting regime where tuned E2E's 6-task non-GSM8K average (`65.58`) stays above `Base` (`65.03`); by contrast, in the high-forgetting SFT settings of `Tables 1, 8–11`, LoPT yields substantially stronger retention on non-GSM8K benchmarks such as IFEval, TruthfulQA, and ARC-C. ### 4. How split location affects quality and efficiency In a two-block partition, split layer $s$ sets both the task-facing depth ($N-s$) and the local reconstruction depth ($s$). Shifting $s$ deeper shortens the task backward graph while lengthening the local graph; shifting it earlier does the reverse. Because peak memory depends on activation lifetimes across both stages, auxiliary/output heads, and checkpointing, memory–throughput behavior cannot be inferred from task-block depth alone. We adopt the midpoint $s=\lfloor N/2 \rfloor$ as a simple, architecture-agnostic default to avoid per-model boundary tuning. `Table 5` tests a distinct variable—partition granularity ($k=2$ vs. $k=4$)—showing that four blocks lower 6 of 7 tasks relative to two blocks. We will clearly separate partition count from two-block boundary placement in the revision.

Thank you for the authors' response and additional experiments. However, the new results have further heightened my concerns. LoPT does not demonstrate clear advantages over existing methods in terms of performance or efficiency, which further raises doubts about the practical value and necessity of the proposed approach. Therefore, I have decided to maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.