- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work introduces LoPT, a local-learning approach that treats task-gradient reach as an explicit design choice for LLM post-training. LoPT splits a Transformer at its midpoint, stops the SFT or GRPO task gradient from entering the first half, and instead updates the lower layers with a lightweight feature-reconstruction objective, while the upper layers learn the original task objective. This separates the two backward graphs and reduces peak activation memory without changing the inference architecture. Across the reported SFT experiments, LoPT achieves competitive but setting-dependent quality while reducing peak memory by about 24–36% and improving throughput by about 2–7%.
Critical strengths
- The paper is clearly written, with intuitive figures and a detailed explanation of the gradient-routing and training procedure, making the method easy to follow.
- LoPT is simple, inference-free, and compatible with existing techniques such as LoRA, gradient checkpointing, and GRPO, while providing meaningful peak-memory reductions in the tested settings.
Critical weaknesses
- The novelty is limited. LoPT mainly replaces one full-depth backward pass with two shorter local backward passes plus an auxiliary head, which is close in spirit to the common engineering practice of freezing the lower layers and tuning only the upper ones. This scheduling can reduce peak activation memory, but it does not remove model parameters, parameter gradients, or optimizer states because both halves are still updated.
- To distinguish LoPT from simply freezing the first half, the method updates the lower layers with a reconstruction objective. However, reconstructing input embeddings does not guarantee preservation of the pretrained representations and may alter or even hurt general capabilities. The reported gains are small or mixed in several settings and are evaluated on a limited set of models and tasks, so the generalization benefit is not yet convincing.
- The placement of the stop-gradient boundary is also underexplored. The paper simply uses the model midpoint as an architecture-agnostic default, without a theoretical argument or a proper split-location sensitivity study. Since different models and tasks may distribute useful representations differently across layers, the midpoint is unlikely to be universally optimal.
Questions for authors
- How sensitive is LoPT to the location of the stop-gradient boundary? Please report the quality–memory–throughput trade-off for several split layers and explain whether the best boundary can be predicted from model depth, layer representations, or task characteristics rather than fixed at the midpoint.
- Would a task-aware local objective, such as local language modeling or distillation from the upper block, provide a better adaptation–preservation trade-off than input-embedding reconstruction? How sensitive is LoPT to the choice of the auxiliary objective across different downstream tasks?
3 comments
Response to Reviewer q3yt part 2
3. Midpoint default and Q1: boundary sensitivity and predicting the optimal split
The midpoint provides a uniform depth-based rule without per-architecture boundary tuning. Shifting a two-block boundary trades task-facing depth against reconstruction depth; because peak memory depends on sequence length, auxiliary/output head overhead, and checkpointing, equal layer depth does not guarantee equal memory cost. Systems balance can be profiled from per-stage activations, while quality optima depend on task-adaptation needs and require empirical verification. Table 5 evaluates partition count ( vs. , with trailing on 6/7 tasks), which we will distinguish clearly from single-boundary placement.
4. Q2: Local language modeling or upper-block distillation
Both alternatives can be formulated with a stop-gradient boundary, but each alters the objective and execution dependencies studied here:
- Local language modeling attaches a direct token-prediction loss to , introducing an auxiliary LM-head projection cost while removing the pure task-agnostic maintenance design.
- Upper-block distillation with detached same-pass upper targets requires executing forward before local backward, breaking
SF-local-immediate's lower-backward-first graph release or necessitating cross-step target caching and extra computation.
Evaluating these alternatives therefore requires jointly accounting for adaptation, retention, and target-generation overhead. We frame input-embedding reconstruction as a clean, well-characterized baseline design and note task-aware local objectives as important follow-up directions.
Thanks for the detailed response. However, my core concerns remain unresolved:
- Limited Novelty: Combining stop-gradient boundaries with an auxiliary reconstruction loss remains conceptually incremental over standard partial-layer freezing, especially since it achieves similar downstream performance to optimizer-matched freezing while incurring higher memory and lower throughput.
- Questionable Role of Input Reconstruction: Updating lower layers via token-embedding reconstruction forces intermediate states to retain low-level input details rather than supporting high-level task abstraction, while also introducing unnecessary feature drift at the boundary. The newly added 3-seed comparison on a single dataset is insufficient to justify this auxiliary objective over simple freezing.
Therefore, I will maintain my original rating of 2: Weak rejection.
Response to Reviewer q3yt part 1
Thank you for recognizing the clarity of gradient routing, implementation details, and framework compatibility. We address novelty and memory savings, the role of reconstruction, boundary choice, and task-aware local objectives below.
1. Novelty and why memory decreases even though parameters and optimizer states remain
LoPT investigates decoupling task-gradient reach from lower-block trainability in LLM post-training. During training, k1 receives local reconstruction updates while k2 optimizes the SFT/GRPO task objective. We acknowledge in
Appendix B.1that gradient isolation and reconstruction predate LoPT (e.g., InfoPro, and we will also cite Pathak et al., 2022 on local feature reconstruction in transformers). Relative to multi-block local supervision designed for vision classification from scratch, our contribution is this single-boundary persistent asymmetric objective assignment for LLM post-training, along with its execution schedule and SFT/GRPO quality–cost trade-offs.Your observation on memory components is exact: because both halves update, weights, gradients, and optimizer states do not vanish. Memory savings come from shortening graph and activation lifetimes. Under
SF-local-immediate, LoPT finishes k1's local backward and releases the lower backward graph before constructing k2's task graph from the cached detached boundary activation.Table 38isolates this effect under a common workload:SF-local-immediatelowers steady allocated memory from 66.120 to 58.677 GiB relative to joint graph constructionSF-joint(−11.26%), and delivers +12.09% higher input-token throughput than explicit boundary recomputation for +0.200 GiB. Combined withTable 18(−15.3% to −35.9% memory and +3%–+5% speed across GC, LoRA, and ZeRO-2/3) andTables 19–20(PP-LoPT−53% memory, +22% speed vs.DDP E2E), the efficiency gains are directly grounded in this execution order.2. Reconstruction and preservation of pretrained capabilities
Reconstruction supplies a local input-recoverability objective for the task-isolated first half, reconstructing sg(h0) from h1. We separate its mechanistic increment from the full recipe's behavior:
Appendix N, 3 seeds): Relative to optimizer-matchedFreeze-HPmatch(NLL 0.14038±0.00014), LoPT lowers validation NLL on all three seeds to 0.13975±0.00011 (Δ=−0.00063±0.00003, p=0.00075) and raises ARC-C by +0.68 pp (p=0.005). Replacing reconstruction gradients with per-tensor norm-matched isotropic noise (Random-matched) reverts NLL to 0.14037±0.00011 (indistinguishable from freezing), confirming a Bonferroni-significant NLL advantage for genuine reconstruction (p=0.00028).Table 7), LoPT cuts peak memory by 22.8%–35.8% and increases throughput by 2.1%–19.1%; in GRPO (Table 2,Table 17), it cuts policy-update memory by 21%–24% and step time by 7%–21% while winning 7/7 benchmarks on NuminaMath/Llama-3.1-8B (+1.08 pp).At the same time, we fully agree that reconstruction does not automatically guarantee invariant representations and have scoped our preservation claims accordingly below.