- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ProxyProp, a local learning algorithm designed to reduce activation memory during the training of deep neural networks, particularly transformers. The central idea is to attach a trainable auxiliary proxy network to each block, which approximates the output of the remaining network through a distillation objective. Instead of propagating gradients through all subsequent layers, ProxyProp propagates the global output gradient through the local proxy to update each block independently.
The authors provide theoretical analyses showing that an optimal linear proxy recovers the expected downstream Jacobian under Gaussian assumptions and yields exact gradients in deep linear networks. For nonlinear residual networks, they establish a gradient approximation error bound under assumptions of small residual Jacobians.
Experimentally, the method is evaluated on CIFAR-100 with vision transformers and on several language understanding tasks with pretrained LLMs. The results suggest that ProxyProp achieves a favorable accuracy–memory trade-off compared with existing local learning methods and zeroth-order optimization baselines.
Critical strengths
-
Practical motivation. The paper addresses an important problem in memory-efficient training. Reducing activation storage while preserving training performance is particularly relevant to large transformers and long-context fine-tuning.
-
Promising empirical results. ProxyProp achieves competitive performance compared with end-to-end backpropagation while substantially reducing peak GPU memory. The evaluations on both vision transformers and pretrained language models demonstrate the potential applicability of the method across different architectures and tasks.
-
Connection between local learning and gradient approximation. Rather than relying solely on auxiliary local classification objectives, the proposed approach explicitly aims to approximate global backpropagation gradients. The use of output distillation to construct a local gradient pathway is intuitive and provides a useful perspective on local learning.
-
Empirical analysis of proxy design. The paper investigates several proxy architectures and training objectives, including linear and nonlinear proxies, residual parameterization, and gradient-based auxiliary objectives. These comparisons provide useful evidence about the effectiveness of the proposed design.
Critical weaknesses
1. Limited methodological novelty and insufficient justification of key design choices
Using auxiliary modules to approximate global gradient information is not new in local learning. ProxyProp appears closely related to Direct Feedback Alignment (DFA) and synthetic-gradient methods, mainly replacing fixed random feedback with a trainable nonlinear proxy learned through output distillation.
While this extension is practically useful, it remains unclear whether the improvements come from the trainable feedback pathway, proxy architecture/capacity, or distillation objective. More controlled ablations with matched auxiliary architectures and capacities, varying only the training objective or gradient propagation rule, would help clarify the methodological contribution.
2. Limited theoretical insight into the advantage of learned proxies
Proposition 3.4 establishes an gradient approximation error under small residual Jacobians. However, this bound largely follows from the near-identity structure of residual networks and also holds for an identity proxy without training, as acknowledged in Appendix B. It therefore does not explain why a learned proxy should outperform identity feedback or how minimizing the distillation loss improves gradient accuracy.
Moreover, output approximation does not necessarily imply Jacobian approximation, and the theoretical connection in Proposition 3.1 relies on Gaussian assumptions and an optimal linear proxy. A stronger justification would connect distillation error to gradient approximation error, ideally supported by direct measurements of gradient alignment during training.
3. Baseline presentation and interpretation need clarification
Several baseline comparisons are not sufficiently clear:
- Full BP: The full backpropagation result is missing from the main body of Table 1, although reported in its caption and Appendix Table A2.
- Backprop (one block) vs. identity proxy: The former often performs substantially better, but their different objectives and gradient pathways are not clearly explained.
- Local loss: Belilovsky et al. (2019) originally used convolutional architectures. It is unclear how its auxiliary classifier and training objective were adapted to transformers, and whether the baseline received comparable optimization and auxiliary-model capacity.
4. Insufficient gradient-matching evaluation and objective comparisons
Since the main motivation of ProxyProp is to approximate global backpropagation gradients, direct gradient-matching experiments on the target architectures are important. The paper reports gradient alignment in toy networks and a frozen pretrained ViT, but does not sufficiently evaluate gradient approximation quality throughout actual ProxyProp training, across blocks and training stages, or relate it to downstream performance.
Additional comparisons would also help isolate the role of the distillation objective:
- Direct gradient supervision: Train the auxiliary module to predict global or local gradients rather than output representations. Although a DNI-style loss is considered, a simpler gradient-matching objective with detached targets may provide a useful comparison.
- Direct local classification: Optimize rather than output distillation. Appendix Table A2 includes a related baseline, but its implementation and distinction from ProxyProp deserve clearer explanation.
5. Memory complexity analysis and comparison with gradient checkpointing are not sufficiently convincing
The memory analysis in Table A1 and Figure 5 requires further clarification. Training memory includes parameters, optimizer states, and activations. ProxyProp reduces activation storage but introduces additional proxy parameters and optimizer states.
First, Table A1 assigns activation storage to BP with checkpointing and suggests that transformer-specific intermediate activations prevent optimal checkpointing. However, these intermediate activations primarily affect per-layer memory costs rather than necessarily changing the asymptotic dependence on depth. Different recomputation strategies also provide different memory–compute trade-offs.
Second, Figure 5 reports substantially lower memory usage for ProxyProp than BP + LoRA + checkpointing, but the checkpointing granularity and recomputation strategies are unclear. Standard transformer checkpointing retains layer-boundary activations while recomputing internal attention/MLP activations. For a ProxyProp block containing multiple layers, these internal activations may be considerably larger than the block input and should contribute substantially to peak memory during local backpropagation.
Questions for authors
-
Novelty and relationship to DFA: What is the essential methodological distinction between ProxyProp and a DFA-style method equipped with a trainable nonlinear feedback module? Can the authors isolate the improvements due to proxy training, proxy architecture, and the output distillation objective?
-
Proxy capacity: How sensitive is the performance to proxy depth, hidden dimension, parameter count, and computational cost? Would a stronger auxiliary network also substantially improve the local-loss or synthetic-gradient baselines?
-
Training objective: What prevents directly training the proxy using the local task loss ? How does this compare with the proposed distillation objective when the proxy architecture and optimization budgets are matched? Please also clarify how the baseline in Appendix Table A2 differs from this formulation.
-
Gradient supervision: Since the algorithm already computes the global output gradient during the initial forward pass, would directly matching the propagated gradient with a detached target provide a simpler alternative to output distillation? Is the large memory cost of the DNI-style objective inherent to gradient matching or specific to its implementation?
-
Theoretical guarantee: Since the bound in Proposition 3.4 also applies to the identity proxy, what theoretical property of the learned proxy guarantees an improvement over identity feedback? Can the approximation bound be expressed in terms of the distillation error or the proxy Jacobian fitting error?
-
Baseline implementations: What auxiliary architectures and optimization settings are used for local loss, DFA, and DNI in the transformer experiments? In particular, how was the convolutional auxiliary network in the original local-loss method adapted to ViT?
-
Memory complexity: Why does Table A1 suggest that internal transformer activations prevent the standard checkpointing memory–compute trade-off? Could the authors distinguish the asymptotic dependence on network depth from the per-layer activation requirements?
-
Checkpointing fairness: Does ProxyProp in Figure 5 use the same gradient checkpointing strategy as BP + LoRA + checkpointing? For a multi-layer ProxyProp block, how many internal activations are saved during the local backward pass, and what is their contribution to peak VRAM?
4 comments
Response to Questions
Question 1
We refer to our answer for Weakness 1
Question 2
The search space of the proxy architecture is potentially large, so we did not perform an exhaustive search in this search space. We identified that a single layer of transformer backbone with a smaller width (at least half of the main network) was cheap and performed relatively well for training a network from scratch. Deeper proxies are more costly in memory. Across the ViT and LLM fine-tuning tasks of Figure 4, this strategy provided stable performance with little RAM usage of the proxy. There is certainly headroom for studying the impact of the proxy capacity further in future work.
Question 3
We have studied the local loss with proxies in our response to Weakness 1. We have also clarified the implementation in Appendix D.2.1
Question 4
We refer to our answer for Weakness 4
Question 5
As we illustrate in our answer to Weakness 2, we now show that the learned proxy is better than the identity proxy in Proposition 3.5. We have not studied extensively the gradient approximation bound in terms of the distillation error beyond Proposition 3.1. Empirically, the new Figure A1 supports that lower distillation error correlates with better alignment.
Question 6
We refer to our answer for Weakness 3. Instead of augmenting the classifier head of every block with a convolutional layer as done in Belilovsky 2019, we augmented the classifier with a transformer layer matching our proxy in Table A6.
Question 7
Regarding the theoretical scaling analysis of Table A1, we added clarifications in the Table caption and in Appendix A.1. This is because the stored activation within each transformer layer is so large that the cost is close to storing input activations of all layers. We agreed with the reviewer that the Table caption was misleading because the asymptotic scaling remains true in theory although it is not favorable in practice.
Question 8
ProxyProp uses the same gradient checkpointing strategy as BP + LoRA + checkpointing. Since gradient checkpointing caches input activations of each transformer layer, in a transformer with L layers, ProxyProp + checkpointing with B blocks would cache L/B activations, whereas BP + checkpointing would cache L activations.
Thank you for the detailed and thoughtful responses. I particularly appreciate the additional ablations disentangling the contributions of the proxy architecture and the training signal, which demonstrate that both components contribute to the performance improvements.
The revised Appendix A.1 also clarifies my concerns regarding gradient checkpointing and memory efficiency. As a minor suggestion, I would recommend distinguishing standard checkpointing from nested checkpointing, as well as ProxyProp with and without checkpointing, in Table A1. Their forward computational costs are different: approximately (2L) forward operations for BP with standard checkpointing and ProxyProp without checkpointing, versus (3L) for BP with nested checkpointing and ProxyProp with checkpointing. This would make the memory–compute trade-off clearer.
Overall, the responses and revisions have addressed most of my concerns, and I will raise my rating to 3.
Dear reviewer, thank you for acknowledging our rebuttal and generously raising the score.
We also thank you for pointing out the nested checkpointing, which we have not discussed in detail. We agree with you that ProxyProp+checkpointing requires forward FLOPs. Nested checkpointing could also reduce activation memory to by using forward FLOPs. We will clarify this comparison in Table A1 and Appendix A.1.
Response to Weakness
Dear reviewer, thank you for the valuable feedback. We respond to your concerns and questions below:
Even without a non-linear proxy, there are subtle and important differences even between linear ProxyProp and a learned DFA. To show this, we add Appendix Table A5, which focuses on the comparison between DFA and ProxyProp with linear proxies. We add a new baseline (non-residual linear proxy) which is arguably most similar to a learned DFA: a direct feedback initialized at random and learned with our distillation loss. It performs lower than our residual linear proxy baseline because it lacks the residual parametrization (init at identity and regularized to identity), which is crucial in residual networks as explained by Proposition 3.3 to 3.5. These differences are also visible empirically with transformer proxies in Fig. 3 and the performance gain on harder tasks as seen in Fig. 4. Also, to the best of our knowledge, there is no learned DFA variant that has theoretical analysis and approximation guarantees in residual networks (Proposition 3.1 to 3.5).
Regarding synthetic gradients, we have done two controls to verify that our gradient model (eq. 1) and distillation loss (eq. 2) are simpler and cheaper than the DNI without loss of performance.
Design choice justification: In addition to the 10 variants of ProxyProp from Table A4 and A5, we add a control and another table to isolate the distillation and the non-linear proxy capacity. We augment Local Loss with a non-linear classifier head parametrized like the proxy. It matches the capacity of non-linear ProxyProp, but trains local auxiliary networks with BP of local loss instead of equation 1 and 2. This control is consistently worse than ProxyProp with non-linear proxies seen in Table A6.
The bound of 3.4 is valid for the identity proxy. We agree that it is important to show that it holds for learned proxies with better error bounds, thus adding a new Proposition 3.5 to generalize 3.4 to the learned proxy from Proposition 3.1. This bound is better than the bound for the identity proxy when the network is not too strongly non-linear.
In Appendix B, we relax the Gaussian assumption of Proposition 3.1 and obtain a slightly weaker relationship between the true Jacobian and linear proxy Jacobian. Propositions generalized without the Gaussian assumption are in the appendix now.
To confirm empirically the relationship between distillation loss and gradient approximation, we add analysis in the new Fig. A1. Across blocks, we observe a strong correlation between gradient alignment and distillation quality.
The BP result is not inside the table because each column corresponds to a different local block partition, so we report BP above the table. We clarify the implementation of Backprop (one block) in Appendix E.1, with clearer naming in Table A4. The implementation detail of Local Loss is in Appendix D.2.1. Local Loss in Table 1 has a linear head per block, and we add a control with non-linear head matching the proxy in Table A6 (see weakness 1).
We agree that ProxyProp reduces the activation storage but not the parameter and optimizer states and introduces the proxy parameters. We highlight, however, that in contemporary LLM finetuning using LoRA with long context, activations dominate the memory storage, and the overheads of the proxy become negligible.
To provide concrete numbers for Fig. 5 with Llama 8B, from BP to ProxyProp with 8 blocks, only 30Mb of additional memory is needed from rank-64 proxy parameters and optimizers, but activation storage drops from 17Gb to 2.9Gb for a sequence at length 10k. Additionally, we add a new experiment in Table 2 to demonstrate that this gain of ProxyProp in a long context is relevant at scale on contemporary benchmarks. We fine-tune Qwen2.5 32B on mathematical reasoning, where we compare training algorithms under a fixed total RAM capacity. Adding an epoch of ProxyProp training with longer context improves the BP baseline.
Regarding the asymptotic depth-dependence of checkpointing in transformers, we refer to Question 7. Regarding clarifications on checkpointing strategies with ProxyProp, we refer to Question 8.