- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
HiLoRe reduces recomputation during the GRPO actor update. For each graph-level recovery unit, it chooses whether to store the state in high precision (H), store a low-precision copy (L), or recompute it (R). These choices are guided by the exact per-token GRPO coefficients , which are available after the forward pass. These coefficients define a "policy-update exposure" score that bounds the gradient error introduced by compressing a unit (Thm. 1). A calibrated version of this score estimates the risk of L. A DP knapsack then selects H/L/R allocations within memory and risk budgets. With rank-16 LoRA on 3–8B models, 2K responses, and memory GC, HiLoRe improves actor-update throughput by 10–13.5% over GC and by 4.9–7.9 points over the best baseline, while keeping downstream scores within 0.6 pp of GC.
Critical strengths
A useful GRPO-specific signal. The coefficients are exact and almost free to compute. They are also quite concentrated: the top 10% of slots account for 52–67% of the mass. As far as I know, previous activation-compression work has not used the update itself to decide state fidelity. I think this is the main idea of the paper, and it may also be usefull for other policy-gradient methods.
A careful evaluation. The authors compare with rematerialization, compression, and hybrid baselines, all using the same tuning budget. They test three models, three tasks, and 1K–4K responses. They also include a memory sweep, a fair packing comparison (Table 3), and gradient checks against GC. Since the timings include all online overhead, I think the speedup is real in the tested setting. Overall, the evalution is quite careful.
Critical weaknesses
The GRPO-specific contribution is limited. In Table 4, H/R allocation alone yields +6.14%, while static-risk H/L/R yields +8.31%. Update conditioning, the core novelty, adds about 1.9 points, bringing the gain to +10.16%. Most of the improvement comes from generic utility-driven scheduling, which is close in spirit to Adacc.
The theory is loosely connected to the method. Thm. 1 uses Jacobian norms, whereas the method uses activation magnitudes and a fitted constant . In practice, fidelity is controlled by an empirical tolerance (), and the risk predictor is only moderately accurate (Spearman 0.77).
The evaluation setting is narrow. The main results use LoRA, 3–8B models, and $\le$4K responses. The full-parameter run uses only one seed. The paper does not report end-to-end GRPO speedup, including rollout, leaving the practical impact of a 10–13% actor-update gain unclear.
Questions for authors
- What is the end-to-end wall-clock speedup, including rollout?
- Several baselines perform below GC, while HiLoRe uses the full memory budget. Did baseline tuning include configurations that use the same budget?
- Does the gain from update conditioning over static risk persist on Phi-3.5, Llama-3.1-8B, and under other memory budgets?
- H/R-only variants show a gradient error of about 0.003, although H and R should be exact. Could this be due to kernel nondeterminism? What is the error between two GC runs?
- The reported 54.8–73.6% zero-coefficient fraction includes padding slots. What is the fraction among valid response tokens?
- What are the one-time profiling and calibration costs, and how often must they be repeated?
9 comments
Response to Reviewer VR3o (Part 2 of 6)
W1/Q3: GRPO-Specific Contribution and Robustness
W1: Update conditioning adds only ~1.9 points; most gains come from generic scheduling.
Q3: Does its gain over static risk persist across Phi-3.5, Llama-3.1-8B and budgets?
Thank you for your careful review and for highlighting our GRPO-specific update signal as strength of the paper. We would like to clarify an important distinction: current-update adaptation is one component of our core contribution, and its 1.85-percentage-point increment does not measure the full GRPO-specific contribution. Static-risk already retains our GRPO-aware risk formulation, with risks fixed at their initialization averages. The controlled ablations below show that the gains from our GRPO-aware risk strategy over the generic-sensitivity alternative exceed Adacc’s overall gains over GC across the three evaluated models.
We use two complementary comparisons:
- Generic-sensitivity vs. Full — value of GRPO-aware risk modeling. Generic-sensitivity replaces GRPO risk scoring with the ALAM-style estimator evaluated in Table 12, using the same calibration data and FP8 gradient-error targets. This comparison retains calibrated risk control and utility-driven scheduling while evaluating the complete GRPO-aware risk strategy.
- Static-risk vs. Full — benefit of current-update adaptation. Static-risk retains each unit’s initialization-average GRPO risk; Full adapts that risk to the current update. This comparison isolates adaptation within GRPO-aware risk modeling.
All variants share recovery units, candidate-storage plans, the FP8 codec, action utilities, and allocator. Comparisons follow the original protocol and common tuning criteria, with all online estimation and scheduling costs included. Task scores summarize three independent training runs per method, with seeds paired across methods and each run evaluated after 60 updates.
Table R1. Measured comparisons of Generic-sensitivity, Static-risk, and Full HiLoRe across three models and additional Qwen memory budgets.
| Model / budget | Method | Peak / $(B_{\mathrm{GC}}$) ↓ | Throughput gain over GC (%) ↑ | Mean grad. error ↓ | MATH500 (%) ↑ | GSM8K (%) ↑ |
|---|---|---|---|---|---|---|
| Qwen2.5-3B / 1.10 | Generic-sensitivity | 1.099 | 4.60 ± 0.37 | 0.0146 | 59.80 ± 0.60 | 76.90 ± 0.19 |
| Static-risk | 1.098 | 8.31 ± 0.51 | 0.0128 | 60.13 ± 0.42 | 77.43 ± 0.19 | |
| Full HiLoRe | 1.096 | 10.16 ± 0.44 | 0.0104 | 60.73 ± 0.31 | 77.84 ± 0.12 | |
| Phi-3.5-mini / 1.10 | Generic-sensitivity | 1.099 | 6.10 ± 0.44 | 0.0142 | 43.73 ± 0.42 | 85.42 ± 0.23 |
| Static-risk | 1.098 | 9.35 ± 0.43 | 0.0123 | 43.93 ± 0.31 | 85.65 ± 0.23 | |
| Full HiLoRe | 1.097 | 12.64 ± 0.47 | 0.0091 | 44.53 ± 0.50 | 86.10 ± 0.19 | |
| Llama-3.1-8B / 1.10 | Generic-sensitivity | 1.099 | 6.90 ± 0.44 | 0.0145 | 54.67 ± 0.61 | 82.46 ± 0.23 |
| Static-risk | 1.099 | 9.70 ± 0.45 | 0.0127 | 54.93 ± 0.42 | 82.71 ± 0.23 | |
| Full HiLoRe | 1.096 | 13.47 ± 0.48 | 0.0100 | 55.53 ± 0.31 | 83.09 ± 0.23 | |
| Qwen2.5-3B / 1.05 | Generic-sensitivity | 1.049 | 1.80 ± 0.32 | 0.0121 | 60.00 ± 0.40 | 77.18 ± 0.23 |
| Static-risk | 1.049 | 4.55 ± 0.42 | 0.0106 | 60.13 ± 0.31 | 77.46 ± 0.19 | |
| Full HiLoRe | 1.048 | 6.24 ± 0.43 | 0.0087 | 60.40 ± 0.40 | 77.56 ± 0.15 | |
| Qwen2.5-3B / 1.15 | Generic-sensitivity | 1.149 | 6.80 ± 0.37 | 0.0148 | 59.67 ± 0.70 | 76.93 ± 0.27 |
| Static-risk | 1.148 | 9.95 ± 0.42 | 0.0140 | 59.93 ± 0.31 | 77.38 ± 0.19 | |
| Full HiLoRe | 1.147 | 12.63 ± 0.46 | 0.0122 | 60.33 ± 0.42 | 77.53 ± 0.19 |
Table R1 supports two conclusions regarding W1 and Q3:
-
W1 — GRPO-specific contribution. At , Full exceeds Generic-sensitivity by 5.56, 6.54, and 6.57 percentage points in throughput gain over GC on Qwen, Phi, and Llama, respectively, with all other recovery and allocation components shared. These increments exceed Adacc’s corresponding overall mean gains of 5.02%, 4.42%, and 6.38% over GC under the same memory ceiling (Table 1). Measured against the same GC baselines, these results demonstrate the substantial contribution of the GRPO-aware risk strategy.
-
Q3 — Robustness of current-update adaptation. Full exceeds Static-risk by 1.85/3.29/3.77 percentage points on Qwen/Phi/Llama at , and by 1.69/2.68 points on Qwen at . Thus, the adaptation benefit persists across the evaluated models and memory budgets.
The GRPO-specific contribution should also be assessed in terms of gradient fidelity and task performance. Across all five settings, Full achieves lower mean gradient error and higher mean MATH500 and GSM8K scores.
We have expanded Section 4.5 to distinguish the contribution of the GRPO-aware risk strategy from the additional benefit of current-update adaptation. The complete comparisons and evaluation protocol have been added to Appendix D.7.
[continued below]
Response to Reviewer VR3o (Part 3 of 6)
W2: Theory, Empirical Fidelity Control, and Predictor Accuracy
W2: The theory is loosely connected to the method. Thm. 1 uses Jacobian norms, whereas the method uses activation magnitudes and a fitted constant . In practice, fidelity is controlled by an empirical tolerance (), and the risk predictor is only moderately accurate (Spearman 0.77).
Thank you for raising this important point. HiLoRe’s risk proxy is constructed directly from the first-order gradient-error term derived from Theorem 1 (Eq. 52), preserving its GRPO-weighted sensitivity–distortion structure. It retains the exact GRPO coefficients and dependency support, approximates relative token sensitivity through activation-based weights, and calibrates the resulting features against measured single-unit gradient errors. The proxy score thus maintains a direct, close connection to the theoretical error term in both its components and structure.
Specifically, Eq. 52 gives
where and is the stabilized reference-gradient norm. Our operational score retains this first-order exposure–distortion factorization:
.
Normalized activation RMS supplies mean-one relative sensitivity weights , while estimates reconstruction distortion. Fitted by state type and layer, maps these features to observed single-unit errors, empirically accounting for effective sensitivity and normalization scales. Calibration remains fixed during training, while exposure tracks the current update.
Computing requires differentiating the parameter-gradient map; reviewer imZF likewise describes exact Jacobian computation as impossible at runtime. As you note, the GRPO coefficients are exact and almost free to compute. Our proxy preserves this exact signal and the theoretical scoring structure using forward reductions and weighted sums, without an additional online backward pass. Theory-guided sensitivity estimation also has precedents in activation compression: GACT uses paired gradient evaluations [1], while ALAM uses gradient-norm variance as a memory-efficient alternative to full-gradient variance [2]. Both support using tractable estimators whose accuracy is evaluated empirically.
HiLoRe’s component ablations directly test these approximations. In post-update diagnostics, removing activation weighting reduces Spearman correlation from 0.78 to 0.67, and removing susceptibility calibration reduces it to 0.74 (Table 13). Both improve the theory-guided score’s predictive value. The Spearman correlation of 0.77 meets common “strong correlation” conventions [3, 4], consistent with reviewer imZF’s observation that empirical alignment looks good. Beyond overall correlation, held-out single-unit interventions across three tasks yield Top-10% recall of 0.78–0.81 and AUROC of 0.86–0.89, exceeding the evaluated alternative signals (Table 12). These results directly support high-risk unit identification.
The theory-guided score governs online allocation through a surrogate risk budget, while selects the operating point using development-set mean complete-update errors. Held-out complete-update checks assess the combined effects of proxy approximation and joint recovery, yielding a mean gradient error of 0.0104 and a maximum observed error of 0.0114 in the primary Qwen setting (Table 14). These checks complement single-unit diagnostics by assessing fidelity under actual training-update allocations. HiLoRe’s throughput advantage also persists across five shared tolerances from 1.0% to 2.0% (Table 17), showing that the efficiency benefit is not specific to .
We have clarified this connection in Sections 3.2, 3.4–3.5 and Appendix C.2, showing how the bound guides the calibrated score and how predictor diagnostics and complete-update checks assess its predictive and operational effectiveness. We hope this clarifies the theoretical basis and empirical support for our allocation strategy.
[1] Liu et al. GACT: Activation Compressed Training for Generic Network Architectures. ICML 2022.
[2] Woo et al. ALAM: Averaged Low-Precision Activation for Memory-Efficient Training of Transformer Models. ICLR 2024.
[3] Schober et al. Correlation Coefficients: Appropriate Use and Interpretation. Anesthesia & Analgesia, 126(5): 1763–1768.
[4] Valentin et al. Incoherence as Oracle-less Measure of Error in LLM-Based Code Generation. AAAI 2026.
[continued below]
Response to Reviewer VR3o (Part 1 of 6)
Dear Reviewer VR3o,
Thank you for your careful review and for recognizing the value of update-aware recovery and the experimental evaluation. To facilitate your review and save time, we summarize our responses, supporting results, and manuscript revisions below.
We sincerely hope that our responses, together with the theoretical clarifications and additional experimental results, help alleviate your concerns. If you find our responses helpful, we would be grateful if you would consider a more favorable assessment of our work. Detailed responses follow.
[continued below]