- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes post-training adaptation for analog in-memory computing. Under independent zero-mean noise, expected layer-output distortion decomposes into adaptation error and noise variance. WARP combines input scaling with noise-aware weight clamping. WARP+ adds GPTQ-style clipping-error redistribution. The threshold problem has closed-form interval solutions for additive noise and a numerical solver for state-dependent PCM noise. Experiments cover Qwen3 models from 0.6B to 32B under simulated analog hardware.
Critical strengths
- The distinction between correcting known rounding errors and reducing sensitivity to unknown programming noise is well motivated.
- The additive-noise threshold solver is simple and technically appealing. Combining it with function-preserving input scaling provides a practical alternative to hardware-aware retraining.
- The evaluation includes multiple model sizes, benchmarks, noise instances, and component ablations. The paper also explicitly acknowledges simulation and model-family limitations.
Critical weaknesses
- The stated WARP+ guarantee does not follow for state-dependent noise. Section 3.4 says fixed thresholds preserve the noise term and only reduce weight error. However, PCM noise also depends on each programmed weight's normalized value. Redistribution changes these values even when the thresholds remain fixed.
- The authors only show results of their proposed WARP approach against standard PTQ methods, not those developed for analog in-memory computing, e.g., Analog Foundation Models Buchel et al. (2025a), RAOQ Zhang et al. (2024b), etc.
- This work only focuses on handling PCM noise. However, in practice, different memory technologies, such as MRAM, RRAM, and SRAM, suffer from different levels and kinds of noise. The applicability and scalability of this work remains unknown.
Questions for authors
- Under PCM noise, can you report the two terms of Equation 3 before and after redistribution on representative existing layers? Is the fixed-noise-term claim intended only for additive noise?
- Can you provide paired WARP+/WARP differences across noise seeds and clarify the three-bit OmniQuant configuration, including its calibration bit width and use of equivalent transformations?
4 comments
Critical Weaknesses: 2
2. The authors only show results of their proposed WARP approach against standard PTQ methods, not those developed for analog in-memory computing, e.g., Analog Foundation Models Buchel et al. (2025a), RAOQ Zhang et al. (2024b), etc.
Thank you for the suggestion. We added a comparison with Analog Foundation Models (AFM) on the two models evaluated in that paper, Llama-3.2-1B-Instruct and Phi-3-mini-4k-instruct, using their noise model and their per-channel noise configuration (please read general response Section 3).
WARP+ recovers 43% and 44% of the GSM8K accuracy drop on Llama-3.2-1B and Phi-3-mini, respectively. On Llama-3.2-1B it is within about 2 pp of AFM on both benchmarks (1.2 pp on GSM8K, 2.2 pp on ARC-C). On Phi-3-mini it matches AFM on ARC-C and is 5.8 pp behind on GSM8K. AFM requires full retraining, reported at about 90 and 230 hours on 96 V100 GPUs for the two models, whereas WARP+ runs in about 10 and 20 minutes on a single A100 GPU.
Regarding RAOQ: it is a quantization-aware training method that targets the output quantization of the converters and does not model weight noise, the error source our method addresses. It requires training, and we are not aware of publicly released checkpoints for the models we evaluate, so we did not include it in the comparison.
Critical Weaknesses: 3
3. This work only focuses on handling PCM noise. However, in practice, different memory technologies, such as MRAM, RRAM, and SRAM, suffer from different levels and kinds of noise. The applicability and scalability of this work remains unknown.
Thank you for raising this point. We have extended the evaluation in two directions: a further memory technology and a wider range of noise levels. The full tables are in the general response.
1. A further memory technology: ReRAM (general response, Section 6). We added a noise model measured on the ReRAM chip of Wan et al. (2022), using the publicly available fit in AIHWKIT. It differs from the PCM model in shape, with the noise peaking in the middle of the conductance range instead of at its edge, and it covers 1 s, 1 day and 2 days after programming. WARP+ is applied unchanged, with only the noise model in the objective replaced. It improves accuracy in all six settings and recovers 43% to 63% of the accuracy loss on Qwen3-4B and Qwen3-14B. At 1 s it brings Qwen3-14B to within 3.1 pp of its noise-free accuracy. At 1 and 2 days, where the noise is about twice as large, a gap to the noise-free model remains.
2. Noise levels (general response, Section 5). To cover different noise levels independently of a specific technology, we use additive Gaussian noise and extended the sweep on Qwen3-14B from 10% to 20% of the weight range. The off-the-shelf model falls below 10% accuracy at a noise level of 9%, whereas WARP+ retains 56.4% at 10% and 38.4% at 14%, and falls below 10% only at 20%.
Questions For Authors: 2
2. Can you provide paired WARP+/WARP differences across noise seeds and clarify the three-bit OmniQuant configuration, including its calibration bit width and use of equivalent transformations?
Paired WARP+ / WARP differences across noise seeds
Both methods are evaluated on the same 10 noise instances, so the differences are paired. For Qwen3-14B, the difference WARP+ − WARP per noise seed, in percentage points:
| Seed | GSM8K | ARC-C | GPQA-D | HumanEval+ | MBPP+ | Avg. |
|---|---|---|---|---|---|---|
| 0 | −0.6 | −0.4 | −4.5 | +0.6 | +0.5 | −0.9 |
| 1 | +1.1 | −0.1 | −1.0 | +3.1 | +3.7 | +1.4 |
| 2 | +1.2 | +0.2 | −3.0 | −2.4 | −0.3 | −0.9 |
| 3 | +3.1 | −0.7 | +2.0 | +0.0 | −1.0 | +0.7 |
| 4 | +1.2 | +0.2 | −2.0 | −4.3 | −2.4 | −1.5 |
| 5 | +1.2 | −0.1 | −3.0 | +0.6 | +2.4 | +0.2 |
| 6 | +1.8 | +0.2 | +1.5 | +0.7 | +0.2 | +0.9 |
| 7 | +1.5 | +0.3 | −2.5 | −2.4 | +1.0 | −0.4 |
| 8 | +3.0 | −0.5 | +1.0 | −1.2 | −0.3 | +0.4 |
| 9 | +2.7 | −0.1 | +1.0 | −0.6 | +2.3 | +1.1 |
| Mean ± std | +1.6 ± 1.1 | −0.1 ± 0.3 | −1.1 ± 2.3 | −0.6 ± 2.1 | +0.6 ± 1.8 | +0.1 ± 1.0 |
On GSM8K, WARP+ is better in 9 of the 10 noise instances. On the other benchmarks the sign varies from seed to seed, and the average over the benchmarks is positive in 6 of the 10 instances. The mean paired differences for all model sizes show the same pattern:
| Qwen3 model size | GSM8K | ARC-C | GPQA-D | HumanEval+ | MBPP+ | Avg. |
|---|---|---|---|---|---|---|
| 0.6B | +1.5 | +2.5 | +1.6 | −0.7 | −0.8 | +0.8 |
| 1.7B | +0.0 | +1.2 | −0.2 | +0.3 | −0.3 | +0.2 |
| 4B | +3.9 | +0.3 | −0.1 | −1.4 | −0.3 | +0.5 |
| 8B | +4.6 | +0.4 | −1.3 | +0.0 | −0.6 | +0.6 |
| 14B | +1.6 | −0.1 | −1.1 | −0.6 | +0.6 | +0.1 |
| 32B | +0.7 | +0.1 | −1.0 | −0.4 | +0.3 | +0.0 |
Error redistribution helps on GSM8K at five of the six sizes, by up to 4.6 pp. On GPQA-D it is slightly negative at five of the six sizes, by about 1 pp for the larger models. Averaged over the benchmarks, WARP+ is better than or equal to WARP at every size, by 0.0 to 0.8 pp.
This small effect is consistent with the analysis of Equation 3 in our reply above: after clamping, the weight-error term that redistribution reduces is about ten times smaller than the noise term, so halving it lowers the total objective by only 3.6%.
The per-seed results for all model sizes will be added to Appendix. Based on this analysis we will revise the statement in Section 4.2 that error redistribution "never hurts": it does not hurt on average, and its benefit is concentrated on GSM8K.
OmniQuant configuration in Table 3
Thank you for pointing this out. The caption of Table 3 mentioned only the 4-bit configuration and should have described the 3-bit one as well. The configuration is:
- Calibration bit width: Each OmniQuant row is calibrated at its own deployment bit width: the W3 row with 3-bit weights and the W4 row with 4-bit weights, both with 16-bit activations (W3A16 and W4A16) and group size 128. The 3-bit row is not a 4-bit calibration rounded to 3 bits.
- Equivalent transformations. Both rows use Learnable Weight Clipping (LWC) only, without the Learnable Equivalent Transformation (LET), following OmniQuant's weight-only quantization.
We will correct the caption of Table 3 and these details will be added to Appendix.
Critical Weaknesses: 1 and Question for the authors: 1
Critical Weaknesses: 1:
1. The stated WARP+ guarantee does not follow for state-dependent noise. Section 3.4 says fixed thresholds preserve the noise term and only reduce weight error. However, PCM noise also depends on each programmed weight's normalized value. Redistribution changes these values even when the thresholds remain fixed.
Question for the authors: 1
1. Under PCM noise, can you report the two terms of Equation 3 before and after redistribution on representative existing layers? Is the fixed-noise-term claim intended only for additive noise?
We answer these two points together. You are right: the statement in Section 3.4 is exact only for range-proportional noise, such as the additive Gaussian model. Under the PCM model the noise of an unclipped weight also depends on its own normalized value, which redistribution changes, so fixed thresholds do not fix the noise term. We will correct Section 3.4 accordingly, and thank you for the careful reading.
As requested, we report both terms of Equation 3 under PCM noise, for every configuration of Table 2. The weight error is the output error caused by changing the weights (here, by clipping), and the noise variance is the output error caused by the programming noise. Their sum, the total, is the objective J of Equation 2: the expected squared error of the layer output. All values are summed over 42 layers of Qwen3-14B (blocks 0, 8, 16, 24, 32 and 39, all seven projections) and given as a percentage of the expected squared norm of the clean layer outputs:
Effect of redistribution. Going from WARP to WARP+, the weight-error term falls by 0.0164 (from 0.0334 to 0.0170, a 49% reduction, between 22% and 87% per layer), while the noise term increases by only 0.0003 (from 0.4129 to 0.4132). Without input scaling the picture is the same: the weight error falls by 0.0360 (0.0782 to 0.0422) and the noise term rises by 0.0009 (0.4734 to 0.4743). The total objective therefore decreases in both cases, although this is an empirical observation under PCM and not a guarantee. Measured on data that was not used for the adaptation, the weight-error reduction is 47% instead of 49%, so it does not come from fitting the calibration set.
Input scaling removes a third of the noise term at zero weight error. Clamping removes more of it (41%) but introduces a clipping error. The two combine: after scaling, clamping reaches a lower noise term with less than half the clipping error (0.0334 against 0.0782), and WARP+ gives the smallest total.
The per-layer tables and the experimental details will be added in Appendix.