acceptodds
Under review as a conference paper at ICLR 2027

Post-Training Adaptation of LLMs to Stochastic Weight Perturbations

Abstract

Analog in-memory computing (AIMC) performs matrix–vector multiplications inside memory arrays that store weights, avoiding costly weight movement during LLM inference. The price it pays is stochastic noise on every weight that is fundamentally different from the deterministic rounding error in digital quantization. Hardware-aware training can mitigate the resulting accuracy loss, but requires a full training pipeline with escalating costs as model sizes grow. Post-training quantization (PTQ) requires only a calibration set, but fails to address stochastic weight noise. We propose a post-training robustification (PTR) method that adapts weights using a noise-aware objective: expected layer-output distortion, decomposed into weight-error and noise-variance terms. Weight-Range Adaptation for Robustness to Perturbations (WARP) combines per-channel input scaling and weight clamping with exactly-solved thresholds for each weight group. WARP+ adds GPTQ-style error redistribution on top. We evaluate 0.6B to 32B models of the Qwen3 family in an AIMC simulator with a hardware-derived programming-noise model. WARP+ improves accuracy across all model sizes and completes in 3 hours on 4 A100 GPUs for the 32B model. On Qwen3-14B, WARP+ recovers 40% of the accuracy loss, more than 2× the recovery achieved by PTQ, and reaches within 3.5% points of noise-free accuracy. On Qwen3-32B, the remaining gap is % points. Using WARP+ for PTQ by applying rounding error as weight noise yields a Qwen3-4B model with 3-bit weights at 9.2% accuracy loss, less than half of OmniQuant (19.3%) and GPTQ (22.4%).

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
analog in-memory computing, post-training quantization, post-training robustification, stochastic weight noise, noise-aware optimization, large language models, hardware-aware deployment, weight clipping, model compression, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PihdOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes post-training adaptation for analog in-memory computing. Under independent zero-mean noise, expected layer-output distortion decomposes into adaptation error and noise variance. WARP combines input scaling with noise-aware weight clamping. WARP+ adds GPTQ-style clipping-error redistribution. The threshold problem has closed-form interval solutions for additive noise and a numerical solver for state-dependent PCM noise. Experiments cover Qwen3 models from 0.6B to 32B under simulated analog hardware.

Critical strengths
  1. The distinction between correcting known rounding errors and reducing sensitivity to unknown programming noise is well motivated.
  2. The additive-noise threshold solver is simple and technically appealing. Combining it with function-preserving input scaling provides a practical alternative to hardware-aware retraining.
  3. The evaluation includes multiple model sizes, benchmarks, noise instances, and component ablations. The paper also explicitly acknowledges simulation and model-family limitations.
Critical weaknesses
  1. The stated WARP+ guarantee does not follow for state-dependent noise. Section 3.4 says fixed thresholds preserve the noise term and only reduce weight error. However, PCM noise also depends on each programmed weight's normalized value. Redistribution changes these values even when the thresholds remain fixed.
  2. The authors only show results of their proposed WARP approach against standard PTQ methods, not those developed for analog in-memory computing, e.g., Analog Foundation Models Buchel et al. (2025a), RAOQ Zhang et al. (2024b), etc.
  3. This work only focuses on handling PCM noise. However, in practice, different memory technologies, such as MRAM, RRAM, and SRAM, suffer from different levels and kinds of noise. The applicability and scalability of this work remains unknown.
Questions for authors
  1. Under PCM noise, can you report the two terms of Equation 3 before and after redistribution on representative existing layers? Is the fixed-noise-term claim intended only for additive noise?
  2. Can you provide paired WARP+/WARP differences across noise seeds and clarify the three-bit OmniQuant configuration, including its calibration bit width and use of equivalent transformations?
4 comments
AuthorsOpenReview ↗

Critical Weaknesses: 1 and Question for the authors: 1

Critical Weaknesses: 1:

1. The stated WARP+ guarantee does not follow for state-dependent noise. Section 3.4 says fixed thresholds preserve the noise term and only reduce weight error. However, PCM noise also depends on each programmed weight's normalized value. Redistribution changes these values even when the thresholds remain fixed.

Question for the authors: 1

1. Under PCM noise, can you report the two terms of Equation 3 before and after redistribution on representative existing layers? Is the fixed-noise-term claim intended only for additive noise?

We answer these two points together. You are right: the statement in Section 3.4 is exact only for range-proportional noise, such as the additive Gaussian model. Under the PCM model the noise of an unclipped weight also depends on its own normalized value, which redistribution changes, so fixed thresholds do not fix the noise term. We will correct Section 3.4 accordingly, and thank you for the careful reading.

As requested, we report both terms of Equation 3 under PCM noise, for every configuration of Table 2. The weight error is the output error caused by changing the weights (here, by clipping), and the noise variance is the output error caused by the programming noise. Their sum, the total, is the objective J of Equation 2: the expected squared error of the layer output. All values are summed over 42 layers of Qwen3-14B (blocks 0, 8, 16, 24, 32 and 39, all seven projections) and given as a percentage of the expected squared norm of the clean layer outputs:

ConfigurationWeight errorPCM noise varianceTotal (J)
Off-the-shelf00.80740.8074
Clamping0.07820.47340.5516
Clamping + redistribution0.04220.47430.5165
Input scaling00.53370.5337
WARP (scaling + clamping)0.03340.41290.4463
WARP+ (WARP + redistribution)0.01700.41320.4302

Effect of redistribution. Going from WARP to WARP+, the weight-error term falls by 0.0164 (from 0.0334 to 0.0170, a 49% reduction, between 22% and 87% per layer), while the noise term increases by only 0.0003 (from 0.4129 to 0.4132). Without input scaling the picture is the same: the weight error falls by 0.0360 (0.0782 to 0.0422) and the noise term rises by 0.0009 (0.4734 to 0.4743). The total objective therefore decreases in both cases, although this is an empirical observation under PCM and not a guarantee. Measured on data that was not used for the adaptation, the weight-error reduction is 47% instead of 49%, so it does not come from fitting the calibration set.

Input scaling removes a third of the noise term at zero weight error. Clamping removes more of it (41%) but introduces a clipping error. The two combine: after scaling, clamping reaches a lower noise term with less than half the clipping error (0.0334 against 0.0782), and WARP+ gives the smallest total.

The per-layer tables and the experimental details will be added in Appendix.

AuthorsOpenReview ↗

Critical Weaknesses: 2

2. The authors only show results of their proposed WARP approach against standard PTQ methods, not those developed for analog in-memory computing, e.g., Analog Foundation Models Buchel et al. (2025a), RAOQ Zhang et al. (2024b), etc.

Thank you for the suggestion. We added a comparison with Analog Foundation Models (AFM) on the two models evaluated in that paper, Llama-3.2-1B-Instruct and Phi-3-mini-4k-instruct, using their noise model and their per-channel noise configuration (please read general response Section 3).

WARP+ recovers 43% and 44% of the GSM8K accuracy drop on Llama-3.2-1B and Phi-3-mini, respectively. On Llama-3.2-1B it is within about 2 pp of AFM on both benchmarks (1.2 pp on GSM8K, 2.2 pp on ARC-C). On Phi-3-mini it matches AFM on ARC-C and is 5.8 pp behind on GSM8K. AFM requires full retraining, reported at about 90 and 230 hours on 96 V100 GPUs for the two models, whereas WARP+ runs in about 10 and 20 minutes on a single A100 GPU.

Regarding RAOQ: it is a quantization-aware training method that targets the output quantization of the converters and does not model weight noise, the error source our method addresses. It requires training, and we are not aware of publicly released checkpoints for the models we evaluate, so we did not include it in the comparison.

AuthorsOpenReview ↗

Critical Weaknesses: 3

3. This work only focuses on handling PCM noise. However, in practice, different memory technologies, such as MRAM, RRAM, and SRAM, suffer from different levels and kinds of noise. The applicability and scalability of this work remains unknown.

Thank you for raising this point. We have extended the evaluation in two directions: a further memory technology and a wider range of noise levels. The full tables are in the general response.

1. A further memory technology: ReRAM (general response, Section 6). We added a noise model measured on the ReRAM chip of Wan et al. (2022), using the publicly available fit in AIHWKIT. It differs from the PCM model in shape, with the noise peaking in the middle of the conductance range instead of at its edge, and it covers 1 s, 1 day and 2 days after programming. WARP+ is applied unchanged, with only the noise model in the objective replaced. It improves accuracy in all six settings and recovers 43% to 63% of the accuracy loss on Qwen3-4B and Qwen3-14B. At 1 s it brings Qwen3-14B to within 3.1 pp of its noise-free accuracy. At 1 and 2 days, where the noise is about twice as large, a gap to the noise-free model remains.

2. Noise levels (general response, Section 5). To cover different noise levels independently of a specific technology, we use additive Gaussian noise and extended the sweep on Qwen3-14B from 10% to 20% of the weight range. The off-the-shelf model falls below 10% accuracy at a noise level of 9%, whereas WARP+ retains 56.4% at 10% and 38.4% at 14%, and falls below 10% only at 20%.

AuthorsOpenReview ↗

Questions For Authors: 2

2. Can you provide paired WARP+/WARP differences across noise seeds and clarify the three-bit OmniQuant configuration, including its calibration bit width and use of equivalent transformations?

Paired WARP+ / WARP differences across noise seeds

Both methods are evaluated on the same 10 noise instances, so the differences are paired. For Qwen3-14B, the difference WARP+ − WARP per noise seed, in percentage points:

SeedGSM8KARC-CGPQA-DHumanEval+MBPP+Avg.
0−0.6−0.4−4.5+0.6+0.5−0.9
1+1.1−0.1−1.0+3.1+3.7+1.4
2+1.2+0.2−3.0−2.4−0.3−0.9
3+3.1−0.7+2.0+0.0−1.0+0.7
4+1.2+0.2−2.0−4.3−2.4−1.5
5+1.2−0.1−3.0+0.6+2.4+0.2
6+1.8+0.2+1.5+0.7+0.2+0.9
7+1.5+0.3−2.5−2.4+1.0−0.4
8+3.0−0.5+1.0−1.2−0.3+0.4
9+2.7−0.1+1.0−0.6+2.3+1.1
Mean ± std+1.6 ± 1.1−0.1 ± 0.3−1.1 ± 2.3−0.6 ± 2.1+0.6 ± 1.8+0.1 ± 1.0

On GSM8K, WARP+ is better in 9 of the 10 noise instances. On the other benchmarks the sign varies from seed to seed, and the average over the benchmarks is positive in 6 of the 10 instances. The mean paired differences for all model sizes show the same pattern:

Qwen3 model sizeGSM8KARC-CGPQA-DHumanEval+MBPP+Avg.
0.6B+1.5+2.5+1.6−0.7−0.8+0.8
1.7B+0.0+1.2−0.2+0.3−0.3+0.2
4B+3.9+0.3−0.1−1.4−0.3+0.5
8B+4.6+0.4−1.3+0.0−0.6+0.6
14B+1.6−0.1−1.1−0.6+0.6+0.1
32B+0.7+0.1−1.0−0.4+0.3+0.0

Error redistribution helps on GSM8K at five of the six sizes, by up to 4.6 pp. On GPQA-D it is slightly negative at five of the six sizes, by about 1 pp for the larger models. Averaged over the benchmarks, WARP+ is better than or equal to WARP at every size, by 0.0 to 0.8 pp.

This small effect is consistent with the analysis of Equation 3 in our reply above: after clamping, the weight-error term that redistribution reduces is about ten times smaller than the noise term, so halving it lowers the total objective by only 3.6%.

The per-seed results for all model sizes will be added to Appendix. Based on this analysis we will revise the statement in Section 4.2 that error redistribution "never hurts": it does not hurt on average, and its benefit is concentrated on GSM8K.

OmniQuant configuration in Table 3

Thank you for pointing this out. The caption of Table 3 mentioned only the 4-bit configuration and should have described the 3-bit one as well. The configuration is:

  • Calibration bit width: Each OmniQuant row is calibrated at its own deployment bit width: the W3 row with 3-bit weights and the W4 row with 4-bit weights, both with 16-bit activations (W3A16 and W4A16) and group size 128. The 3-bit row is not a 4-bit calibration rounded to 3 bits.
  • Equivalent transformations. Both rows use Learnable Weight Clipping (LWC) only, without the Learnable Equivalent Transformation (LET), following OmniQuant's weight-only quantization.

We will correct the caption of Table 3 and these details will be added to Appendix.

Reviewer_5XTrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a method of adapting LLM weights on analog computers to reduce stochastic noise. The method, WARP, builds on previous post-training quantization methods by introducing techniques to reduce the analog signal of the weights, due to high signal values at any weight in a channel increasing the noise across the entire channel. This signal reduction is done through input scaling across each output channel, lowering the analog signal while retaining the proportional weight values. Outlier weight values that still exist after this scaling are handled through weight clipping, where a clipping threshold is selected which minimizes the sum of the weight error and the stochastic noise.

Critical strengths

The paper presents a mathematically sound solution to a problem that does not seem to have a pre-existing approach, combining earlier work into a new method which shows excellent potential.

The results section, particularly Table 2, do an excellent job of conveying not only the advantages of WARP+ to other PTQ methods, but also the progressive improvements from applying each individual component of WARP+ (clamping, input scaling, redistribution, etc.).

Critical weaknesses

Figure 2a needs additional information. It goes without saying that applying WARP+ will improve the accuracy drop when compared to off-the-shelf models; how does it compare to other PTQ methods?

Although this limitation is noted by the authors, it would be beneficial for the paper to report results on WARP+ applied to additional LLM model architectures.

Questions for authors

There was a brief discussion of other methods which target different sources of error than WARP+. What benefit, if any, is observed when applying these methods in tandem with WARP+? Could they be used to bring the noisy weights even closer to their original values?

How far can the additive noise discussed in section 4.4 be taken before WARP+ collapses? The accuracy drop-off looks fairly linear in Figure 2b; does that trend continue, or does accuracy reduce more steeply at higher values of α?

4 comments
AuthorsOpenReview ↗

Critical Weaknesses: 1

1. Figure 2a needs additional information. It goes without saying that applying WARP+ will improve the accuracy drop when compared to off-the-shelf models; how does it compare to other PTQ methods?

Thank you for the suggestion. We evaluated OmniQuant at every model size of the Qwen3 family, with the same configuration as in Table 2, and will add it to Figure 2a. We chose OmniQuant because it was the strongest PTQ baseline on Qwen3-14B (Table 2). The other PTQ methods did not improve on the off-the-shelf model there. WARP+ gives the smallest mean accuracy drop at every model size. OmniQuant reduces the drop at four of the six sizes, but at 4B and 8B it is no better than the off-the-shelf model, and at 32B the two methods are comparable (1.8 pp against 1.5 pp). The full table is in the general response, Section 2.

AuthorsOpenReview ↗

Critical Weaknesses: 2

2. Although this limitation is noted by the authors, it would be beneficial for the paper to report results on WARP+ applied to additional LLM model architectures.

The submission was limited to the Qwen3 family. We have extended the evaluation to three models from two further families:

  • Llama-3.3-70B (general response, Section 1): the off-the-shelf model collapses completely under the AIMC pipeline, while WARP+ brings it to within 2.2 pp of its noise-free accuracy (78.5% against 80.7% on average).
  • Llama-3.2-1B and Phi-3-mini (general response, Section 3): WARP+ improves over the off-the-shelf model on both, for example from 17.7% to 29.6% on GSM8K for Llama-3.2-1B and from 54.7% to 65.6% for Phi-3-mini.

These models are dense decoder-only transformers like Qwen3, but they differ in size, training data and training pipeline. We leave other architectures, such as state-space models, hybrid models and mixture-of-experts models, to future work, and have updated the limitations section accordingly.

AuthorsOpenReview ↗

Questions For Authors: 1

1. There was a brief discussion of other methods which target different sources of error than WARP+. What benefit, if any, is observed when applying these methods in tandem with WARP+? Could they be used to bring the noisy weights even closer to their original values?

Thank you for the question. NORA targets the quantization error of the converters by rescaling the activations, which shifts the error burden onto the weights. WARP's input scaling adjusts the same split between activations and weights, with the opposite aim of reducing the weight noise, so the two compete for the same degree of freedom. There is also little left for NORA to gain: WARP+ already removes almost all of the converter loss, reaching within 0.5 pp of the noise-free accuracy at every model size except 8B (1.1 pp) when evaluated with the converters but without programming noise (Appendix F.2). ROMER, in contrast, is complementary: it calibrates the routers of mixture-of-experts models, while WARP+ adapts the weights inside each layer. Our study covers dense models only, so we leave this combination to future work.

AuthorsOpenReview ↗

Questions For Authors: 2

2. How far can the additive noise discussed in section 4.4 be taken before WARP+ collapses? The accuracy drop-off looks fairly linear in Figure 2b; does that trend continue, or does accuracy reduce more steeply at higher values of α?

We extended the sweep from 10% to 20% (general response Section 5, Figure 2b will be updated). The trend continues approximately linearly: the decline steepens only moderately at higher noise levels, without the abrupt collapse seen for the off-the-shelf model and OmniQuant. WARP+ falls below 10% at α = 20%.

Reviewer_JZ3TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To overcome the prohibitive training costs of hardware-aware retraining, the paper introduces post-training robustification (PTR) via WARP and WARP+, a framework designed to adapt large language models for analog in-memory computing (AIMC). By optimizing a novel noise-aware objective that rigorously decomposes expected output distortion into weight-error and noise-variance terms, the method combines per-channel input scaling with hyperparameter-free weight clamping and GPTQ-style error redistribution. Evaluated across the Qwen3 family, WARP+ operates efficiently, completing a 32B model in under three hours on four A100 GPUs, while recovering 40% of accuracy loss on Qwen3-14B and bringing Qwen3-32B within 1.5 percentage points of its noise-free baseline.

Critical strengths
  • The proposed method formulates a rigorous layer-wise optimization goal that exactly decomposes expected output distortion into a weight-error term and a noise-variance term, allowing it to effectively balance clipping error against stochastic programming noise.

  • Operating layer-by-layer using only a small calibration set, WARP+ requires no retraining and completes rapidly making it drastically more cost-effective than hardware-aware retraining alternatives.

  • The approach successfully bridges the performance gap caused by analog hardware non-idealities, recovering 40% of the accuracy loss on Qwen3-14B (more than double the recovery of traditional PTQ methods).

Critical weaknesses
  • Missing analysis on data communication overheads between analog and digital compute paths. Since some layers are handled by AIMC while the remaining layers (e.g., SiLU, Attention, Norm) are transferred to digital datapth, the architectural implications should be discussed even though ICLR is an AI conference.

  • The size of AIMC array is critical in terms of hardware utilization and performance. Since the crossbar array is 512 x 512, how do you deal with decode phase computations, where token-by-token generation increases the amount of GEMV operations?

  • Significant accuracy drop even with the proposed quantization scheme that hinders the usage of AIMC on LLM serving (specifically for the model size from 0.6B to 14B). It is unclear whether the proposed method is effective or an LLM with a larger capacity becomes error-tolerant by itself.

Questions for authors
  • It seems like the input scaling suppresses large weights while increasing the corresponding activation values. Isn't the scaling effect opposite to the need for suppressing activation outliers in LLMs (that SmoothQuant targets for)? Any evaluation on the scaling conflict between the weight outliers (for accuracy preservation on AIMC) and the activation outliers would be greatly appreciated.

  • It is unclear why we need to run LLM workloads using AIMC sacrificing the accuracy so much. Why not utilize digital-based near-memory-processing with DRAM or Flash?

  • Accuracy comparison between the proposed method and the low-overhead fine-tuning method (using LoRA) for compensating analog noise needs to be added. "https://research.ibm.com/publications/efficient-transformer-adaptation-for-analog-in-memory-computing-using-low-rank-adapters"

  • It seems like multiple layers of AIMC arrays are activated at the same time. What would be the power/thermal overhead activating multiple arrays at the same time?

7 comments
AuthorsOpenReview ↗

Critical Weaknesses: 1

1. Missing analysis on data communication overheads between analog and digital compute paths. Since some layers are handled by AIMC while the remaining layers (e.g., SiLU, Attention, Norm) are transferred to digital datapth, the architectural implications should be discussed even though ICLR is an AI conference.

We agree that data communication is an essential architectural consideration for AI accelerators. Reducing repeated weight movement is itself the central motivation for pursuing AIMC. However, the reviewer is correct in pointing out that distributing computation between analog tiles and digital units for auxiliary functions introduces activation transfers. We analyze two approaches that the research community has pursued to address these costs. The first approach is integrated mixed-signal accelerators, which place digital processing close to the analog tiles. In [1], the chip integrates a local digital processing unit next to each tile supporting activation functions, normalization, and partial sum accumulation, along with a global digital processing unit for additional LSTM processing. In [2], the fabricated chip combines analog tiles with digital SRAM-CIM and digital processing units to target auxiliary operations. The architecture proposed in [3] uses a heterogeneous mesh integrating analog tiles with specialized digital units to support CNNs, LSTMs, and transformers. These demonstrations and architectural evaluations establish practical approaches to placing digital datapaths close to analog tiles, reducing the cost of transferring activations between successive processing steps. The second approach is the deployment of LLMs in heterogeneous inference systems, extending the specialization across separate devices and compute substrates. Large autoregressive LLMs introduce attention computation and a growing KV-cache, motivating different hardware for attention and static-weight operations. The study in [4] maps FFNs and attention across NAND-based in-memory computing and DRAM-based near-memory computing devices, respectively, introducing inter-device communication to benefit from hardware specialized for each operator. Related strategies have also been explored using hardware substrates that do not include AIMC [5, 6, 7]. Together, these studies show that local analog-digital integration and disaggregation across specialized hardware can allow the benefits of hardware specialization to outweigh the added activation communication costs. They motivate the complementary algorithmic goal studied here: making LLMs sufficiently robust to analog non-idealities for AIMC to become a practical candidate for static-weight computation. Our experiments address this accuracy requirement, while exact communication costs and end-to-end gains depend on the chosen architecture.

[1] Le Gallo et al., “A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference”, Nature Electronics 6, 680–693 (2023).

[2] Wen et al., “Fusion of memristor and digital compute-in-memory processing for energy-efficient edge computing”, Science 384, 325–332 (2024).

[3] Jain et al., “A Heterogeneous and Programmable Compute-In-Memory Accelerator Architecture for Analog-AI Using Dense 2-D Mesh”, IEEE TVLSI 31, 114–127 (2023).

[4] Liang et al., “DIAMoND: Dynamic Inference for Adaptive Edge MOE with Heterogeneous In-NAND and Near-DRAM Compute Architecture”, ISCA (2026).

[5] Heo et al., “NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing”, ASPLOS (2024).

[6] Yu et al., “Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM”, MICRO (2024).

[7] Sun et al., “Lincoln: Real-Time 50~100B LLM Inference on Consumer Devices with LPDDR-Interfaced, Compute-Enabled Flash Memory”, HPCA (2025).

AuthorsOpenReview ↗

Critical Weaknesses: 2

2. The size of AIMC array is critical in terms of hardware utilization and performance. Since the crossbar array is 512 x 512, how do you deal with decode phase computations, where token-by-token generation increases the amount of GEMV operations?

The AIMC array size affects both hardware efficiency and downstream task accuracy. Larger arrays usually allow for increased memory density and reduced weight fragmentation, i.e., reducing partial results that require digital accumulation. However, they offer less mapping flexibility, can leave more memory cells unused for layers whose dimensions do not align with the array, and can increase crossbar-related non-idealities, such as IR drop. Array size selection in AIMC accelerators, therefore, involves balancing multiple factors. For this study, we adopt 512x512 as a representative hardware configuration, frequently used in fabricated prototypes, architectural evaluations, and accuracy studies [8, 9, 10, 11, 12]. Array size also determines the granularity of tile-level weight normalization and input/output quantization in our studies. Increasing the array size, for example to 1024x1024, makes this granularity coarser and as a result more values share normalization and quantization parameters, increasing sensitivity to outliers, quantization error, and programming noise. Smaller arrays provide finer granularity but require more tiles and partial-result accumulation for a given weight matrix. Here, 512x512 describes a weight tile's input and output feature dimensions, not a requirement for 512 tokens. A single token supplies one activation vector, partitioned into 512-element blocks for larger weight matrices. During decode, these partial results must be combined to produce the output activations. As discussed in our response to Question 1, AIMC architectures commonly integrate digital units to perform this accumulation locally. This provides a mechanism for handling partial results, although the associated performance impact depends on the architecture and mapping. It must be noted, however, that the decode phase is particularly well suited for AIMC. Each sequence only contributes one activation vector per step, limiting the opportunity for conventional accelerators to amortize weight fetches across multiple vectors. AIMC instead computes directly with resident weights across successive steps, addressing the memory-bound character of the decode phase [4]. Thus, while array size indeed influences the quantitative tradeoffs in accuracy and hardware performance, the fundamental motivation of using AIMC during decode remains: eliminating repeated movement of static weights for low-reuse GEMV operations. Our study addresses the underlying requirement of preserving downstream accuracy under analog non-idealities, enabling AIMC to be a viable candidate for this timely application.

[4] Liang et al., “DIAMoND: Dynamic Inference for Adaptive Edge MOE with Heterogeneous In-NAND and Near-DRAM Compute Architecture”, ISCA (2026).

[8] Ambrogio et al., “An analog-AI chip for energy-efficient speech recognition and transcription”, Nature 620, 768-775 (2023).

[9] Pasotti et al., "A 28-nm FD-SOI CMOS Analog-IMC Core Based on PCM Featuring 8 512 × 512-Weight Layers and 28M Weights×TOPs/W/mm2", IEEE JSSC 61, 7, 3193-3207 (2026).

[10] Andrulis et al., "RAELLA: Reforming the Arithmetic for Efficient, Low-Resolution, and Low-Loss Analog PIM: No Retraining Required!", ISCA (2023).

[11] Büchel et al., “Efficient scaling of large language models with mixture of experts and 3D analog in-memory computing”, Nature Computational Science 5, 13–26 (2025).

[12] Büchel et al., “Analog Foundation Models”, NeurIPS (2025).

AuthorsOpenReview ↗

Critical Weaknesses: 3

Thank you for this question. It has two parts, which we answer separately.

The remaining gap for models from 0.6B to 14B. We agree that a gap remains for the smaller models, and we state this in the paper. Two complementary routes can close it further. First, for small models hardware-aware training is feasible. Our new comparison (general response, Section 3) shows that it reaches higher accuracy than WARP+ on some benchmarks, at a much higher cost, so it can be the better choice when retraining is affordable. Second, the noise itself can be reduced at the hardware level: improvements in device technology and programming techniques can lower the error further [14]. Such techniques are complementary to our method, since WARP+ adapts the weights to whatever noise remains.

Is the method effective, or do larger models become tolerant by themselves? Both effects exist and they are separate. Within the Qwen3 family, WARP+ removes a similar share of the accuracy drop at every model size: 40% at 0.6B, 38% at 1.7B, 45% at 4B, 34% at 8B, 40% at 14B and 63% at 32B (Figure 2a). The benefit of the method therefore does not diminish as the model grows. In addition, model size alone does not guarantee robustness. We evaluated Llama-3.3-70B (general response, Section 1): the off-the-shelf model collapses completely under the same analog noise, to 0% on every benchmark, while WARP+ brings it to within 2.2 pp of its noise-free accuracy. A larger model is thus not necessarily inherently noise-tolerant.

[14] A. Vasilopoulos et al., "Exploiting the State Dependency of Conductance Variations in Memristive Devices for Accurate In-Memory Computing," in IEEE Transactions on Electron Devices, vol. 70, no. 12, pp. 6162-6171, Dec. 2023, doi: 10.1109/TED.2023.3321014.

Comments on the paper

8 comments
AuthorsOpenReview ↗

General response

We thank the reviewers for their constructive feedback. This comment summarizes the major new experiments added to the paper. The individual replies refer to the sections below. The revised paper, including these experiments, will be uploaded by the end of the rebuttal period.

Summary of additions

  • Models: a larger model from a second family, Llama-3.3-70B (Reviewers JZ3T, 5XTr).
  • Baselines: OmniQuant evaluated at every model size from 0.6B to 32B (Reviewer 5XTr), a comparison with hardware-aware training, Analog Foundation Models (Reviewer Pihd) and a comparison with adaptation for AIMC via low-rank adapters, AHWA-LoRA (Reviewer JZ3T) .
  • Noise: the additive Gaussian noise sweep extended from 10% to 20% (Reviewers 5XTr, Pihd), and a second measured state-dependent noise model from a different memory technology, ReRAM (Reviewer Pihd).
AuthorsOpenReview ↗

General response 1. Larger model and second model family: Llama-3.3-70B

Accuracy (%) on the five benchmarks under the AIMC pipeline with PCM programming noise (mean ± std over 10 noise instances):

ConfigurationGSM8KARC-CGPQA-DHumanEvalMBPPAvg.
Noise-free93.8 ± 0.293.6 ± 0.242.6 ± 0.386.686.880.7
Off-the-shelf0.00.00.00.00.00.0
WARP+93.1 ± 0.692.0 ± 0.738.0 ± 2.083.3 ± 1.086.0 ± 1.378.5

Under noise, the off-the-shelf model collapses completely, to 0% on every benchmark. With WARP+ it reaches 78.5% average accuracy, within 2.2 pp of the noise-free model. The collapse is caused by a few extreme weight outliers. In several layers of Llama-3.3-70B, single weights reach values of 60 to 125, while the median weight in the same layers is below 0.01. Programming noise scales with the largest weight of each group of 512, so every weight that shares a group with such an outlier receives noise far above its own size: in the v-proj of the first block, 82% of the weights end up with noise larger than themselves. For reference, the largest weight anywhere in Qwen3-14B is 2.5, consistent with its much more gradual degradation. Input scaling and clamping remove these outliers, which is why WARP+ recovers the model. Llama-3.3-70B is both larger than the models in the submission and from a different family than Qwen3, so the result shows that the method is not specific to one model family, and that a larger model is not robust by itself. The results will be added in Appendix.

AuthorsOpenReview ↗

General response 2. OmniQuant across all model sizes

We extended the OmniQuant baseline, previously evaluated only on Qwen3-14B, to every model size from 0.6B to 32B. Average accuracy drop relative to the noise-free model under the AIMC pipeline with PCM programming noise (lower is better):

Qwen3 model sizeOff-the-shelfOmniQuant (LWC)WARP+
0.6B17.4 ± 3.514.4 ± 3.110.4 ± 3.0
1.7B22.3 ± 3.719.1 ± 3.513.9 ± 5.8
4B12.3 ± 3.913.7 ± 3.96.8 ± 3.3
8B7.0 ± 2.17.8 ± 2.34.6 ± 1.8
14B5.8 ± 1.64.7 ± 0.53.5 ± 1.1
32B4.1 ± 1.81.8 ± 1.11.5 ± 1.0

Entries are mean ± one standard deviation over 10 noise instances, in percentage points. The average is taken over the five benchmarks within each instance. WARP+ consistently gives the smallest mean drop at every model size. OmniQuant is less consistent: it reduces the drop at four of the six sizes, but at 4B and 8B it is no better than the off-the-shelf model. At 32B the two methods are comparable (1.8 pp against 1.5 pp, within the spread across noise instances). Figure 2a in the paper will be updated with these results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.