acceptodds
Under review as a conference paper at ICLR 2027

Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters

Abstract

Analog in-memory computing (AIMC) offers a transformative alternative for model training by executing matrix operations directly where weights are stored. However, scaling AIMC to train modern deep models remains an open challenge due to severe hardware non-idealities, including physical weights with finite dynamic range and write granularity, analog-digital converters with finite resolution, and noisy and asymmetric updates. Guided by the insight that gradient accumulation is sensitive to precision and rounding errors, we adopt a mixed-precision training paradigm: executing forward and backward matrix multiplications in the analog domain while computing weight gradients in the digital domain. To enable scalable training, we present a holistic system-algorithm co-design that co-optimizes weight mapping to ensure well-conditioned physical and logical weight profiles, couples a preconditioned optimizer with threshold-triggered open-loop pulsing to stabilize training trajectories, and aligns converter dynamic ranges to suppress quantization errors. Evaluated via hardware-calibrated architectural simulations calibrated with electrochemical RAM measurements, our framework scales Transformer training up to parameters; writing parameter count as , its validation loss scales as , comparable to for digital training.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Analog AI, in-memory computing, mixed-precision training, Scaling law

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BsyXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose a mixed-precision training framework for AIMC. The approach circumvents some of the undesirable effects of analog training that arise from non-idealities in analog, by leveraging analog devices' characteristically efficient matrix computations for both inference and training while computing gradients in digital. In simulations, the authors show this approach scales comparably with an end-to-end digital pipeline on a Transformer. Thus theirs is highly relevant to the frontier where improvements in efficient training and energy consumption are most badly needed!

Critical strengths

Although my expertise in AIMC is somewhat limited, I found the paper to be highly informative, comprehensive and enjoyable to read . I found the account of non-idealities and attempts to model them generally convincing as far as individual tiles are concerned. I also generally support work in this direction and think a building more efficient training infrastructure is something that is clearly needed and given insufficient attention. This work is a great introduction to the overall state of the literature for backprop based training.

Critical weaknesses

I have several criticisms, none of which are too damning, but all of which , I think, would strengthen this work.

First and most importantly (this will affect my rec. for acceptance):

I believe you need more than one seed. That is a major weakness of these results.

Also, the whole work is built on the assumption that tiles are infinitely accommodating. In practice, some degree of partitioning would be involved. Again, my understanding of physical implementation of these devices is limited, but this is a major issue in the AIMC literature that I am familiar with . In practice this is going to necessarily introduce further non-idealities that your model doesnt really account for . This especially affects your largest experiments and the 'scaling' claim you're trying to make. I think it would be good to be more explicit about this in your limitations section. While my background is not sufficient to comment on this, I think there are also other sources of noise that would warrant passing mention and are not modelled here (again for 'limitations' section) e.g. low frequency /environmental noise , temperature effects , etc.

Secondly, I am more familiar with the literature on AIMC from the perspective of equilibrium propagation. It seems to me that some of the non-ideality issues can be avoided , though imposing additional constraints on architecture etc that make transformers and "scale" at present difficult . Nevertheless, several of the problems which the digital component of your pipeline aims to address are avoided for free with eqprop since we lose the requirement of separate forward and backward MVMs. One can avoid (at least such extensive) DAC/ADC , expensive outer products, storage of moments etc. Since this is another area of interest in the literature on AIMCs, your paper might do well to at least pay lip-service to it, and say, even very generally , why do we need backprop. While the estimates for energy savings from BP are 10 to 100 times, apparently. EP estimates are closer to 10^4 (see the Yi paper below). You can even say "nonetheless, such approaches do not yet scale to frontier data/architectures etc." But some mentioned of the proposed application of local learning on AIMC would be nice, given a shared broader motivation. Likewise the Wright paper has an approach I'm not sure you discuss here. Some more systematic account of the variety of approaches proposed in AIMC and the relative costs and benefits would be useful for readers.

Some references: Yi et al 2023 Activity-difference training of deep neural networks using memristor crossbars; Wright et al 2022 Deep physical neural networks trained with backpropagation Scellier et al 2023 Energy-based learning algorithms for analog computing: a comparative study; Scellier et al 2024 A fast algorithm to simulate nonlinear resistive networks; Nest et al 2024 Towards training digitally-tied analog blocks via hybrid gradient computation

Finally, I think more detailed estimate of the actual benefits/energy-savings/speedups oriented to be expected from your implementation vs. other proposed approaches would help a less specialized reader like myself to better assess the state of the literature and where your method falls in terms of concrete benefits. Since analog is in a relatively early state of development, we would like to make a realistic case for its application and try to anticipate some of the roadblocks that prevent money from flowing into this infrastructure. Your paper does a good job at stating from a high level "we can avoid this, this, and this issue and achieve backprop like scaling." It doesn't really speak directly to the advantages of your approach. Can we expect the "ten or 100 times" improvements in energy efficiency here , or is that no longer applicable since digital is picking up a significant share of the work? etc.

Questions for authors

I have none other than the above. I would really like to see more seeds for the main result. I'm curious if this will take prohibitively long.

4 comments

Thank you for finding the paper informative and for recognizing the relevance of this research direction. We address each concern below, in the order raised: multiple seeds, finite-array partitioning, additional hardware effects, alternative learning approaches, and concrete energy and speed benefits.

Q1: More than one seed is needed

Thank you for highlighting the need to quantify run-to-run variability. We have now completed a three-seed comparison (seeds 1337, 1338, and 1339) at all model scales. The results are already updated in the revision.

ScaleParametersMP SGDMP Adam+ Converter alignment+ Initialization (ours)Digital AdamW
Tiny6.8M10.7718 ±0.014310.6035 ±0.01906.8110 ±0.01346.5050 ±0.01616.3567 ±0.0129
Small16.0M10.8554 ±0.024710.5566 ±0.03945.9143 ±0.04285.6263 ±0.00635.3112 ±0.0399
Medium29.9M10.7980 ±0.056010.7397 ±0.08695.2073 ±0.05284.7461 ±0.08234.3988 ±0.0634
Large50.9M10.8760 ±0.020210.7071 ±0.01594.6799 ±0.01783.9795 ±0.02243.8390 ±0.0176
XLarge123.6M10.7313 ±0.030110.8410 ±0.03843.4903 ±0.03753.3459 ±0.03753.2526 ±0.0351

Q2: Also, the whole work is built on the assumption that tiles are infinitely accommodating. In practice, some degree of partitioning would be involved.

Thank you for highlighting the physical partitioning requirement. Full-network training has completed at all four tested tile limits through Large and at the 1024×10241024\times1024 limit for XLarge. Smaller tiles can reduce quantization error, but reconstructing the full output requires summing multiple partial reads. The completed comparison shows convergence without a monotonic loss trend across tile sizes, with execution differences retained as a limitation; see our response to Reviewer zV7w (W1/Q2) for the endpoints and App. C.2 of the revision for the partitioning scheme and modeling scope.

Q3: I think there are also other sources of noise that would warrant passing mention and are not modelled here (again for 'limitations' section) e.g. low frequency /environmental noise , temperature effects , etc.

Thank you for highlighting these effects. The training simulations represent device and cycle variability, noisy updates, and effective stochastic read-path perturbations, alongside quantization and clipping. They do not explicitly model the temporal correlations of low-frequency noise, environmental disturbances, temperature-dependent device and converter behavior, or long-term drift. Effective read noise should therefore not be interpreted as validation against these mechanisms.

We will make these omissions explicit in the limitations and distinguish the assumed calibrated operating point from an implemented calibration or temperature-tracking procedure. Separate circuit tests with injected thermal, 1/f, and random-telegraph noise examine selected noise-propagation mechanisms, but do not establish measurement-calibrated robustness of the training runs; see our response to Reviewer D4o6 (Q1/W2).

Q4: Why use backpropagation?

Thank you for your insighful discussion! EP is a particularly interesting idea with strong potential for analog training. EP and related activity-difference methods derive local learning signals from relaxed network states, offering a complementary route to explicit backpropagation. We use BP here to retain compatibility with the decoder-only Transformer and AdamW pipeline evaluated in this work. In Appendix A.2 of the revision, we now discuss the suggested works by Yi et al. (2023), Scellier et al. (2023), Scellier (2024), Wright et al. (2022), and Nest and Ernoult (2024), distinguishing local energy-based learning, simulation advances, physics-aware BP, and hybrid BP–EP.

We also believe several design principles in this work remain useful for hardware designed for EP: high-precision W-grad accumulation, particularly when small phase-contrast updates are accumulated across samples, and alignment of physical and logical weights with the conductance operating region and network signal scales. Differences between nearby equilibrium states can produce small net updates that are sensitive to quantization and noise, motivating investigation of how much accumulation precision is needed. These are potential extensions rather than EP requirements established by our BP experiments. Appendix A.2 outlines these opportunities; evaluating them in EP hardware is left to future work.

Q5: What concrete energy savings or speedups should be expected relative to other approaches? Are 10–100× improvements still applicable when digital computation handles a significant share of the work?

Thank you for requesting a clearer assessment of practical energy savings and speed.

Compared with a purely digital baseline, offloading the forward and backward linear projections onto analog arrays while keeping weight-gradient outer products, attention, and optimizer updates in digital yields an idealized compute-energy reduction of up to ~66% (2/3), reflecting that two of the three primary matrix multiplications per linear layer (forward pass and backward activation propagation) are executed in the analog domain, assuming negligible analog array energy and that linear projections dominate total FLOPs at scale.

End-to-end benefit also depends on tile parallelism, converter sharing, digital gradient computation, and programming latency. We therefore do not claim a quantified training speedup from the present simulations.

Our contribution establishes scalable training under the modeled hardware non-idealities, addressing a prerequisite for realizing analog efficiency gains. Quantifying those gains requires a specified hardware implementation and an energy–latency comparison at a common target validation loss.


We appreciate your constructive suggestions and we hope our response resolves your concerns. Thanks!

I am satisfied that my main concerns have been addressed. I intend to reread the full paper and other rebuttals before considering with an open mind my final score. I will at least not be lowering my recommendation/score which I would have done had the seed issue not been addressed.

Thank you very much for your feedback and support! We would be happy to discuss further if there are any follow-up questions.

Reviewer_zV7wOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors consider training of Transformer neural network in a mixed-signal (digital and analog) hardware environment using in-memory compute. They propose a hybrid method in which some operations are performed in the analog domain and others are performed in the digital domain. This is a pretty standard approach, whether for in-memory compute (the authors' chosen hardware) or other analog mixed-signal hardware.

Critical strengths

This paper is reasonably well written, which would make it stand out as an ICLR 2027 submission.

Critical weaknesses

The paper's framing suggests this is an algorithmic approach to solving the scaling of training on in-memory compute devices, but a key evaluation is missing in the evaluation. When performing matrix-vector multiplications, analog mixed-signal devices typically will break matrices down into tiles such that the result of a single MVM is the sum of partial results, one from each tile. A standard paper of this sort would straightforwardly perform an evaluation across tile sizes to illustrate to the reader the efficacy of the approach in a real system. (Note Appendix C.5, p. 25 “without array-size tiling limits.”).

Questions for authors

This reviewer does not have experience with in-memory compute, but their experience with other analog mixed-signal technologies indicates that matrix-vector multiplications are typically performed over tiles. Can you run your simulations using a variety of tile sizes and report results?

Can you report results over multiple seeds and provide error estimates?

1 comment

Thank you for recognizing the writing and for identifying finite-array evaluation as a central issue. We address matrix partitioning, the distinction from established mixed-precision architectures, and seed variability below.

Q1: Mixed-precision method is pretty standard.

We thank the reviewer for asking us to clarify the novelty. We agree that mixed-precision analog–digital training is established, and we build on Nandakumar et al. (2020) and the noise/bound-management techniques of Gokmen et al. (2017). Our contribution is coordinating weight representation and converter operation to preserve trainability as Transformer size increases, supported by theory and component ablations.

1. A specific scale-alignment design. Stable initialization requires logical-weight standard deviations to decrease with the inverse square root of layer width. Applying this shrinking scale directly to physical conductances wastes device range and makes finite write granularity increasingly consequential. Our mapping separates these requirements: it maintains a width-independent target conductance spread while preserving width-dependent logical initialization. Converter ranges are then aligned with the resulting forward and backward signals. This explicitly connects network initialization, physical weight representation, and converter precision.

2. A theoretical basis for converter alignment. We formulate converter-range selection over ranges proportional to vector norms, jointly considering the norm and its multiplier. Under the stated assumptions, we prove that maximum-based alignment approaches the minimum expected reconstruction distortion within this family as converter precision increases. Finite-precision bounds characterize its dependence on signal dimension and distribution tails.

3. Evidence beyond adopting mixed precision or Adam. At 123.6M parameters, the reported validation losses are:

ConfigurationValidation loss ↓
Mixed-precision Adam control10.8410
Adding converter alignment3.4903
Adding proposed initialization and mapping: full method3.3459
Digital control3.2526

These ablations retain the mixed-precision partition and start with Adam already present. Converter alignment substantially improves training, and the proposed initialization and mapping further reduce the gap to the digital reference. The controls adapt prior design choices to our Transformer setting. They demonstrate that the architectural partition and optimizer choice alone do not explain the achieved performance; the coordinated alignment contributes materially. We will sharpen this distinction in the contribution statement and discussion of prior work.

W1 / Q2: Run the simulations using a variety of tile sizes and report results

Thank you for highlighting the need for finite-array validation. The comparison below reports completed full-network training. We partition every analog matrix, including the tied embedding and vocabulary projection, into subarrays. Each subarray applies converter range management, DAC/ADC conversion, and read noise independently in both forward and backward computation; converted partial sums are accumulated digitally in FP32.

The table reports validation cross-entropy after the full training budget (lower is better) . Due to time constraints, we have not yet completed three-seed verification of the finite-array experiments. We will complete this verification and report the mean and sample standard deviation across seeds in the next revision.

Scale∞×∞\infty\times\infty1024×10241024\times1024512×512512\times512256×256256\times256128×128128\times128
Tiny6.5050 ± 0.01616.58866.57006.57356.7745
Small5.6263 ± 0.00635.57275.65705.56575.5764
Medium4.7461 ± 0.08234.55444.53884.57144.5505
Large3.9795 ± 0.02243.91973.89883.90373.8988
XLarge3.3459 ± 0.03753.33323.33213.34373.3310

Partitioning introduces two competing effects. Shorter dot products with per-subarray range management can reduce quantization error, but reconstructing the full output requires summing multiple partial reads, whose read-noise contributions also accumulate. Therefore, the effect of reducing array size is not monotonic; see App. C.2 of the revision for detailed discussion.

Q3: Multiple seeds and error estimates.

Thank you for highlighting the need to quantify run-to-run variability. We have completed three independent seeded runs (1337, 1338, and 1339) for both SAMP and Digital AdamW at all five model scales. Revised Table 1 reports the following validation cross-entropy values. See the table in our response to Reviewer BsyX (Q1).


We hope this response addresses the finite-array and seed-variability concerns and clarifies the intended contribution. Thank you for your feedback.

Reviewer_D4o6OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To enable scalable training using analog in-memory computing, the authors present a holistic system-algorithm co-design that co-optimizes weight mapping to ensure well-conditioned physical and logical weight profiles, couples a preconditioned optimizer with threshold-triggered open-loop pulsing to stabilize training trajectories, and aligns converter dynamic ranges to suppress quantization errors. However, the simulation is very unrealistic, without even a convincing tool. ALso the ADC and DAC error and shifting is not considered. This paper is a bit out of the scope of the conference.

Critical strengths

This paper tries to address the hardware non-idealities, including physical weights with finite dynamic rate and write granularity, analog-digital converters with finite-resolution, and noisy and asymmetric updates.

The authors adopt a mixed-precision training paradigm: executing forward and backward matrix multiplications in the analog domain while computing weight gradients in the digital domain.

Critical weaknesses
  1. The authors adopt a mixed-precision training paradigm: executing forward and backward matrix multiplications in the analog domain while computing weight gradients in the digital domain. -> This is not enough innovation, and it seems to be the key innovation of this paper. The

  2. The reviewer is very much concerned with the simulation environment. The reviewer understands that real experiments are very difficult for analog computing. But at least the authors should consider realistic experiments such as Spice or device simulation tools. The device and crossbar characteristics need to be extracted from these low-level simulations. It seems that only the range and quantization of ADC/DAC are considered. There are multiple prior work in this domain.

  3. For analog computing the crossbar size cannot be too big, which does not match well with transformers which require long and big matrix multiplications. There is lack of discussion here.

  4. The results are also confusing. There is no accuracy or benchmarking. The results are clearly not enough. The validation loss scaling seems to be the only result, and it is only a linear regression fitting.

Questions for authors

Please see above.

4 comments

Thank you for recognizing our effort to address finite device range, write granularity, converter resolution, and noisy asymmetric updates. We address your concerns about novelty, simulation fidelity, finite crossbar size, converter modeling, and conference scope below.

Q1 / W2: the ADC and DAC error and shifting is not considered. At least the authors should consider realistic experiments such as SPICE or device simulation tools. The device and crossbar characteristics need to be extracted from these low-level simulations. It seems that only the range and quantization of ADC/DAC are considered.

Thank you for highlighting the need for realistic simulation. We first clarify our post-calibration DAC/ADC abstraction, then present supporting SPICE results, and finally discuss the model limitations and the scope of the evidence.

Post-calibration converter abstraction. We consider DACs and ADCs operating after calibration and compensation. Therefore, it does not explicitly consider error and shifting. This operating assumption is motivated by analog/digital calibration methods that reduce systematic offset and gain errors [R1.1, R1.2], together with characterization and compensation of temperature-dependent shifts [R1.3]. We represent finite codebooks and clipping with the saturating quantizer in (1), and residual stochastic disturbances with effective random noise:

\qquad \operatorname{ADC}(z)=Q_{K_{\mathrm{ADC}},C_{\mathrm{ADC}}}(z+\xi_{\mathrm{ADC}}),$$ and the MVM computing $$y=sC\operatorname{ADC}\!\left((\widehat W-\widehat W^\diamond+\xi_{\mathrm W})\operatorname{DAC}_{\mathrm{eff}}(x/C)+\xi\right),$$ where $\xi_{\mathrm{DAC}}$, $\xi_{\mathrm{ADC}}$, and $\xi_{\mathrm W}$ represent effective noise at the DAC output, ADC input, and weight respectively. Their distributions and hardware-specific parameters should be characterized using circuit-level simulations or hardware measurements at the relevant operating point. The model thus captures two sources of hardware error that remain after offset and gain calibration: quantization due to finite codebooks and stochastic circuit noise. **SPICE validation and parameter extraction.** We implemented a schematic-level SKY130 crossbar read chain with a switched R–2R DAC [R1.4], transistor buffers and readout, and a sample-and-hold SAR ADC using a StrongARM comparator [R1.5], which explicitly includes the following non-idealities: - Parasitic line resistance and capacitance, series selector resistance, and array IR drop. - Branch loading and asymmetry in the differential readout of nonnegative-conductance main and reference arrays. - Switch on-resistance, finite drive capability, nonlinearity, and finite settling time in the DAC and buffers. - Finite gain, bandwidth, output swing, port loading, and input-dependent error in the TIA and differential readout. - Quantization, comparator offset, finite settling, and sampling/conversion dynamics in the SAR ADC. - MOS threshold-voltage and oxide-thickness variations, and resistor/capacitor mismatch, held fixed within each virtual chip across repeated reads. - ECRAM read noise [R1.6], peripheral MOS white, 1/f, and high-frequency noise represented by finite-band surrogates, and cell/peripheral-resistor thermal noise. The results show that the native AIHWKit and SPICE pre-ADC outputs differ by 0.066–0.125 ADC LSB RMS across 2×2, 4x4, 8x8 arrays in both read directions. We fit the effective variances to the SPICE results, obtaining standard deviations of $\xi_{\mathrm{DAC}}: 0.072$, $\xi_{\mathrm{ADC}}: 0.0098$, and $\xi_{\mathrm W}: 0.056$. The fitted model yields comparable overall distribution to SPICE; See App. C.2 in the revision. We have also completed Transformer training reruns using these fitted noise coefficients; See revised Tab. 1. We acknowledge that our training model is a computationally tractable architectural abstraction, rather than a full circuit-level simulation. Residual errors from imperfect calibration and temporal or cross-channel noise correlations are not explicitly modeled; incorporating these effects is left to future work. We will discuss the scope of the abstract in App. C.2. **Scope of the SPICE validation.** We acknowledge that parasitic IR drop, line capacitance, and peripheral loading scale nonlinearly with array dimensions. Accordingly, we view these SPICE experiments as a circuit-level sanity check validating our read-chain and converter abstraction rather than a full-scale array simulation. In practice, physical crossbars are partitioned into smaller tiles specifically to bound IR drop and settling delays within manageable limits. Extending circuit-level characterization to larger tile sizes is left for future work.

Q2: This paper is a bit out of the scope of the conference.

Thank you for raising the question of relevance. The central hypothesis of our paper is that Transformer training can retain its scaling behavior on hardware with non-ideal computation when the numerical interface between the learning algorithm and the hardware is appropriately aligned. Scaling behavior is a central question in modern generative AI: it concerns how predictive loss improves as model size and training resources increase. We investigate how finite precision, limited dynamic range, and imperfect weight updates affect this behavior, and which design principles allow systematic improvement with scale to continue. We therefore believe that investigating the persistence of Transformer scaling under non-ideal computation falls within the conference's scope.

W1: Mixed-precision training paradigm is not enough innovation, and it seems to be the key innovation of this paper.

Thank you for your valuable comment on the innovation. We would like to clarify that our main methodological contribution is the three-fold scale-alignment: physical conductance, logical weight scaling, and converter input ranges; see our response to Reviewer zV7w (Q1).

W3: There is lack of discussion about the crossbar size.

Thank you for highlighting the need to discuss finite crossbar sizes. We now report completed full-network training with 1024×10241024\times1024, 512×512512\times512, 256×256256\times256, and 128×128128\times128 physical array limits through Large, and with 1024×10241024\times1024 arrays at XLarge. Each subarray uses its own DAC/ADC range management; the completed results show convergence without a monotonic improvement as arrays become smaller. See our response to Reviewer zV7w (W1/Q2) for the endpoint comparison and App. C.2 of the revision for the partitioning scheme and modeling scope.

W4: The results are also confusing. There is no accuracy or benchmarking. The results are clearly not enough. The validation loss scaling seems to be the only result, and it is only a linear regression fitting.

Thank you for highlighting the need to make the evaluation clearer. Our results include both language-model validation loss and task-level accuracy, measured directly from training and evaluation runs in our architectural simulator. We clarify the main experimental evidence and the purpose of regression below.

1. Measured language-model performance and convergence. The central results are Table 1 and Figure 6. We train Transformers of different sizes from scratch on OpenWebText. Table 1 reports the validation losses recorded at the scheduled evaluations near the end of these training budgets; Figure 6(d–e) shows the recorded validation-loss trajectories throughout training. The tabulated losses, plotted measurement points, and convergence trajectories come from the actual runs; they are not generated by linear regression.

2. Regression is used for quantifying the benefit of scaling. Following the scaling-law perspective of Hoffmann et al. (2022), we ask whether analog training can benefit from increasing Transformer size and the associated training-token budget, as digital training does. The linear regression in log–log space in Figure 6(a–c) estimates the rate at which the measured validation loss decreases with scale, using the descriptive relationship L=AX−αL=A X^{-\alpha}. The observed convergence and similar fitted rates support the conclusion that analog training benefits from scaling over the evaluated range like digital training.

3. Downstream language-task accuracy. Beyond validation loss, App. F.1 (Table 5) reports zero-shot accuracy of the independently trained digital and analog checkpoints on 22 downstream tasks: 10 commonsense/multiple-choice reasoning tasks and 12 language-understanding tasks, including HellaSwag, PIQA, ARC, GLUE, and SuperGLUE tasks. These accuracy values are obtained by evaluating the trained checkpoints on the selected task splits under a common length-normalized likelihood protocol.

4. Benchmarks beyond language modeling. In the revision, we also evaluate CNNs on CIFAR-10 and ViTs on CIFAR-10/CIFAR-100, comparing the proposed method and baselines; see Tables 9 and 11. Representative image-classification results are shown below.

BenchmarkDigital controlDigital accuracyAGAD accuracySAMP accuracy
CNN9, CIFAR-10SGD90.6968.9988.41
ViT V25, CIFAR-10Adam71.0958.8067.60
ViT V25, CIFAR-100Adam42.7925.1242.69

We hope the response above resolves all concerns and appreciate your reconsideration of the rating of our paper. We are keen to engage in further discussion to ensure the clarity and rigor of our work. Thanks!

[R1.1] Le Gallo, et al. Demonstration of 4-quadrant analog in-memory matrix multiplication in a single modulation. npj Unconventional Computing, 2024.

[R1.2] Analog Devices. Adjusting and Calibrating Out Offset and Gain Error in a Precision DAC. Application note, 2009. Source.

[R1.3] Freescale Semiconductor/NXP. How to Increase the Analog-to-Digital Converter Accuracy in an Application. Application Note AN5250, 2016. Source.

[R1.4] Kester. Basic DAC Architectures II: Binary DACs. Analog Devices Tutorial MT-015, Rev. A, 2008. Source.

[R1.5] Razavi. The StrongARM Latch [A Circuit for All Seasons]. IEEE Solid-State Circuits Magazine, 7(2):12–17, 2015.

[R1.6] Su, et al. Noise and Drift Models for Protonic ECRAM and Their Impact on Analog In-Memory Computing. Advanced Computing, 2026.

AuthorsOpenReview ↗

Circuit-Level Validation and Planned Calibrated Transformer Experiments

Dear Reviewer D4o6:

Thanks for recommending circuit-calibrated simulations. Building on our earlier experiments, we have extended our SPICE evaluation to 8×8, 64×64, and 128×128 arrays. The preliminary DC results below show the effect of per-channel offset correction using zero-input DC simulations.

Array sizeOriginal RMSECompensated RMSEOriginal relative L2 errorCompensated relative L2 error
8×80.40730.279079.94%54.77%
64×643.35430.2506353.01%26.38%
128×1286.75410.4241618.06%38.81%

Given the limited rebuttal period, we have not yet completed the corresponding large-scale Transformer experiments. In the subsequent revision, we will complete the model calibration and validation, rerun the Transformer training simulations using the calibrated hardware models, and incorporate the methodology and results into the manuscript. We hope this additional evaluation will further address your concern about the fidelity of the hardware simulations.

Best,

Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair,

Thank you for your time and continued support throughout the review process. To facilitate your evaluation of our manuscript and responses, we have summarized the key questions raised by the reviewers along with our corresponding replies below:

  • R1: Novelty beyond mixed-precision training (Reviewers D4o6 W1 and zV7w Q1): We clarified that our method contributes coordinated alignment of physical conductance, logical weight scales, and converter ranges. Theoretical bounds and component ablations demonstrate its contribution beyond the established mixed-precision partition and optimizer choice.
  • R2: Run-to-run variability and multi-seed evaluations (Reviewers zV7w Q3 and BsyX Q1): We completed comprehensive three-seed evaluations across all model scales, providing full error estimates for our validation losses. We are also pleased to note that Reviewer BsyX has acknowledged that their main concerns have been successfully addressed.
  • R3: Finite-array tiling evaluation (Reviewers D4o6 W3, zV7w W1 and BsyX Q2): We reported completed full-network training results across a variety of subarray tile sizes (1024×10241024 \times 1024 down to 128×128128 \times 128), demonstrating robust convergence under partitioning.
  • R4: SPICE validation and hardware modeling (Reviewers D4o6 Q1/W2 and BsyX Q3): We clarified our post-calibration converter abstraction, provided supportive SPICE validation results incorporating parasitics, noise, and mismatch, and updated our noise coefficients and Transformer training runs accordingly. We also clarified modeling limitations concerning calibration residuals, correlated noise, environmental and temperature effects, and long-term drift.

Our point-by-point responses address each concern raised by the reviewers. As the discussion period is drawing to a close, would you kindly consider encouraging the reviewers to engage in the discussion if they have any remaining questions? We want to ensure all potential points are fully addressed within the available window.

We deeply appreciate your time and consideration of our work.

Sincerely,

Authors

Area_Chair_TNE5OpenReview ↗

Please Engage in the Author–Reviewer Discussion

Dear Reviewers,

As the author–reviewer discussion period is approaching its deadline (November 18, AoE), I would like to kindly remind you to review the authors' rebuttals and actively engage in the discussion as soon as possible.

Please address any outstanding questions or concerns and update your assessments and ratings if appropriate.

Thank you very much for your time and efforts!

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.