acceptodds
Under review as a conference paper at ICLR 2027

Hardware-Software Co-Design of Energy-Efficient Analog Recurrent Computations

Abstract

Always-on AI applications, from environmental sensors to biomedical implants, require ultra-low power consumption. Analog circuits offer a path to sub-microwatt inference, yet existing analog implementations are limited to feedforward architectures: extending them to recurrent dynamics has been considered impractical due to noise accumulation through temporal feedback. We demonstrate that this barrier can be overcome through hardware-software co-design. Specifically, we identify that Bistable Memory Recurrent Units (BMRUs), a class of Recurrent Neural Networks (RNNs) with discrete-valued outputs and hysteretic dynamics, admit an ultra-low power current-mode analog implementation which we design from first principles. The resulting circuit establishes a one-to-one correspondence between each learned parameter and a circuit element. The discrete outputs suppress analog noise by at least 20-fold at each cell boundary, breaking the noise accumulation that prevents analog recurrence. We reformulate BMRUs for first-quadrant operation with fixed thresholds, enabling the direct correspondence while preserving expressivity and trainability. Transistor-level simulations in 180 nm Complementary Metal-Oxide-Semiconductor (CMOS) show near-perfect agreement between software predictions and circuit-level behavior, with the software model thereby serving as a high-fidelity simulator of the physical hardware at low computational cost. We leverage this fidelity to conduct large-scale noise immunity and power scaling analyses: the power cost of adding recurrence scales linearly with state dimension, while the feedforward layers dominating total power scale quadratically, meaning recurrence is added at linear marginal cost relative to the feedforward backbone. End-to-end keyword spotting achieves sub-microwatt inference at the RNN core. To our knowledge, this is the first demonstration of an analog recurrent neural network with structural noise immunity.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
analog neural network, neuromorphic computing, ultra-low-power edge AI, analog in-memory computing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_53EdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper co-designs a recurrent cell and an ultra-low-power analog circuit.

Critical strengths
  1. A genuine, verified algorithm–circuit correspondence. Every learned parameter has an explicit circuit counterpart: βhi→Ithresh\beta_{hi}\to I_{thresh}, βlo→Ithresh−Iwidth\beta_{lo}\to I_{thresh}-I_{width}, α→Igain\alpha\to I_{gain}, and weights → width ratios. The paper validates this end-to-end at transistor level, not just block by block: per-stage overlays of candidates, states, skip outputs and logits (App. J), 49/50 HW–SW prediction agreement, Monte Carlo, and PVT corners. This depth of validation is uncommon in ML-venue hardware papers. It makes the "software model as a cheap hardware simulator" idea credible, at least at the validated scale, which is why I lean positive.
  2. A clean architectural insight. The BMRU state is binary with hysteresis and needs no settling dynamics, so it can be realized without capacitors. Its thresholding also confines analog errors to near-threshold events, instead of letting them integrate through time. The candidate→state error reduction in App. J (≈60 pA → ≈3 pA) illustrates the mechanism concretely, and the idea plausibly transfers to other analog inference platforms.
Critical weaknesses
  1. Hardware evidence covers a very small configuration, and the key scaling claims are extrapolated.
    • Scale. All transistor-level evidence is at d=4d=4, two layers, binary KWS. Software accuracy there is 93.86%, below the 97–98% of the sub-µW digital 2-class KWS chips listed in Table 3.
    • Extrapolation. Larger-dd power (Table 6), the noise study on sMNIST/pMNIST/dKWS (Fig. 3), and multi-class KWS all extrapolate from d=4d=4 measurements. The quantities most likely to change with dd are exactly the ones being extrapolated. Current-summing nodes with fan-in dd accumulate mirror mismatch, so the "measured analog noise" from d=4d=4 blocks need not hold at d=64d=64. Fig. 3 shows only a ≈2× margin before a sharp failure.
    • Mismatch. At d=4d=4, the main-text example is misclassified in 11.5% of Monte Carlo instances. Monte Carlo covers only 11 clips, so the chip-to-chip accuracy distribution is unknown.
    • Power budget. "Sub-µW up to d=16d=16" is not itemized. The extrapolated core is ≈640 nW at d=16d=16 (Table 6). App. K says shift-register power (≈50 nW at d=4d=4) scales linearly with the number of parameters, which grows roughly with d2d^2. Several components are also excluded: input current generation for the MFCCs, majority-vote logic, and the front-end.
    • Noise-suppression claim. "Suppress analog noise by at least 20-fold at each cell boundary" (abstract) generalizes from a mean absolute error on a single example. For binary states, the relevant metric is the HW–SW state-flip rate. The overlays show that flips and timing shifts do occur:
      • Fig. 39, dim 4: a spurious HW pulse.
      • Fig. 47, dim 3: different transition timing.
      • Fig. 46, dim 3: the HW candidate is ≈0.7 where SW reaches ≈2.7. "Near-perfect agreement at every stage" is therefore overstated.
  2. Novelty and generality claims outrun what is shown.
    • Scope of recurrence. The FQ reformulation is a modest change to an existing cell. Recurrence is per-unit (diagonal) with no recurrent weight matrix: the candidate depends only on the current input. Hence "linear marginal cost of recurrence" holds by construction. The noise-accumulation barrier is avoided by restricting recurrence to binary self-hysteresis, rather than overcome for general analog recurrent dynamics.
    • "First fully analog RNN". This needs qualification. Analog and mixed-signal neuromorphic hardware with recurrent spiking connectivity, which also communicates through discrete events, has existed for years. The paper should state precisely which property is new.
    • Expressivity argument. Prop. B.4 is an informal sketch ("except on a set of measure ε"), not a proof. The ListOps gap (37.5 vs 41.7) suggests that fixed thresholds do cost trainability.
    • Table 1 backbone. Table 1 evaluates FQ BMRU inside a backbone with LayerNorm, GLU MLPs, sigmoid output gating and positional encoding, none of which the circuit can realize. It therefore does not characterize the deployable model.
Questions for authors
  1. Can you report the accuracy distribution across mismatch instances on the full test set? This could use Cadence, or the software proxy with the extracted mismatch statistics applied as static per-instance perturbations. Please do so at d=4d=4 and d=16d=16. This directly affects how much the robustness claim can be trusted.
  2. How was the "measured analog noise" in Fig. 3 modelled: static per-network mismatch, per-timestep noise, or both? How does its magnitude scale with FC fan-in? Is the ≈2× margin expected to hold at d=64d=64?
  3. Please give an itemized power budget at d=16d=16, covering core, mirror banks, shift registers, bias generation, input current generation and majority-vote logic. Is the total below 1 µW?
  4. Please report HW–SW binary state-flip rates per layer over the 50 clips, rather than only candidate/state MAE.
  5. Please state what is new relative to De Geeter et al. (2026) and Brandoit et al. (2026). In particular, did either already introduce unipolar outputs, fixed thresholds or a Schmitt-trigger realization? Please also report the hardware-backbone accuracies for the Table 1 tasks.
Reviewer_LUfhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a software approach for designing recurrent neural networks using bistable memory recurrent units, and shows that this software design can be verified using 180nm level transistor simulations. The solution Is deployed on end-to-end keyword splitting.

Critical strengths

Strengths:

  1. There is a clear mapping between learned parameters at the design time and the circuit elements being programmed.
  2. The approach matches relatively older baselines of LRU and minGRO on simple benchmarks such as sMNIST and pMNIST.
Critical weaknesses

Weaknesses:

  1. No silicon tests are reports and all of the results are simulations.
  2. AI helped formulate and write proofs. Perhaps, they could be verified in Lean and Lean proofs presented.
  3. Simulation results for large circuits is missing.
Questions for authors

When do your simulations and CMOS 180nm simulations start to diverge?

Reviewer_HAW3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper co-designs a hardware-compatible BMRU variant and current-mode CMOS circuits for energy-efficient analog recurrent inference, using bistable discrete states to suppress analog errors. Validation combines software benchmarks and robustness analyses with 180 nm transistor-level simulations of a small keyword-spotting network, reporting approximately 100 nW core power and agreement with software predictions on 49 of 50 samples; larger-scale hardware performance and full-system power remain unverified.

Critical strengths

The strongest contribution is the concrete correspondence between the reformulated recurrent model and its analog circuit implementation: learned thresholds and amplitudes map to bias currents, while weights map to current-mirror ratios. Transistor-level simulations support this correspondence through intermediate signal comparisons and agreement on 49 of 50 predictions at approximately 100 nW core power. This provides a credible proof of concept for ultra-low-power recurrent inference, making the contribution more substantial than an architectural proposal alone.

Critical weaknesses
  1. The expressivity guarantee is not established by the provided proof. Proposition B.4 allows gating disagreement on a set of small measure, whereas Theorem B.5 claims uniform approximation of the recurrent network. The former does not imply the latter, particularly because a single incorrect state update can persist through subsequent timesteps. This undermines the theoretical claim that the hardware-oriented reformulation preserves expressivity.

  2. The validation does not sufficiently support the claimed scalability and system-level efficiency. Transistor-level network validation is limited to a two-layer, four-state binary classifier; larger-network robustness relies on software noise injection, and power scaling is extrapolated from the small circuit. Moreover, feature extraction remains off-chip, and the implementation and power cost of temporal majority voting are not established. These gaps leave uncertainty about whether the reported advantages hold for a complete, larger-scale deployment.

Questions for authors
  1. Theoretical clarification: How does the small-measure gating disagreement in Proposition B.4 imply the uniform network approximation claimed in Theorem B.5? What assumptions on threshold margins, input distributions, and sequence length are required, given that an incorrect state update may persist?

  2. Validation clarification: What evidence establishes that the software noise model remains representative beyond the validated d=4d=4 circuit? Please specify how it distinguishes fixed device mismatch from time-varying noise and captures correlations across devices and timesteps.

  3. Power-accounting clarification: Where is the 101-timestep majority vote implemented, and is its power included in the reported figures? For the claimed sub-microwatt programmable d=16d=16 system, what is the complete power breakdown, including parameter storage, bias generation, and readout?

Comments on the paper

2 comments
AuthorsOpenReview ↗

Update: 2 × 16 network validated at transistor level

Dear Reviewers,

We did not mention the 2 × 16 network below in our first responses because its simulations were still running. We chose to answer on time with the results we had, and to share these new ones as soon as they were ready.

All three reviews noted that the network validated at transistor level is small (2 × 4). We are happy to share that we have now simulated a 2 × 16 network in Cadence (5168 transistors). It uses the same topology as the 2 × 4 circuit, with the "yes" KWS model at 98.08 % in software (Table 4). To keep simulation time reasonable, we reused the 50 samples of Section 3.2, and the 11 Monte Carlo samples of Appendix H with 100 instances each.

In short: the 2 × 16 network behaves like the software model, it is more robust to mismatch than the 2 × 4 network, and its power matches our estimates. We attribute the better robustness to larger margins: the 2 × 16 network is more accurate, so its outputs sit further from the decision boundary. The software virtual chips show the same trend, with a smaller accuracy loss under mismatch at d = 16 than at d = 4.

  • Agreement. Hardware and software predictions match on 50 of 50 samples (49 of 50 at 2 × 4).
  • Mismatch. Only 3 of the 1100 Monte Carlo runs change the prediction (0.27 %, against 9.0 % at 2 × 4). All 3 are on sample 67, which already flipped in 41 % of runs at 2 × 4. No other sample flips.
  • Signals. Binary states differ between hardware and software in 0.33 % of frames in layer 1 and 0.89 % in layer 2, similar to 2 × 4 (0.3 % and 0.6 %). Stage-by-stage agreement is as close as at 2 × 4. The only visible gap is the one explained in our response to Reviewer LUfh: when a cell is high, the probed candidate reads lower in hardware than in software because the feedback caps the input current, and the state is not affected.
  • Power. BMRU cells draw 150.3 ± 11.6 nW and FC layers 468.3 ± 48.4 nW, against 160 and 480 nW extrapolated in Table 6. The RNN core draws about 620 nW, close to the 616 nW of the random-weight simulation in our response to Reviewer LUfh.

Figures:

Proposed changes:

  • Keep the 2 × 4 network as the main example (about 100 nW), and present the 2 × 16 network as a more accurate and more robust option, at a higher power cost but still sub-µW at the RNN core.
  • Add an appendix for the 2 × 16 network with the same figures as for 2 × 4 (logits, SW/HW agreement, Monte Carlo, power split, state flips), for direct comparison.
  • Update Table 2 and the limitations to include the 2 × 16 transistor-level results.

If these changes match your expectations, we will quickly upload a revised PDF. We plan to upload the revised PDF no later than Monday, November 16.

Best regards, The Authors

AuthorsOpenReview ↗

Revised PDF uploaded: summary of changes

Dear Reviewers,

As announced, we just uploaded a revised version of the paper, as well as a new abstract. Following your questions and comments, we summarize the changes below. Numbers refer to the revised PDF. Since we added tables and figures, some numbers have shifted.

New results

  • 2 × 16 network at transistor level. New Appendix L, summarized in Section 3.2. It has the same figures as the 2 × 4 network. Hardware and software agree on 50 of 50 samples, only 3 of 1100 Monte Carlo runs change the prediction (0.27 %, against 9.0 % at 2 × 4), and the RNN core draws about 620 nW. Table 2 now includes this simulated row.
  • Expressivity proof. Prop. B.4 and Thm. B.5 are replaced by an explicit construction, with exact equality on any finite dataset, for any sequence length, and a bound without margin. A short paragraph in Section 2.3 states the result.
  • Lean 4 formalization. New Appendix B.4. The Lean files are in the repository.
  • Noise model. New Appendix I describes the software noise model and how fan-in is handled. Table 10 and Figures 37 to 41 compare mismatch, PVT and transient noise in Cadence. Section 4 summarizes it.
  • Software model vs. Cadence Monte Carlo. New Table 9 (Appendix H), for 2 × 4 and 2 × 16. The software model is never less pessimistic than Cadence.
  • Virtual chips. New Table 5 (Appendix C.1.6) gives the accuracy distribution over 200 virtual chips at d = 4, 16 and 64, also discussed in Section 4.
  • State flips. New Table 11 (Appendix K), summarized in Section 3.2. Appendix K also explains Figures 44, 51 and 52, including the probe reading on high cells.
  • Itemized power at d = 16. New Table 8 (Appendix E), with shift registers, bias generation and majority vote.
  • Hardware backbone on the Table 1 tasks. New row "FQ BMRU (HW)" in Table 1.

Revised claims and wording

  • Sub-µW claim restricted to the RNN core. "Clockless" applies to the analog core only.
  • "20-fold noise suppression" and "near-perfect agreement" are replaced by state flip rates and "close agreement, down to the binary state of each cell".
  • "First fully analog RNN" becomes "first capacitor-free analog implementation of a non-spiking, parallel-trainable recurrent cell in a complete network", with DYNAP-SE and BrainScaleS-2 cited.
  • Diagonal recurrence is stated clearly.
  • New paragraph on prior BMRU work (Appendix A.1).
  • Each result is labeled as transistor-level simulation, software model or extrapolation.
  • Limitations rewritten (no silicon yet, validated scales, diagonal recurrence, off-chip front-end and vote).

We hope that this version meets your expectations, and we remain available for any further question until the end of the discussion period.

As we read the ICLR guidelines, OpenReview automatically generates a comparison between the revised and the submitted PDF, but we cannot see it from the authors' side or on the public page. If it is not available to you and would be useful, we can send a PDF that marks every change on request.

The code has been updated in the same anonymous repository, including the Lean files: https://anonymous.4open.science/r/ICLR_2027_Anonymous-67C4

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.