acceptodds
Under review as a conference paper at ICLR 2027

Conditional Dynamical Systems for Image Generation

Abstract

Image generation is dominated by deep generative models on GPUs, whose computational and energy costs raise sustainability concerns. Emerging non-von Neumann substrates, including quantum, compute-in-memory, photonic, and thermodynamic platforms, promise greater efficiency, yet most work ports conventional neural network architectures onto them and mainly accelerates operations such as matrix multiplication. This does not fully exploit a native capability of many hardware substrates: relaxation toward low-energy states can itself compute at little cost. Building on this primitive, we develop dynamical systems for image generation based on Ising spins and Kuramoto oscillators. The internal states of both models evolve under dynamics with an explicit Lyapunov energy, and a compact decoder renders the final states as images. For conditional generation, we introduce local conditioning: programmed interactions remain fixed, and one class-dependent bias per node reshapes the landscape, avoiding additional reprogramming and calibration costs. On CIFAR-10, the Ising and Kuramoto models reach clean-FID scores of and , respectively, with nodes. With a CMOS-compatible hardware realization, the dynamical-system core is projected to cost nJ per image, and the full system, even with the decoder on a GPU, outperforms all GPU baselines in latency and energy. These results suggest that energy-descending dynamics on non-von Neumann substrates can serve as generative computation, offering an efficient path beyond GPUs. Our code is available at https://anonymous.4open.science/r/CF9E.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
image generation, non-von neumann computing, ising machine

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tVwcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a conditional image generator built around a continuous dynamical system with Ising- or Kuramoto-style latent variables. The latent dynamics are designed to decrease an energy function, and class conditioning is introduced locally through class-dependent pairwise couplings rather than by broadcasting a global conditioning vector. A convolutional decoder maps the final latent state to an image. The paper reports CIFAR-10 generation results, ablations over conditioning choices, robustness to quantization and noise, and projected energy/latency figures for a possible mixed-signal implementation.

At the model level, the main claim is that local conditioning in the latent dynamics contributes more than placing the condition only in the decoder or reservoir. At the hardware level, the paper argues that the dynamical-system structure could support substantially more efficient generation than a GPU implementation.

Critical strengths
  1. The co-design idea is coherent: conditioning is part of the physical interaction law rather than an additional digital pathway. The comparison against reservoir-only and decoder-only conditioning supports the claim that the latent dynamics are doing useful conditional computation.

  2. The paper does more robustness analysis than is common in this area. Quantization, device noise, coupling perturbations, and several architectural ablations are reported, so the model-level behavior is not supported by a single idealized experiment.

Critical weaknesses
  1. The headline hardware numbers combine evidence from different sources and levels of abstraction: a 64-spin 65 nm post-layout block, an extrapolation to 4096 spins, and peak efficiency figures from third-party compute-in-memory designs. I could not reconstruct a single end-to-end system corresponding to the reported 5.9 microjoules and 57.4 microseconds. Data conversion, memory movement, control, I/O, and decoder costs are not accounted for consistently, and the GPU baseline is not specified well enough to reproduce. This makes the strongest efficiency claim premature.

  2. The scaling argument does not yet match the dense model that is evaluated. A 4096-spin dense pairwise system contains roughly 8.39 million couplers, whereas the most favorable energy case assumes bounded degree. There is no area, routing, or parasitic analysis for the dense array. Similarly, the Lyapunov argument holds for the ideal symmetric continuous dynamics, but the robustness experiments do not show that monotone energy descent survives the saturation, asymmetry, discretization, and noise expected in the proposed implementation.

Questions for authors
  1. Can the authors provide one auditable end-to-end cost table for the proposed generator, separating oscillator/dynamics, coupling storage and access, converters, control, decoder, and I/O? Please use the same inclusion rules for the hardware and GPU baselines.

  2. What physical topology realizes the dense 4096-spin interaction matrix, and what are the resulting area, wire, and bandwidth costs? If the intended topology is sparse, what generation result is obtained with that exact sparsity?

  3. Does the implemented noisy, quantized, and asymmetric system still exhibit monotone energy descent? A trajectory-level measurement would directly connect the hardware study to the theoretical argument.

4 comments
AuthorsOpenReview ↗

Response to Reviewer tVwc (Part 1/2)

Thank you for the detailed review. Your questions helped us make the hardware analysis more complete and auditable. We address each point below and will include the new results in the revised paper.

W1 / Q1. Energy and latency breakdown, efficiency gains, and GPU evaluation.

Energy and latency. We added an end-to-end breakdown with the same inclusion rules for our design and the GPU baseline. Tables A and B give coarse-grained energy and latency breakdowns, and Table C (at the end of the comment) gives a fine-grained breakdown by operation. Some highlights:

  • The core design principle of our dynamical system hardware is no ADC during computation. The final state is digitized once at the end. Other accelerators usually convert to and from digital memory at every step. Our state stays in the circuit dynamics, so this conversion and memory traffic disappear.
  • The overheads are supported by silicon measurements from a fabricated linear Ising machine chip (shown in the revised paper).
  • The ADC-less design also keeps the overheads minimal. Together, the overheads account for only ~5% of the energy and <1% of the latency, so our efficiency gain stays nearly the same.
  • The prototype we are finalizing for tape-out has sparser, banded connectivity (see W2), so the numbers in the tables below are a reasonable upper bound.
  • As independent silicon evidence, a fabricated chip running the baseline model is reported to use ~500-900 nJ per image. The proposed oscillator model has a simpler form and performs better (Table 1). The proposed Ising model is based on the linear Ising formulation, and recent surveys credit both linear and oscillator-based Ising machines with favorable power efficiency [1,2].
(A) Energy breakdownEnergy (nJ), Case A / Case BShare (%), Case A / Case B
Storage87.4 / 87.51.42
Bias programming9.60.16
Initialization22.80.37
DS core81.3 / 110.61.32 / 1.79
DS interconnect1.40.02
Readout143.42.34 / 2.32
DS interface43.70.71
Decoder5,74093.51 / 93.03
Control and communication9.0 / 10.70.15 / 0.17
Total6,138.6 / 6,169.8100.00
(B) Latency breakdown, sequential stagesTime (ns), Case A / Case BShare (%), Case A / Case B
Bias programming100.02
Initialization100.02
DS core10.1 / 80.80.02 / 0.14
Readout250.04
DS interface3200.55
Decoder57,30099.35 / 99.23
Total57,675 / 57,746100.00

Hardware evaluation. We evaluate the complete DS implementation in Cadence Virtuoso AMS. The digital control logic is synthesized with Synopsys Design Compiler and placed and routed with IC Compiler. The resulting gate-level netlist is integrated into the mixed-signal simulation. Analog and digital parasitics are extracted using Mentor Calibre and Synopsys IC Compiler. SRAM access and retention power are estimated using the TSMC memory compiler. The mixed-signal circuit power is obtained from AMS simulation results.

GPU measurement. The timing and energy of all GPU baselines and decoders are measured on one NVIDIA L40S (350 W) with PyTorch 2.11, CUDA 12.8, FP32 weights, and TF32 tensor cores. We time each model from noise or labels to the output image, after warm-up. For our models (the -GPU family), only the decoder runs on the same L40S. Energy comes from the NVML board counter, with idle power included.

AuthorsOpenReview ↗

Response to Reviewer tVwc (Part 2/2)

W2-a / Q2. Scaling to larger systems. Physical topology of the interaction. Area, routing, and parasitic analysis.

Scaling to larger systems. Our goal is not to replace GPU-native generators by scaling the dense design, but to show that hardware can generate images with its own physics. To scale further, we are finalizing a banded design for tape-out, in which each spin connects only to a fixed number of neighbors. With band width K = 32, each spin couples to its 2K = 64 nearest neighbors, so the design needs ~64x fewer couplers than the dense one (128x with K = 16). The coupler count and power then grow linearly with n instead of quadratically. The results below show a modest FID cost against the dense design, and both banded models still can have better performance than the baseline.

CouplingDS couplersClean-FID (3 seeds)Mean ± std
Banded, K = 320.131M7.81 / 6.71 / 7.247.25 ± 0.55
Banded, K = 160.065M7.28 / 9.42 / 7.588.09 ± 1.16

Physical topology of the interaction. Area, routing, and parasitic analysis. The dense interaction matrix is implemented using a 4 * 4 tiled architecture, with each tile containing 1024 * 1024 coupling elements, corresponding to a total coupling area of 67.09 mm². Unity-gain voltage followers distribute spin voltages between tiles, while grounded metal shielding minimizes crosstalk. Calibre PEX extraction shows 0.1 fF inter-wire coupling and 9.2 fF ground parasitic capacitance per 500 µm. The four-stage voltage follower chain is designed for a 650-MHz end-to-end bandwidth, with all interactions operating concurrently in the analog domain. To reduce area and routing overhead, we are finalizing a banded sparse architecture for tape-out.

W2-b / Q3. Energy descent in non-ideal systems.

Strict monotone descent holds only for the ideal system. In non-ideal systems such as the hardware, the energy fluctuates but still descends, so energy minimization remains the core mechanism. Noise can even help: Ising machines for optimization inject noise on purpose to escape local minima. We show the energy trajectories of image generation from the same initial state in the revised paper (App. F): the ideal system descends smoothly, and the non-ideal system follows the same trend with small fluctuations. Saturation removes terms from the energy, so the energy stays non-increasing (Sec. 2.2). Models trained with asymmetric couplings keep their quality (clean-FID 5.96 and 5.39, App. B).

References

[1] H. Lee, J. P. Kim, and S. Kim. Fundamental challenges, physical implementations, and integration strategies for Ising machines in large-scale optimization tasks. Advanced Electronic Materials, 12(17):e00682, 2026.

[2] T. Zhang, Q. Tao, B. Liu, A. Grimaldi, E. Raimondo, M. Jiménez, M. J. Avedillo, J. Nuñez, B. Linares-Barranco, T. Serrano-Gotarredona, G. Finocchio, and J. Han. A review of Ising machines implemented in conventional and emerging technologies. IEEE Transactions on Nanotechnology, 23:704–717, 2024.

Detailed breakdown

(C) Group (time)OperationPower (mW), Case A / Case BEnergy (nJ), Case A / Case B
StorageCoupling coefficients, 10 MiB1.1566.4 / 66.5
Conditioning bias LUT, 25 KiB0.01731.0
DS results, 4 KiB0.002780.2
Seed retention0.34419.8 / 19.9
Bias programming (10 ns)Local class lookup and code update9579.6
Initialization (10 ns)Shared random-bit source0.3200.0032
4096-bit register shift6.310.0631
State loading and wire charging2,28022.8
DS core (10.1 / 80.8 ns)Spin evolution1,23012.4 / 99.3
Coupling-network evolution6,790 / 10668.5 / 8.6
Conditioning bias34.40.3 / 2.8
DS interconnect (during DS core)Layout parasitic effect136 / 17.01.4
Readout (25 ns)Sample / hold and voltage ADC4,510112.6
Readout source / buffer1,23030.7
DS interface (320 ns)Terminal SRAM access71.022.7
Terminal data wires65.621.0
Decoder (57,300 ns)Decoder computation—5,740
Control and communicationDS controller (during DS core)24.10.2 / 1.9
Initialization / readout sequencer (365 ns)0.3760.1
Seed-register clock distribution95.61.0
Seed-register interstage wires4.100.041
Class-select / reset / capture broadcast (40 ns)1917.7
Total (57,675 / 57,746 ns)—6,138.6 / 6,169.8

Thank you for the detailed response. It addresses most of my concerns, and I will raise my score from 2 to 3.

Thank you for your positive feedback, which have helped us further strengthen the paper. We sincerely appreciate your time and attention.

Reviewer_aAYAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes conditional image generation using continuous Ising and Kuramoto dynamical systems with explicit Lyapunov energies. Its local conditioning mechanism introduces class-specific node biases while keeping pairwise couplings fixed, avoiding hardware reprogramming. A convolutional decoder converts the evolved states into images. On CIFAR-10, the proposed models achieve clean-FID scores of 6.40 and 5.37 with 4096 nodes, alongside projected hardware energy and latency of 5.9 μJ and 57.4 μs per image.

Critical strengths

The proposed local conditioning mechanism enables class switching without modifying pairwise couplings while preserving the Lyapunov energy property. This is particularly valuable for physical substrates where reprogramming and recalibration are expensive.

Critical weaknesses
  1. The reported 5.9 μJ energy and 57.4 μs latency are projections combining a scaled dynamical core with efficiency and throughput estimates from different CIM chips. Full transistor-level validation covers only 64 spins, while the 1024-spin simulation uses partially idealized circuit blocks. Moreover, initialization, data conversion, communication, and other system overheads are not fully quantified, weakening the claimed advantages over GPU implementations.
  2. The improvements over Un-0 are reported without retraining the baseline under the same Sinkhorn-based objective and training budget, making it unclear how much improvement comes from the proposed architecture.
Questions for authors
  1. Since the headline hardware results are projections rather than measurements from a complete 4096-node implementation, can you provide a detailed end-to-end energy and latency breakdown, including initialization, readout, data conversion, and communication overheads?
  2. Given that the proposed methods use a different training objective from Un-0, can you compare them under identical training objectives, computational budgets, and evaluation protocols to isolate the contribution of the dynamical architecture?
2 comments
AuthorsOpenReview ↗

Response to Reviewer aAYA (Part 1/2)

Thank you for the careful review and constructive questions. Your questions helped us clarify both our hardware and model contributions. We address each point below and will include the new breakdown and ablation in the revised paper.

W1 / Q1. Energy and latency breakdown, efficiency gains, and circuit validation at scale.

Energy and latency. We added an end-to-end breakdown with the same inclusion rules for our design and the GPU baseline. Tables A and B give coarse-grained energy and latency breakdowns, and Table C (at the end of the comment) gives a fine-grained breakdown by operation. Some highlights:

  • The core design principle of our dynamical system hardware is no ADC during computation. The final state is digitized once at the end. Other accelerators usually convert to and from digital memory at every step. Our state stays in the circuit dynamics, so this conversion and memory traffic disappear.
  • The overheads are supported by silicon measurements from a fabricated linear Ising machine chip (shown in the revised paper).
  • The ADC-less design also keeps the overheads minimal. Together, the overheads account for only ~5% of the energy and <1% of the latency, so our efficiency gain stays nearly the same.
  • The prototype we are finalizing for tape-out has sparser, banded connectivity, so the numbers in the tables below are a reasonable upper bound.
  • As independent silicon evidence, a fabricated chip running the baseline model is reported to use ~500-900 nJ per image. The proposed oscillator model has a simpler form and performs better (Table 1). The proposed Ising model is based on the linear Ising formulation, and recent surveys credit both linear and oscillator-based Ising machines with favorable power efficiency [1,2].
(A) Energy breakdownEnergy (nJ), Case A / Case BShare (%), Case A / Case B
Storage87.4 / 87.51.42
Bias programming9.60.16
Initialization22.80.37
DS core81.3 / 110.61.32 / 1.79
DS interconnect1.40.02
Readout143.42.34 / 2.32
DS interface43.70.71
Decoder5,74093.51 / 93.03
Control and communication9.0 / 10.70.15 / 0.17
Total6,138.6 / 6,169.8100.00
(B) Latency breakdown, sequential stagesTime (ns), Case A / Case BShare (%), Case A / Case B
Bias programming100.02
Initialization100.02
DS core10.1 / 80.80.02 / 0.14
Readout250.04
DS interface3200.55
Decoder57,30099.35 / 99.23
Total57,675 / 57,746100.00

Hardware evaluation. We evaluate the complete DS implementation in Cadence Virtuoso AMS. The digital control logic is synthesized with Synopsys Design Compiler and placed and routed with IC Compiler. The resulting gate-level netlist is integrated into the mixed-signal simulation. Analog and digital parasitics are extracted using Mentor Calibre and Synopsys IC Compiler. SRAM access and retention power are estimated using the TSMC memory compiler. The mixed-signal circuit power is obtained from AMS simulation results.

Circuit validation at scale. The 1024-spin simulation breaks the circuit into functional blocks, each modeled by its transfer function. We added an experiment that benchmarks these block models against the 64-spin post-layout transistor-level circuit under identical weights, biases, and initial voltages, without refitting. Across 10 different initialization cases, the average voltage discrepancy over all 64 spins at readout was 0.097%. This agreement supports using the same models at 1024 spins. The simulated evolution times for the 1024-spin array also fall within the range predicted in Table 5.

W2 / Q2. Extended ablation study.

We extended the ablation to cover both model and loss (clean-FID, lower is better):

ModelParamsBaseline lossOur loss
Un-0.n409619.43M8.866.69 ± 0.10
Ising-4096 (ours)9.01M9.99 ± 0.506.53 ± 0.15
Kuramoto-4096 (ours)10.75M7.20 ± 0.135.41 ± 0.10

All runs use the same compute budget and evaluation protocol as the baseline. Our loss improves every model, including Un-0. Under our loss, both of our models beat Un-0 with about half the parameters. Under the baseline loss, our Kuramoto model shows better generation quality.

AuthorsOpenReview ↗

Response to Reviewer aAYA (Part 2/2)

References

[1] H. Lee, J. P. Kim, and S. Kim. Fundamental challenges, physical implementations, and integration strategies for Ising machines in large-scale optimization tasks. Advanced Electronic Materials, 12(17):e00682, 2026.

[2] T. Zhang, Q. Tao, B. Liu, A. Grimaldi, E. Raimondo, M. Jiménez, M. J. Avedillo, J. Nuñez, B. Linares-Barranco, T. Serrano-Gotarredona, G. Finocchio, and J. Han. A review of Ising machines implemented in conventional and emerging technologies. IEEE Transactions on Nanotechnology, 23:704–717, 2024.

Detailed breakdown

(C) Group (time)OperationPower (mW), Case A / Case BEnergy (nJ), Case A / Case B
StorageCoupling coefficients, 10 MiB1.1566.4 / 66.5
Conditioning bias LUT, 25 KiB0.01731.0
DS results, 4 KiB0.002780.2
Seed retention0.34419.8 / 19.9
Bias programming (10 ns)Local class lookup and code update9579.6
Initialization (10 ns)Shared random-bit source0.3200.0032
4096-bit register shift6.310.0631
State loading and wire charging2,28022.8
DS core (10.1 / 80.8 ns)Spin evolution1,23012.4 / 99.3
Coupling-network evolution6,790 / 10668.5 / 8.6
Conditioning bias34.40.3 / 2.8
DS interconnect (during DS core)Layout parasitic effect136 / 17.01.4
Readout (25 ns)Sample / hold and voltage ADC4,510112.6
Readout source / buffer1,23030.7
DS interface (320 ns)Terminal SRAM access71.022.7
Terminal data wires65.621.0
Decoder (57,300 ns)Decoder computation—5,740
Control and communicationDS controller (during DS core)24.10.2 / 1.9
Initialization / readout sequencer (365 ns)0.3760.1
Seed-register clock distribution95.61.0
Seed-register interstage wires4.100.041
Class-select / reset / capture broadcast (40 ns)1917.7
Total (57,675 / 57,746 ns)—6,138.6 / 6,169.8
Reviewer_e6VwOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes leveraging the spontaneous relaxation of internal circuit states in non-von Neumann hardware substrates for lower-latency and lower-energy image generation, and develops compatible model architectures and training methods. Model-level experiments and circuit simulations support the effectiveness of the proposed approach and indicate reduced projected generation latency and energy in the small-scale CIFAR-10 setting.

Critical strengths

The paper introduces a novel hardware-native generative modeling paradigm that uses the intrinsic relaxation dynamics of non-von Neumann substrates, offering a conceptually distinct path toward lower-energy and lower-latency generation.

The paper develops this idea into a trainable framework and supports it with competitive CIFAR-10 results and circuit-level simulations, providing meaningful initial evidence for the feasibility of the proposed algorithm-hardware co-design.

Critical weaknesses

The claimed hardware-efficiency gains rely primarily on circuit simulations and performance projections rather than end-to-end measurements on physical hardware.

Modern image generative models operate at much larger parameter scales, while scaling the current fully connected architecture toward comparable parameter counts may require a very large number of nodes and potentially prohibitive power.

Questions for authors

Are the FID and IS results in Table 1 obtained by simulating the Ising/Kuramoto dynamics with a numerical ODE solver on a GPU? If so, could the authors report the solver, step size, and number of steps?

Does the energy reported in Figure 4 include the costs of programming the per-node biases, initializing the relaxation state, sample-and-hold and voltage/phase readout, control, transferring the terminal state to the decoder hardware, and keeping that hardware powered while it waits for the terminal state? If not, could the authors provide measurements or conservative upper-bound estimates for these costs, thereby providing a fairer comparison with GPU-only sampling? Besides, could the authors explain how the per-image GPU sampling energy was obtained?

Could the authors analyze the expected readout-time and readout-circuit errors in a physical implementation, and evaluate whether generation quality is sensitive to these errors?

Can the current model architecture be extended beyond class-conditional generation to richer conditioning modalities such as text-to-image generation? If so, how would text conditioning be incorporated?

2 comments
AuthorsOpenReview ↗

Response to Reviewer e6Vw (Part 1/2)

Thank you for the positive feedback and your constructive suggestions. We address each point below and will include the new results in the revised paper.

W1 / Q2. Energy and latency breakdown, efficiency gains, and GPU evaluation.

Energy and latency. We added an end-to-end breakdown with the same inclusion rules for our design and the GPU baseline. Tables A and B give coarse-grained energy and latency breakdowns, and Table C (at the end of the comment) gives a fine-grained breakdown by operation. Some highlights:

  • The core design principle of our dynamical system hardware is no ADC during computation. The final state is digitized once at the end. Other accelerators usually convert to and from digital memory at every step. Our state stays in the circuit dynamics, so this conversion and memory traffic disappear.
  • The overheads are supported by silicon measurements from a fabricated linear Ising machine chip (shown in the revised paper).
  • The ADC-less design also keeps the overheads minimal. Together, the overheads account for only ~5% of the energy and <1% of the latency, so our efficiency gain stays nearly the same.
  • The prototype we are finalizing for tape-out has sparser, banded connectivity (see W2), so the numbers in the tables below are a reasonable upper bound.
  • As independent silicon evidence, a fabricated chip running the baseline model is reported to use ~500-900 nJ per image. The proposed oscillator model has a simpler form and performs better (Table 1). The proposed Ising model is based on the linear Ising formulation, and recent surveys credit both linear and oscillator-based Ising machines with favorable power efficiency [1,2].
(A) Energy breakdownEnergy (nJ), Case A / Case BShare (%), Case A / Case B
Storage87.4 / 87.51.42
Bias programming9.60.16
Initialization22.80.37
DS core81.3 / 110.61.32 / 1.79
DS interconnect1.40.02
Readout143.42.34 / 2.32
DS interface43.70.71
Decoder5,74093.51 / 93.03
Control and communication9.0 / 10.70.15 / 0.17
Total6,138.6 / 6,169.8100.00
(B) Latency breakdown, sequential stagesTime (ns), Case A / Case BShare (%), Case A / Case B
Bias programming100.02
Initialization100.02
DS core10.1 / 80.80.02 / 0.14
Readout250.04
DS interface3200.55
Decoder57,30099.35 / 99.23
Total57,675 / 57,746100.00

Hardware evaluation. We evaluate the complete DS implementation in Cadence Virtuoso AMS. The digital control logic is synthesized with Synopsys Design Compiler and placed and routed with IC Compiler. The resulting gate-level netlist is integrated into the mixed-signal simulation. Analog and digital parasitics are extracted using Mentor Calibre and Synopsys IC Compiler. SRAM access and retention power are estimated using the TSMC memory compiler. The mixed-signal circuit power is obtained from AMS simulation results.

GPU measurement. The timing and energy of all GPU baselines and decoders are measured on one NVIDIA L40S (350 W) with PyTorch 2.11, CUDA 12.8, FP32 weights, and TF32 tensor cores. We time each model from noise or labels to the output image, after warm-up. For our models (the -GPU family), only the decoder runs on the same L40S. Energy comes from the NVML board counter, with idle power included.

W2. Scaling to larger systems.

Our goal is not to replace GPU-native generators by scaling the dense design, but to show that hardware can generate images with its own physics. To scale further, we are finalizing a banded design for tape-out, in which each spin connects only to a fixed number of neighbors. With band width K = 32, each spin couples to its 2K = 64 nearest neighbors, so the design needs ~64x fewer couplers than the dense one (128x with K = 16). The coupler count and power then grow linearly with n instead of quadratically. The results below show a modest FID cost against the dense design, and both banded models still can have better performance than the baseline.

CouplingDS couplersClean-FID (3 seeds)Mean ± std
Banded, K = 320.131M7.81 / 6.71 / 7.247.25 ± 0.55
Banded, K = 160.065M7.28 / 9.42 / 7.588.09 ± 1.16

Q1. Solver details.

The results in Table 1 are simulated on a GPU with an Euler solver, step size 0.1, and 10 steps. Finer Euler steps and RK4 give results in the same range, so we use the cheapest setting for training and evaluation. We added these settings and the implementation details to the code repository.

AuthorsOpenReview ↗

Response to Reviewer e6Vw (Part 2/2)

Q3. Readout errors.

We added an evaluation under readout error. Generation quality is not sensitive to it. We tested readout clock jitter/skew of ±5% and readout voltage error of ±5%, both well above circuit-level errors: simulations show <0.5 ps jitter/skew for a 0.25 ns clock period (timing error <0.2%), and an 8-bit ADC can achieve a total unadjusted error of ±1 LSB (±0.39% of full scale). The FID stays between 6.476 and 6.491, against 6.33 with ideal readout.

Q4. Multimodal generation and scaling.

Yes, the design can be extended to multimodal generation, and this is our next step. For this, we are integrating a loop-based, linear, sparse attention mechanism that follows dynamics similar to Eq. 4. Loops are cheap in our analog design because they need no ADC. The core idea of our work is to remove such overheads and to accept and mitigate analog noise in exchange for better efficiency at comparable performance.

References

[1] H. Lee, J. P. Kim, and S. Kim. Fundamental challenges, physical implementations, and integration strategies for Ising machines in large-scale optimization tasks. Advanced Electronic Materials, 12(17):e00682, 2026.

[2] T. Zhang, Q. Tao, B. Liu, A. Grimaldi, E. Raimondo, M. Jiménez, M. J. Avedillo, J. Nuñez, B. Linares-Barranco, T. Serrano-Gotarredona, G. Finocchio, and J. Han. A review of Ising machines implemented in conventional and emerging technologies. IEEE Transactions on Nanotechnology, 23:704–717, 2024.

Detailed breakdown

(C) Group (time)OperationPower (mW), Case A / Case BEnergy (nJ), Case A / Case B
StorageCoupling coefficients, 10 MiB1.1566.4 / 66.5
Conditioning bias LUT, 25 KiB0.01731.0
DS results, 4 KiB0.002780.2
Seed retention0.34419.8 / 19.9
Bias programming (10 ns)Local class lookup and code update9579.6
Initialization (10 ns)Shared random-bit source0.3200.0032
4096-bit register shift6.310.0631
State loading and wire charging2,28022.8
DS core (10.1 / 80.8 ns)Spin evolution1,23012.4 / 99.3
Coupling-network evolution6,790 / 10668.5 / 8.6
Conditioning bias34.40.3 / 2.8
DS interconnect (during DS core)Layout parasitic effect136 / 17.01.4
Readout (25 ns)Sample / hold and voltage ADC4,510112.6
Readout source / buffer1,23030.7
DS interface (320 ns)Terminal SRAM access71.022.7
Terminal data wires65.621.0
Decoder (57,300 ns)Decoder computation—5,740
Control and communicationDS controller (during DS core)24.10.2 / 1.9
Initialization / readout sequencer (365 ns)0.3760.1
Seed-register clock distribution95.61.0
Seed-register interstage wires4.100.041
Class-select / reset / capture broadcast (40 ns)1917.7
Total (57,675 / 57,746 ns)—6,138.6 / 6,169.8

Comments on the paper

1 comment

We thank the PC, SAC, AC and all reviewers for their time. Our main contributions are as follows:

  • Generative models on emerging non-von Neumann hardware. Ising machines, which have so far mainly targeted combinatorial optimization, can now generate images using their native physics, with minimal hardware overhead and far lower energy and latency than GPUs.
  • Broad applicability. The approach extends to other emerging substrates, including quantum, compute-in-memory, photonic, and thermodynamic platforms, and opens a clear path toward compute efficiency beyond GPUs. Our model code is open source to support adoption on these platforms.
  • Pareto frontier. The local conditioning has a clear energy-based explanation and removes the overhead of reprogramming the hardware. Together with the new objective, it improves generation quality and pushes the FID-energy and FID-latency Pareto frontiers.
  • A hardware case study. It covers the circuit design, robustness to hardware non-idealities, and detailed circuit simulations with parameters from fabricated hardware.

Our work leverages new semiconductor technologies, computes with physics, reduces the data movement and opens a path toward computing beyond GPUs. All reviewers are interested in a detailed energy and latency breakdown. We added the breakdown and it further confirms our efficiency claims. Here are the main points:

  • No ADC during computation. The state remains in the circuit dynamics and is digitized only once at the end. This is the main source of our efficiency gain and keeps all overheads to ~5% of the energy and <1% of the latency.
  • Silicon support. Measurements from a fabricated linear Ising machine chip support the overhead estimates, as shown in the revised paper.
  • Independent evidence. A fabricated chip running the baseline model is reported to use ~500–900 nJ per image. Our models offer a simpler design, lower algorithmic overhead, and better generation quality.

We also added:

  • An extended ablation that separates the training objective from the architecture (addressed Reviewer aAYA's concern). It strengthens our model-level contributions.
  • A banded-coupling design, now being finalized for tape-out. It uses 64-128x fewer couplers with little loss in image quality and shows a clear path to scaling.
  • Energy trajectories under noise and quantization. Energy descent remains the core mechanism of generation, and the model remains robust across these settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.