acceptodds
Under review as a conference paper at ICLR 2027

DS-Flow: Dual-Stream Dynamical System Correction for Few-Step Diffusion Sampling

Abstract

Diffusion models have shown remarkable success in generative modeling, but maintaining high quality at extremely low numbers of function evaluations (NFE) remains challenging due to large truncation errors under coarse ODE discretization. Existing methods either rely on costly distillation or perform step-level correction that adjusts the sampling direction only once per interval, which limits fine-grained trajectory refinement. In this work, we propose DS-Flow, which embeds an electronic dynamical-system (DS) corrector within each sampling step, enabling continuous in-step trajectory refinement through multiple hardware-efficient micro-steps without additional denoiser calls. While a PCA subspace is the natural choice for keeping each DS update lightweight, our analysis reveals that online PCA captures only 70% of truncation error; the residual exhibits spatial structure that a low-dimensional subspace cannot adequately represent. This motivates a dual-stream design: a global DS with dense coupling corrects the dominant low-dimensional error, while a local DS with grid coupling captures spatial residuals directly in pixel space. Extensive experiments demonstrate that DS-Flow consistently outperforms existing methods. For example, on CIFAR-10, DS-Flow reduces the Euler baseline's FID from 49.66 to 7.95 at NFE5 while the corrector adds only 0.1% parameters. On DS hardware, the corrector runs at mW power and s latency, achieving energy reduction and 6–7 speedup per correction interval over GPU-based alternatives.

Primary area:
generative models
Keywords:
Diffusion Model, Efficient Inference, Dynamical System, Software Hardware Co-Design, Image Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_BVQkOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DS-Flow, a learned dual-stream corrector for few-step diffusion sampling. Instead of repeatedly calling the denoiser within each coarse solver interval, DS-Flow reuses the current denoising direction and performs multiple lightweight micro-step corrections. A global branch operates in an online PCA subspace, while a local branch corrects spatial residuals in pixel space. The corrector is trained from dense teacher trajectories(e.g. 50step) and is designed to map naturally onto analog dynamical-system hardware, where the repeated micro-corrections are intended to be cheap. Experiments on CIFAR-10, ImageNet-64, and LSUN Bedroom at 5–10 NFE show improved FID over several solver and learned-correction baselines. The hardware latency and energy results are obtained from circuit-level simulation rather than a fabricated DS chip.

Critical strengths
  1. Novel algorithm-hardware co-design.
    The idea of combining learned diffusion correction with dedicated dynamical-system hardware is interesting and relatively novel. Most learned solvers focus only on the algorithmic side, while this work explicitly designs the corrector with efficient hardware execution in mind.

  2. Relatively strong few-step performance.
    DS-Flow achieves strong results in the 5–10 NFE regime, outperforming several higher-order solvers and learned-correction methods such as AMED across multiple benchmarks. This supports the effectiveness of the proposed in-step correction strategy.

Critical weaknesses
  1. The hardware co-design is not validated on actual hardware.
    Hardware efficiency is a central motivation of the method, but all generation results are obtained in software and the DS latency/energy numbers come from circuit-level simulation rather than a fabricated end-to-end GPU–DS system. Since the claimed advantage relies heavily on making the 50 micro-corrections cheap, real hardware validation is important to support the practical claim.

  2. The comparison with recent learned few-step solvers is incomplete.
    The paper mainly compares with methods such as AMED, PAS, and conventional ODE solvers, but misses more recent learned few-step approaches such as LD3 [1] and DyWeight [2]. These methods target the same low-NFE regime and also include a learning phase, so including them would be important for establishing the competitiveness of DS-Flow against more current baselines.

[1] Learning to Discretize Denoising Diffusion ODEs

[2] DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling

Questions for authors
  1. Could the authors provide a more concrete validation of the proposed hardware co-design, ideally with an actual hardware prototype or hardware-in-the-loop implementation? Since the efficiency of the repeated micro-corrections is central to the motivation of DS-Flow, real end-to-end latency and energy measurements would substantially strengthen the paper beyond circuit-level simulation.

  2. Could the authors compare DS-Flow with more recent learning-based few-step methods, such as LD3 [1] and DyWeight [2]? These methods target a similar low-NFE setting.

  3. Since each denoiser evaluation is followed by up to 50 learned micro-corrections, could the authors report the actual end-to-end wall-clock cost of DS-Flow in the software/GPU setting? NFE alone does not capture this additional computation, so a matched-quality latency comparison would make the efficiency claim clearer.

  4. Does DS-Flow require a separately trained corrector for each NFE setting? The paper appears to generate teacher trajectories for each dataset and NFE configuration. It would be useful to know whether a single corrector can generalize across different sampling budgets.

[1] Learning to Discretize Denoising Diffusion ODEs

[2] DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling

5 comments
AuthorsOpenReview ↗

Response to Reviewer BVQk (1/3)

We sincerely thank Reviewer BVQk for the thorough and constructive review, and for recognizing the algorithm-hardware co-design as novel and the 5 to 10 NFE results as strong. We answer each point below.

W1. Validation on real hardware. Thank you for raising this important point. We agree that a fabricated DS-Flow chip does not exist yet; Section 4.1 and the Reproducibility Statement say so. The reported speedup and energy are nonetheless grounded in three ways:

  • A detailed circuit-level estimate, not an idealized model. The DS circuit is simulated in Cadence mixed-signal simulation in 45 nm at the typical-typical corner and 25 °C, with 8-bit converters, device mismatch and thermal noise. The 1.4 µs per interval covers data conversion as well as computation: one DAC write (0.25 µs), on-array annealing of all 50 micro-steps (1.0 µs) and one ADC read (0.15 µs), since conversion happens once per interval rather than per micro-step (Appendix G). The analog effects are also counted in quality: 8-bit conversion costs about 0.2 FID relative to fp32 (Table 5), and thermal noise up to 10% adds about 0.2 FID (Table 6).
  • The standard practice of the hardware community. New analog architectures are evaluated in simulation before tape-out: recent ReRAM in-memory accelerators such as RAELLA (ISCA 2023) and ASADI (HPCA 2024) were proposed and evaluated this way, without a fabricated prototype, and modeling tools such as CiMLoop (ISPASS 2024) exist for this purpose. Prior DS hardware, NP-GL and EADS, was evaluated with the same pre-silicon protocol that we follow.
  • Silicon precedent. Our timing and power constants are taken from the BRIM design and the simulated NP-GL and EADS designs. Ising hardware has been built in CMOS, both as digital annealers, e.g., Yamaoka et al. (JSSC 2016) and STATICA (ISSCC 2020), and as physics-based coupled-oscillator chips, e.g., Moy et al. (Nat. Electron. 2022, 1,968 nodes) and Lo et al. (Nat. Electron. 2023, 48 all-to-all nodes). What remains unbuilt is the DS-Flow chip itself, which we plan to tape out next year.

The algorithm itself does not depend on the hardware: run on a GPU, DS-Flow remains competitive end to end (Q3).

AuthorsOpenReview ↗

Response to Reviewer BVQk (2/3)

W2. Recent learned few-step solvers. Thank you for pointing us to LD3 and DyWeight, which we had not compared against; we are glad to add both. Their published numbers use settings that differ from ours, so we report each comparison together with its setting. All numbers use the same EDM checkpoints and 50k-sample FID.

LD3 (CIFAR-10 / ImageNet-64, as listed in DyWeight's comparison table; DS-Flow from Table 1):

MethodBase solverNFE 5NFE 6NFE 8
LD3best of three third-order solvers at each NFE6.39 / 11.553.35 / 7.532.81 / 5.40
DS-Flowone first-order Euler solver4.55 / 10.673.23 / 7.602.57 / 4.33

LD3 learns the time discretization of a multi-step solver. As listed in DyWeight's table, its value at each NFE is the best over its base solvers; on CIFAR-10 these are DPM-Solver++(3M), iPNDM(3M) and UniPC(3M) (LD3, Table 18). Even with this choice, DS-Flow on a single first-order base is ahead in five of six settings.

DyWeight (CIFAR-10, NFE 5, FID / FD_DINOv2, measured by us with 50k samples):

Matched settingDyWeightDS-Flow
Neither trained on Inception features (DyWeight with L2)7.27 / 2554.51 / 260
Both trained with DyWeight's Inception-feature loss (DS-Flow column: DS-Flow added on DyWeight's solver)3.02 / 2282.85 / 201

DyWeight learns weights for combining past gradients and supervises the final sample in the features of the Inception network that FID itself uses, so part of its gain may be specific to FID (Kynkäänniemi et al., ICLR 2023); we therefore also report FD_DINOv2. With this objective it reports 3.02, 2.61 and 2.18 on CIFAR-10 and 6.30, 6.15 and 4.22 on ImageNet-64 at NFE 5, 6 and 8. Without it, DS-Flow has the lower FID and the two are within 2% on DINOv2 features (Stein et al., NeurIPS 2023), which neither method trains on; with it, the DS-Flow corrector is complementary and lowers both of DyWeight's scores further. DyWeight's own default setting (10K teacher samples) also gives 4.69 on ImageNet-64 at NFE 8 (its Table A.4), above DS-Flow's 4.33.

DS-Flow also works at a different level: LD3 and DyWeight improve the outer solver, i.e. where to step and how to combine past gradients, whereas DS-Flow refines the state within each step.

AuthorsOpenReview ↗

Response to Reviewer BVQk (3/3)

Q1. Prototype or hardware-in-the-loop validation. Thank you for the suggestion. A prototype or hardware-in-the-loop measurement needs the fabricated DS-Flow chip, which does not exist yet; we plan to tape it out next year. Until then, the circuit-level simulation in W1 follows the standard pre-silicon practice described there, since a fabrication cycle takes many months.

Q2. Comparison with LD3 and DyWeight. Thank you; please see W2. We will add both comparisons, with their settings, to the revision.

Q3. End-to-end wall-clock on GPU. Thank you for this request, which makes the efficiency claim clearer. CIFAR-10, per sample on one A100, with the corrector run on the GPU; FID is taken from Table 1:

MethodNFEFIDLatency per sample
GITS83.248.11 ms
PAS83.879.56 ms
DS-Flow82.7010.12 ms
GITS102.4910.32 ms
PAS102.8412.07 ms

At equal NFE=8, DS-Flow gives the best FID among these methods, and at matched quality its latency is comparable: GITS needs NFE=10 (10.32 ms) to pass DS-Flow's 2.70 at NFE=8 (10.12 ms), and PAS at NFE=10 (12.07 ms) remains worse. From the measured latencies above, the 50 micro-corrections take about 2 ms per sample on the GPU at NFE=8 (about 0.25 ms per interval), DS-Flow's gap to GITS, which adds negligible computation to its iPNDM base solver. On DS hardware they cost 1.4 µs per interval in simulation, about 11 µs per sample at NFE=8, so the end-to-end time is dominated by the denoiser calls.

Q4. One corrector per NFE. Thank you for the question. Yes, we currently train one corrector per (dataset, NFE) setting, with the same architecture and hyperparameters. The cost is small: about 58K parameters and about 3 A100-hours per setting on CIFAR-10, with the denoiser frozen. This does not limit DS-Flow's practical use: a deployment usually runs at a fixed NFE budget, and moving to another budget only means programming a different set of the same ~52K couplings into the same circuit (Appendix F). A corrector that generalizes across budgets is possible in principle: it would replace the learned interval embedding e_i with the interval's noise level and step size as continuous inputs and train on teacher trajectories from randomly sampled intervals, much as diffusion models are trained on random timesteps. We have not tested this and will note it as future work in Appendix H.

Thank you again for the thoughtful review and constructive questions. We hope these answers address your concerns, and we would be glad to discuss any point further. If any additional experiment would help, we would be happy to run it during the discussion period.

Reviewer_o9RUOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces two (i.e., global and local) stream dynamical system correctors to allow for effective few-step (e.g., from 5 to 10 NFEs) diffusion sampling. At each timestep, the inputs to the two systems are designed to include different types of features in order to make an accurate prediction of the next diffusion state. In particular, the two dynamical systems intend to make corrections to the denoising direction over each coarse time-interval. The coefficients of the two dynamical systems are first trained offline by using a teacher ODE solver per time step with a certain number of random noise samples. At each timestep in the inference stage, the GPU applies a single NFE to compute the denoising direction and then passes the obtained information to the DC circuit so that the two dynamical systems can be solved fast in the analog domain. The output from the DC circuit is the diffusion state, which is sent back to the GPU for the next step's computation.

Critical strengths

The paper introduces two dynamical system correctors into few-step diffusion sampling, which is both interesting and innovative. The DC circuit is employed in the pipeline in order to solve the two dynamical system correctors effectively in the analog domain. The two systems need to be trained offline via a teach ODE solver before being employed for sampling in the inference stage.

Critical weaknesses

(1) My major concern is that it is not clear if the new method also works for large-scale latent diffusion models, such as Stable Diffusion 3 or SDXL. It may happen that more complex latent-space errors might require more PCA components, which could increase the complexity and cost of the dynamical-system hardware.

(2) Table 1 should also include the FIDs of the teacher ODE solver for comparison. In its current form, it is not clear how much of a performance gap remains between the new method and the reference teacher ODE solver.

(3) The new sampling method essentially attempts to reduce the prediction error for the next diffusion state relative to the teacher solver. It would be great if the authors could include the results of the error reduction in percentage compared to applying an Euler sampling procedure.

Questions for authors

see my comments above. In addition, I have a few questions below:

(1) I don't find the overall training time for the two dynamic systems in the main paper.

(2) I would assume that the two dynamical systems need to be trained per time step. As the number of timestep increases, the number of total trainable parameters would also increase. Am I right? Please clarify it in the revision.

If the authors could address my raised concerns properly, I am happy to raise the score.

5 comments
AuthorsOpenReview ↗

Response to Reviewer o9RU

We sincerely thank Reviewer o9RU for the careful review and clear summary, and for finding the two-DS-corrector design interesting and innovative. We address each point below and will add all of it to the revision.

W1. Large latent models and the number of PCA components. Thank you for raising this important question. The corrector only needs the base direction d_i and the current state, which a latent sampler provides in the same form, so it carries over without architectural change. We evaluated latent models from 256×256 to 1024×1024 with the same corrector architecture and the base model frozen (FID):

ModelSettingBaselinesDS-Flow
FFHQ-LDM, 256×256NFE=5UniPC 18.66, DPM-Solver++ 27.9411.85
BK-SDM-Tiny, SD v1.x latent, 512×512 text-to-imageMS-COCO, NFE=8Euler 28.4118.35
SDXL, 1024×1024 text-to-imageMS-COCO, NFE=8Euler 21.23 (80 steps: 20.29)19.78

For the text-to-image runs, the only substantive change to the paper's recipe is the loss: LPIPS is defined on RGB images and cannot be applied to the 4-channel VAE latent, so we replace L2 + LPIPS with a pseudo-Huber latent loss, as in improved consistency training (Song & Dhariwal, 2024). The frozen base U-Net runs in fp16, as is standard for Stable Diffusion; text conditioning uses classifier-free guidance (scale 7.5), which enters only through the base direction d_i. The optimizer (Adam, lr 10⁻³, cosine schedule), n=50 and the 10 teacher checkpoints per interval are unchanged. We will add all three results to the appendix.

On the number of PCA components, the text-to-image runs keep the paper's k=4, with no extra components: repeating Figure 2 in the SDXL 1024×1024 latent space (24 prompts), the top 4 PCs explain 98.0% of the trajectory curvature variance, close to the >99% of Figure 2a in pixel space. Two properties keep the hardware cost bounded even if more components were needed:

  • The global DS works on k-dimensional projections, so its coupling parameters depend on k and not on the latent size. Doubling k from 4 to 8 would add about 2K coupling parameters to the ~52K in total (Appendix F).
  • The local stream is meant for the residual that a small subspace misses, and its trained parameters do not grow with the latent resolution because its 3×3 kernel is shared. Its node and coupling count grows with the spatial grid H×W (Appendix H): SDXL and SD3 latents at 1024×1024 are both 128×128 grids, smaller than the 256×256 LSUN Bedroom images already evaluated in the paper.

W2. Teacher FID in Table 1. Thank you for this helpful suggestion; we agree and will add a teacher row to Table 1. The teacher is the Euler solver that generates the training trajectories (Appendix B), the same for all three datasets. Under the protocol of Table 1 (50k samples), it reaches an FID of 2.37 on CIFAR-10, 2.62 on ImageNet-64 and 3.99 on LSUN Bedroom. DS-Flow comes closest at NFE=10 with AFS: 2.39 on CIFAR-10, within 0.02 of the teacher.

W3. Error reduction relative to Euler. Thank you for the suggestion. At the same NFE=5 without AFS, DS-Flow reduces the FID of the Euler sampler by 84% on CIFAR-10 (49.66 to 7.95), 61% on ImageNet-64 (43.81 to 17.03) and 57% on LSUN Bedroom (34.34 to 14.90), all from Table 1. We will report these reductions in the revision.

Q1. Training time. Thank you for noting this omission. With the denoiser frozen, training the two DS streams together for one NFE setting takes about 3 A100-hours on CIFAR-10, about 5 on ImageNet-64 and about 24 on LSUN Bedroom. We will state this in the main paper.

Q2. Parameters per timestep. Thank you for the careful question; the answer depends on which time step is meant.

  • Solver intervals, one per NFE: your understanding is correct that the parameter count grows with the number of intervals, but only through the learned interval embedding e_i (Eqs. 9 and 12), a 64-dimensional vector per stream, i.e. 128 parameters per extra interval on top of the ~52K shared couplings. The same corrector size already serves every NFE from 5 to 10 in Table 1, and DS-Flow targets this low-NFE regime, so the hardware cost of more intervals stays negligible.
  • Micro-steps within an interval: all 50 micro-steps share one corrector, so the count does not grow with them. Fifty is also enough: in Table 4, going from 40 to 50 micro-steps lowers FID by less than 0.4 on all three datasets, and at NFE=10 DS-Flow is within 0.02 of the teacher (W2).

A separate corrector is trained for each (dataset, NFE) setting with the same architecture (Appendix B); since a deployment usually runs at one fixed NFE, this does not limit practical use. We will state this in Section 3.2.

Thank you again for the constructive review. We hope these answers and additions address your concerns, and we would be glad to clarify anything further.

Many thanks for addressing my raised concerns.

(1) What are the FIDs of the teacher ODE solver in the new experiments for FFHQ-LDM, BK-SDM-Tiny, and SDXL? I wonder if the image quality gap becomes larger for big model sizes?

(2) I have another question. Does the number of parameters per timestep in DS-stream scale with the model size? If not, I would think that some information may not be captured by the DS-stream as the model size increases? The performance gap between the teacher ODE solver the new method would increase as the model size increases.

AuthorsOpenReview ↗

Response to Reviewer o9RU follow-up

Thank you for the follow-up, and we are glad the earlier answers addressed your concerns.

(1) Teacher FID in the latent experiments. We measured each teacher, the reference solver that generates the training trajectories with 10 sub-steps per student interval. For BK-SDM-Tiny and SDXL we use the protocol of our SDXL runs: 5,000 MS-COCO val2017 captions, guidance 7.5, and clean-FID against the 40,504 val2014 images. For FFHQ-LDM we use 50,000 samples and the FFHQ 256×256 statistics of the diff-sampler code, with the checkpoint's EMA weights, which the LDM code uses when sampling.

Model (denoiser size)NFEBase samplerDS-FlowTeacher
FFHQ-LDM, 256×2565UniPC 18.6611.8511.46 (50 steps)
BK-SDM-Tiny, 512×5128Euler 28.4118.3517.05 (80 steps)
SDXL, 1024×10248Euler 21.2319.7820.29 (80 steps)

The SDXL teacher has converged under this protocol: with 50, 60 and 80 steps it scores 20.28, 20.29 and 20.29, in line with published many-step SDXL values on COCO, such as 19.36 at guidance 6 and 20.39 at guidance 8 for the SDXL teacher in DMD2 (Yin et al., 2024).

The signed FID differences (DS-Flow minus teacher) are +0.39 on FFHQ-LDM, +1.30 on BK-SDM-Tiny, and -0.51 on SDXL. These experiments do not show a monotonically increasing gap with denoiser size.

We also note that on SDXL, DS-Flow even achieves a lower FID than the teacher: 19.78 versus 20.29. We believe a plausible explanation is that matching the teacher's trajectories and minimizing FID against real images are different objectives. Small deviations introduced by the learned correction may move the generated feature statistics closer to those of real images, yielding a lower FID without exactly reproducing the teacher's outputs. Finite-sample estimation may also contribute to this difference.

(2) Corrector size and model size. The corrector does not grow with the denoiser: it has 54,345 trained parameters for both BK-SDM-Tiny and SDXL, close to the ~58K of the pixel-space corrector, and its interval embedding adds 128 parameters per solver interval as before. A fixed corrector size does not by itself imply information loss as the denoiser grows, for three reasons:

  • The denoiser's knowledge enters through its output. At every micro-step the corrector receives the direction d_i computed by the full model, so it does not need to represent the model; it learns only how the state moves inside one coarse step, that is, the truncation error of the solver.
  • That error lives in the state space, whose size is set by the latent resolution rather than by the number of denoiser parameters, and it stays concentrated: in the SDXL 1024×1024 latent space the top 4 PCs explain 98.0% of the trajectory curvature variance, close to the >99% in pixel space (first reply).
  • In (1), DS-Flow is 1.30 FID above its teacher on BK-SDM-Tiny and 0.51 below its teacher on SDXL, while the trained corrector keeps the same size. This does not show a widening teacher gap between these two settings, although it cannot guarantee sufficient capacity at every model size. If a larger model needed more capacity, the subspace size k could be raised; as noted in W1 of our first reply, doubling k from 4 to 8 would add about 2K coupling parameters.

Thank you again for the careful questions; we would be glad to clarify anything further.

Reviewer_3NHJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

DS-Flow proposes a learned corrector for few-step diffusion sampling. It splits each solver step into multiple micro-steps and applies a correction at each one without additional denoiser calls. The corrector has two streams, one in a PCA subspace and one in pixel space. The paper also proposes running the corrector on electronic dynamical-system (DS) hardware and reports large latency and energy savings based on circuit simulation.

Critical strengths

[S1] Figures 1–3 clearly convey the motivation for in-step correction and the dual-stream design.

[S2] With only a small number of additional parameters, the method substantially improves over the Euler baseline at low NFE.

Critical weaknesses

[W1] Novelty Claim

The paper claims to be the first method to bridge electronic dynamical systems with diffusion sampling. However, prior work such as [1] already solves the diffusion ODE with analog circuits. I may be missing the exact distinction, but this claim seems to need clarification or more restrained wording.

[W2] Extension to Latent Diffusion Models

It is unfortunate that the experiments are limited to low-resolution, pixel-space image generation. Most recent diffusion models are latent diffusion models capable of generating high-resolution images. If extending the method to latent diffusion models is difficult, I believe the contribution of this paper is considerably reduced.

[W3] Fairness of Comparison

DS-Flow is trained on teacher trajectories, which makes it closer to a distillation-based method. However, it is compared mainly against training-free solvers and very lightweight learned methods. It would be more convincing to compare with learned solvers of similar scale, such as EPD-Solver (ICCV 2025).

[W4] Hardware Evaluation

All hardware results are obtained from simulation, and no actual chip has been fabricated. It is therefore difficult to judge how realistic the reported speedup and energy savings are.

Questions for authors

[Q1] How does DS-Flow differ from prior analog-hardware diffusion work such as [1]?

[Q2] Can you report performance on at least one high-resolution (1024×1024 or higher) latent diffusion model?

[Q3] How does DS-Flow compare against learned solvers with similar training cost and parameter count?

[Q4] The paper reports only the per-interval latency of the corrector on simulated DS hardware. Could you report the following?

  • The latency of the corrector when it runs on a standard GPU instead of DS hardware.
  • The end-to-end wall-clock time to generate a single image, including denoiser calls.
  • The training time of the corrector.

[Q5] Generating high-resolution images would require a large number of nodes in the local stream. Is this feasible to realize in hardware?

[1] Yang, Jichang, et al. "Resistive memory-based neural differential equation solver for score-based diffusion model." Nature Communications (2026).

8 comments
AuthorsOpenReview ↗

Response to Reviewer 3NHJ (1/3)

We sincerely thank Reviewer 3NHJ for the careful and constructive review, and for finding Figures 1 to 3 clear and the low-NFE gains over Euler substantial. We answer each point below.

W1 / Q1. Relation to Yang et al. (Nat. Commun. 2026). Thank you for pointing us to this work, which we had missed; we will cite and discuss it in Section 2. We respectfully maintain that the two works are different, and that the difference goes beyond scale.

A DS is not a general analog circuit. In our paper, an electronic DS (Section 2.3) is the class of circuits that grew out of Ising machines: capacitor voltages coupled by programmable resistors relax to a minimum of a learned energy, a Hamiltonian (Eqs. 3 to 5), and the answer is read at that equilibrium. Yang et al. build a different kind of analog computer, a neural differential equation solver: resistive-memory crossbars compute a small score network and an op-amp integrator feeds it back, so the circuit's transient traces the reverse diffusion equation, and the sample is the state at the end of the integration rather than an equilibrium.

Why DS-Flow is designed this way. We first tried the route Yang et al. take, with a DS: letting it carry out the whole generation. At image scale it did not work: analog noise accumulates along the trajectory, and a DS of practical size cannot represent the score of natural images. We therefore kept the pretrained denoiser on the GPU and gave the DS only the part that a small energy-based circuit does well, the in-step correction of a coarse solver step, whose error has a dominant low-dimensional part and a spatially structured remainder (Section 3.1).

What this changes. Yang et al. show that a fully analog solver can generate samples on small proof-of-concept tasks: a 2-D circular distribution, three EMNIST letters in a 2-D VAE latent space and, in their supplement, three FashionMNIST classes in an 8-D latent space. This is a valuable demonstration, but it has not yet reached natural-image benchmarks. DS-Flow brings a DS into the sampling of real pretrained image models, which takes three steps that are new in our work:

  1. Finding where a small DS can help: the in-step correction gap above, and the dual-stream corrector designed for it, which improves sampling even without DS hardware (Q4).
  2. Making the DS work on real tasks: the corrector improves pretrained diffusion models on standard image benchmarks, from CIFAR-10 to LSUN Bedroom 256×256 and latent text-to-image models (W2), rather than small proof-of-concept tasks.
  3. Keeping the hardware benefit: on the DS, the 50 corrections of an interval take about 1.4 µs in circuit simulation (W4), with no extra denoiser calls.

We will therefore keep the claim but make it precise in the contributions and the conclusion: DS-Flow is the first to use an energy-based DS inside the sampling loop of a pretrained diffusion model.

W2 / Q2. Latent diffusion. Thank you for this important question. The corrector only needs the base direction d_i and the current state, which a latent sampler provides in the same form, so it carries over without architectural change. We evaluated latent models from 256×256 to 1024×1024 with the same corrector architecture and the base model frozen (FID):

ModelSettingBaselinesDS-Flow
FFHQ-LDM, 256×256NFE=5UniPC 18.66, DPM-Solver++ 27.9411.85
BK-SDM-Tiny, SD v1.x latent, 512×512 text-to-imageMS-COCO, NFE=8Euler 28.4118.35
SDXL, 1024×1024 text-to-imageMS-COCO, NFE=8Euler 21.23 (80 steps: 20.29)19.78

For the text-to-image runs, the only substantive change to the paper's recipe is the loss: LPIPS is defined on RGB images and cannot be applied to the 4-channel VAE latent, so we replace L2 + LPIPS with a pseudo-Huber latent loss, as in improved consistency training (Song & Dhariwal, 2024). The frozen base U-Net runs in fp16, as is standard for Stable Diffusion; text conditioning uses classifier-free guidance (scale 7.5), which enters only through the base direction d_i. The optimizer (Adam, lr 10⁻³, cosine schedule), n=50 and the 10 teacher checkpoints per interval are unchanged. We will add all three results to the appendix. Q5 below discusses what the 1024×1024 size means for the DS hardware.

AuthorsOpenReview ↗

Response to Reviewer 3NHJ (2/3)

W3 / Q3. Comparison with learned solvers such as EPD-Solver. Thank you for pointing us to EPD-Solver, which we had not compared against; we are glad to add it. We compare using the numbers in its paper, which evaluates the same EDM checkpoints with 50k-sample FID and reports the same training-free baseline values as our Table 1 at NFE 5 on CIFAR-10 and ImageNet-64.

When reading its budgets, computation matters. EPD-Solver reports parallel NFE: each step evaluates the denoiser once and then K=2 more times in parallel, with the analytical first step replacing the very first evaluation (its Eq. 11 and Sec. 4.1), so its parallel NFE 3, 5 and 7 correspond to 5, 8 and 11 denoiser evaluations. Parallel execution keeps its GPU latency unchanged, but the computation per sample grows accordingly. At an equal or smaller number of denoiser evaluations, DS-Flow gives lower FID throughout:

Denoiser evaluationsCIFAR-10, EPD / OursImageNet-64, EPD / Ours
510.40 / 4.5518.28 / 10.67
84.33 / 2.576.35 / 4.33
11 for EPD, 10 for ours2.82 / 2.395.26 / 3.48

On training scale, EPD-Solver learns 6 to 45 parameters in about 3 minutes on CIFAR-10, while DS-Flow trains about 58K parameters in about 3 A100-hours; both stay far below distillation, where consistency distillation retrains the full ~56M-parameter model for about 1,156 A100-hours, as estimated in SFD (NeurIPS 2024). Among solvers whose offline cost is comparable to or above ours, DPM-Solver-v3 (NeurIPS 2023) is the closest reference: it estimates empirical model statistics in about 28 A40 GPU-hours on the same EDM CIFAR-10 checkpoint and reports 12.21, 8.56 and 3.50 at NFE 5, 6 and 8, against DS-Flow's 7.95, 4.59 and 2.70 without AFS.

AuthorsOpenReview ↗

Response to Reviewer 3NHJ (3/3)

W4. Hardware evaluated in simulation. Thank you for raising this point. We agree that no DS-Flow chip has been fabricated yet; Section 4.1 and the Reproducibility Statement say so. The reported speedup and energy are nonetheless grounded in three ways:

  • A detailed circuit-level estimate, not an idealized model. The DS circuit is simulated in Cadence mixed-signal simulation in 45 nm at the typical-typical corner and 25 °C, with 8-bit converters, device mismatch and thermal noise. The 1.4 µs per interval covers data conversion as well as computation: one DAC write (0.25 µs), on-array annealing of all 50 micro-steps (1.0 µs) and one ADC read (0.15 µs) (Appendix G). The analog effects are also counted in quality: 8-bit conversion costs about 0.2 FID relative to fp32 (Table 5), and thermal noise up to 10% adds about 0.2 FID (Table 6).
  • The standard practice of the hardware community. New analog architectures are evaluated in simulation before tape-out: recent ReRAM in-memory accelerators such as RAELLA (ISCA 2023) and ASADI (HPCA 2024) were proposed and evaluated this way, without a fabricated prototype, and modeling tools such as CiMLoop (ISPASS 2024) exist for this purpose. Prior DS hardware, NP-GL and EADS, was evaluated with the same pre-silicon protocol that we follow.
  • Silicon precedent. Our timing and power constants are taken from the BRIM design and the simulated NP-GL and EADS designs. Ising hardware has been built in CMOS, both as digital annealers, e.g., Yamaoka et al. (JSSC 2016) and STATICA (ISSCC 2020), and as physics-based coupled-oscillator chips, e.g., Moy et al. (Nat. Electron. 2022, 1,968 nodes) and Lo et al. (Nat. Electron. 2023, 48 all-to-all nodes). What remains unbuilt is the DS-Flow chip itself, which we plan to tape out next year.

Q4. GPU latency, end-to-end time and training time. Thank you for these concrete requests; we answer them in order. CIFAR-10, per sample on one A100; FID is taken from Table 1, and latency is measured with the corrector run on the GPU:

MethodNFEFIDLatency per sample
GITS83.248.11 ms
PAS83.879.56 ms
DS-Flow82.7010.12 ms
GITS102.4910.32 ms
PAS102.8412.07 ms
  1. The corrector on a GPU: from the measured latencies above, it takes about 2 ms per sample at NFE=8 (about 0.25 ms per interval), DS-Flow's gap to GITS, which adds negligible computation to its iPNDM base solver.
  2. End to end, including denoiser calls: on a GPU, DS-Flow takes 10.12 ms per sample at NFE=8, with the best FID among these methods; GITS needs NFE=10 (10.32 ms) to pass it, and PAS at NFE=10 (12.07 ms) does not. With the corrector on DS hardware, the 50 micro-corrections cost about 1.4 µs per interval, about 11 µs per image at NFE=8 (W4), so the end-to-end time is essentially that of the 8 denoiser calls.
  3. Training: with the denoiser frozen, the corrector for one NFE setting trains in about 3 A100-hours on CIFAR-10, about 5 on ImageNet-64 and about 24 on LSUN Bedroom.

Q5. Local stream at high resolution. Thank you for the question. For the latent models that generate high-resolution images today, yes; pixel-space generation at 1024×1024 would need larger DS hardware, as Appendix H notes. Two points bound the cost. First, the trained local-stream parameters do not grow with resolution, because the 3×3 grid-coupling kernel is shared across positions; what grows is the number of nodes and physical couplings, which follows the grid size. Second, in latent diffusion the local stream runs on the latent grid, not on pixels: a 1024×1024 SDXL image is a 4×128×128 latent, a 128×128 grid, smaller than the 256×256 grid of LSUN Bedroom that the paper already evaluates and that Appendix H describes as manageable.

Thank you again for the thoughtful review. We hope these answers address your concerns, and we would be glad to discuss any point further.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion for the Area Chair and Reviewers

We sincerely thank the Area Chair and Reviewers 3NHJ, o9RU and BVQk for their time and for a constructive discussion. Below we summarize our contribution, our clarifications, the new experiments, and the remaining concerns.

1. Contribution. DS-Flow opens a new direction: using an energy-based dynamical system (DS), a class of analog circuits so far used mainly for combinatorial optimization, inside the sampling of modern pretrained diffusion models. To our knowledge, it is the first work to do so on real image models and standard benchmarks. The contribution has three parts:

  • A new finding. At low NFE, the error within each solver step has a dominant low-dimensional part and a spatially structured remainder. A small DS can correct an error of this structure.
  • A new algorithm. A dual-stream corrector refines the state within each solver step, with the pretrained denoiser unchanged and no extra denoiser calls. It already improves sampling in software, without any DS hardware.
  • A new hardware path. The closest analog work runs the whole generation in analog and remains at small proof-of-concept tasks: even for images, its solver works on a 2- to 8-dimensional latent state, with a VAE decoder mapping it to pixels. DS-Flow keeps the denoiser on the GPU and gives the DS only the in-step correction, which costs microseconds in circuit simulation. This division of work is what lets a DS reach natural-image benchmarks.

2. Clarifications.

  • Novelty and prior analog work (3NHJ W1, Q1). We explained how DS-Flow differs from Yang et al. (Nat. Commun. 2026) in the kind of circuit and in reaching standard image benchmarks, and made the novelty sentence more precise.
  • Hardware evaluated in simulation (3NHJ W4; BVQk W1, Q1). The estimate is a circuit-level simulation that includes data conversion and analog noise, the standard pre-silicon practice for new analog hardware.
  • Hardware at scale (3NHJ Q5 and follow-up). The global stream is small at any image size and, in our ablation on CIFAR-10 and ImageNet-64, suffices from NFE=8 on; the local stream can be tiled.
  • Training cost and corrector size (3NHJ Q4; o9RU Q1, Q2 and follow-up; BVQk Q4). The corrector is small, trains in about 3 to 24 GPU-hours with the denoiser frozen, and has the same size for BK-SDM-Tiny and SDXL.

3. New experiments.

  • Latent diffusion models up to SDXL at 1024×1024 (3NHJ W2, Q2; o9RU W1). The same corrector architecture improves FFHQ-LDM, BK-SDM-Tiny and SDXL over their base samplers at NFE 5 or 8.
  • Comparison with recent learned solvers: EPD-Solver, LD3 and DyWeight (3NHJ W3, Q3 and follow-up; BVQk W2, Q2). At an equal number of denoiser calls DS-Flow has the lower FID in most settings; at equal sequential steps EPD-Solver is ahead on ImageNet-64 at the lower budgets. DS-Flow is also complementary: added on top of DyWeight, it improves it further.
  • End-to-end GPU latency (3NHJ Q4; BVQk Q3). Run entirely on a GPU, DS-Flow has latency comparable to GITS and lower than PAS at matched quality.
  • Teacher FID (o9RU W2 and follow-up). On CIFAR-10, DS-Flow is within 0.02 FID of its teacher at NFE=10, and the latent experiments do not show a gap that grows with denoiser size.

4. Status of each reviewer's concerns.

  • Reviewer BVQk wrote that W1 and W2 are addressed and raised the score. The remaining concern is limited applicability in a niche field. Our view: the algorithm does not depend on the hardware, and this work shows one concrete way to widen the use of such hardware.
  • Reviewer o9RU thanked us for addressing the raised concerns and asked two follow-up questions, on teacher FID and on corrector size for larger models, which we answered.
  • Reviewer 3NHJ wrote that the revised novelty claim and the latent results address W1 and W2, and accepted the simulation-based evaluation. The reviewer keeps the score, with three remaining points: the novelty claim is narrower than in the submission; the practical benefit seems small, since GITS is better on a GPU at similar latency and the energy saving applies only to the corrector; and the reviewer states that they are not in a position to judge hardware feasibility at scale and leaves it to the AC. Our position: the method and experiments are unchanged and only the wording is more precise; we do not claim to be the fastest GPU sampler, and the value is better quality from the same pretrained model at almost no added cost.

5. Limitations we acknowledge. No DS-Flow chip has been fabricated yet, so hardware results come from circuit simulation. This is the standard practice of the hardware community: new analog architectures at venues such as ISCA and HPCA are commonly proposed and evaluated in simulation before tape-out, without a fabricated prototype (e.g., RAELLA, ASADI), as was prior DS hardware.

We are grateful for the careful reading and suggestions, which are reflected in the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.