acceptodds
Under review as a conference paper at ICLR 2027

Kaiwu-PyTorch-Plugin: Bridging Deep Learning and Photonic Quantum Computing for Energy-Based Models

Abstract

This paper introduces the Kaiwu-PyTorch-Plugin (KPP) to bridge Deep Learning and Photonic Quantum Computing across multiple dimensions. KPP integrates the Coherent Ising Machine into the PyTorch ecosystem, addressing classical inefficiencies in Energy-Based Models. The framework facilitates quantum integration in three key aspects: accelerating Boltzmann sampling, optimizing training data via Active Sampling, and constructing hybrid architectures like QBM-VAE and Q-Diffusion. Empirical results on single-cell and OpenWebText datasets demonstrate KPP’s ability to achieve SOTA performance, validating a comprehensive quantum-classical paradigm.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Artificial Intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_n6G3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Kaiwu-PyTorch-Plugin (KPP), a framework that connects PyTorch-based energy models with Ising solvers, including coherent Ising machines (CIMs) and simulated annealing. The framework aims to provide a reusable interface for incorporating Boltzmann machines into existing deep learning architectures while keeping neural network training and optimization within PyTorch. The authors demonstrate the framework through three applications: QDiffusion, which uses energy-based guidance for generation; QVAE, which incorporates a Boltzmann-machine prior for latent representation learning; and Boltz-Flow, which uses conditional Boltzmann states to initialize flow-matching policies. Experiments cover mathematical reasoning, text and molecular generation, single-cell data integration, and robotic manipulation. The results suggest that the proposed framework can support different energy-based components across several application domains, with improvements over selected baselines.

Critical strengths

The paper addresses a practical challenge in combining energy-based models with existing deep learning frameworks. Its separation between PyTorch-based model training and external Ising optimization provides a potentially useful abstraction for developing hybrid learning systems. The architecture and workflow diagrams in Figures 1 and 2 help illustrate how this interface can be reused across different model compositions. The overall organization is also reasonably clear, making the main implementation idea accessible to readers from different backgrounds.

Critical weaknesses

Although the motivation for integrating PyTorch with Ising solvers is understandable, the paper could better distinguish its core technical contributions from the individual application models. In particular, it is not always clear which improvements arise from the proposed KPP interface itself and which are primarily attributable to the task-specific designs of QDiffusion, QVAE, and Boltz-Flow. The introduction would benefit from a concise statement of the main contributions, emphasizing what is technically new relative to existing solver-assisted Boltzmann-machine frameworks. A clearer explanation of the distinctive capabilities of KPP would also help readers understand its broader significance. Since the title emphasizes photonic quantum computing, the paper should more clearly distinguish the benefits of the software interface from any benefits specifically attributable to the photonic backend.

Questions for authors
  1. Could the authors clarify the main technical novelty of KPP compared with existing PyTorch-based Boltzmann-machine implementations and solver integration approaches? A concise list of contributions or a more explicit discussion of the interface design would be helpful.
  2. Could the authors provide additional controlled comparisons using matched inference budgets, particularly for the text-generation experiments in Table 2 and the different replanning intervals in Table 7? Such comparisons would help isolate the benefits of the proposed energy-based components.
  3. Could the authors provide more information about computational efficiency and reproducibility, including end-to-end inference latency, solver-call overhead, CIM versus simulated-annealing performance, and variability across experimental runs? These details would help readers assess the practical benefits of adopting KPP.
Reviewer_sCFLOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors develop a plugin named Kaiwu-PyTorch-Plugin (KPP) that integrates PyTorch based energy models with Ising solvers while preserving the flexibility of model training and optimization within PyTorch. The authors demonstrate its versatility through three applications: QDiffusion for guided generation, QVAE for latent representation learning, and Boltz-Flow for flow-matching. The experimental results show that the flexiblity of KPP for diverse machine learning tasks.

Critical strengths

KPP offers a unified PyTorch-to-Ising interface and demonstrates its use in multiple architectures, including diffusion, VAEs, and flow matching, which could represent a useful infrastructure contribution.

Critical weaknesses
  1. Downstream task results alone do not establish that the proposed interface is technically novel, efficient, or responsible for the improvements.
  2. Several methods used for comparison are presented without appropriate references or sufficient descriptions.
Questions for authors
  1. How does KPP differ from existing PyTorch-based solvers?
  2. Why is the proposed interface needed? What technical limitations of existing tools does it resolve?
  3. Does using an Ising solver improve performance, efficiency, or solution quality compared with classical alternatives?
  4. What are the conversion overhead, communication cost, wall-clock time, memory requirements, and scalability?
  5. What happens when the Ising solver is replaced by a conventional sampler or optimizer while keeping the architecture fixed?
Reviewer_QavDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces Kaiwu-PyTorch-Plugin, a software interface connecting PyTorch energy-based models to external Ising solvers, such as simulated annealing and coherent Ising machines. Examples of this software interface are demonstrated via: 1) QDiffusion, which uses conditional energy models and evaluated on language, mathematical reasoning, molecule generation, and vision-language-action policies, 2) QVAE, using a Boltzmann-machine prior for latent representation learning, evaluated on single-cell integration, and 3) Boltz-Flow: Uses a conditional Boltzmann model to generate the initial state of a flow-matching robotic policy. The paper then reports improvements relative to some baselines across several benchmarks, including MATH-500, molecular generation, single-cell integration, and LIBERO robot manipulation.

Critical strengths
  1. Interface correctness: this interface is demonstrated across diverse tasks, such as language generation, molecular generation, single-cell integration, and robotic manipulation. The resulting performance on these benchmarks is evidence of the correctness of this software interface.
Critical weaknesses
  1. Presentation (formatting and figures): The text formatting (such as subtitle fonts, color) seems quite off that of the ICLR template. Most of the figures in the paper need a lot of polishing in terms of visual quality, and function calls in many of the figures, such as “get_hidden” in Figure 3 aren’t mentioned or explained anywhere.
  2. Contribution relative to existing tools is not established: Closely related tooling already exists (like D-Wave PyTorch Plugin). Mostly, I feel there is no comparison with existing tools and that it is insufficient. This paper presents itself as a software infrastructure contribution, but the interface (mapping BM parameters to an Ising matrix, converting returned spins to binary states, and backpropagating through energies on those states) follows a standard pattern. There is no discussion on any technical challenges that KPP solves beyond supporting a different backend.
  3. Solver backends are not specified: No implementation details are given for simulated annealing. Section 3 states “We therefore report backend settings, retained-state counts, and filtering separately” but they do not appear in the paper.
Questions for authors
  1. For each experiment in Tables 1–8, was the solver a physical CIM, a CIM simulator, or SA? What are the detailed settings for each, and what is the “Kaiwu solver”?
  2. What is the architecture of the QDiffusion energy model, and for the math experiments, how many solver calls were made?
  3. Mostly, I feel this PyTorch-to-Ising interface is a standard pattern already implemented in existing tools. What technical challenges does Kaiwu exactly solve that other standard patterns do not?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.