acceptodds
Under review as a conference paper at ICLR 2027

Transport Adjoint Matching

Abstract

Sampling from high-dimensional discrete distributions specified only through unnormalized mass functions is a fundamental problem in statistical physics, combinatorial optimization, and probabilistic inference. Adjoint matching provides an effective framework for learning nonequilibrium neural samplers, but the translation-based discrete Schr\"odinger-bridge construction relies on additive reference dynamics and deterministic cyclic endpoint correspondences, restricting its applicability when such structure is unavailable. We introduce Transport Adjoint Matching (TAM), which represents the optimal-control potential ratio through transport-compatible couplings between reference endpoint kernels, thereby removing the need for deterministic endpoint correspondences. For coordinate-local reference dynamics satisfying a shared-fiber marginal condition, we further derive a fiberwise Rao–Blackwellized target that marginalizes both the auxiliary transport variable and the local endpoint symbol. The resulting target preserves the exact population adjoint identity, requires only known local endpoint weights, local energy differences, and corrector potential ratios, and avoids explicit transport-matrix construction. We evaluate TAM on high-dimensional Ising and four-state Potts models across three temperature regimes, and further test a genuinely non-additive Potts reference for which the deterministic cyclic-shift construction is unavailable. TAM achieves accurate sampling across the standard benchmarks and remains effective under the non-additive reference.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
discrete sampling, adjoint matching, Schrödinger bridge, transport coupling, Rao–Blackwellization, unnormalized distributions

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VCAhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies discrete Schrödinger-bridge neural samplers and generalizes the DASBS controller target. DASBS relies on a deterministic cyclic translation between reference endpoint kernels; this paper replaces it with any transport-compatible coupling. For coordinate-local transitions whose endpoint kernels share a context marginal, it derives a coupling-free, fiberwise Rao–Blackwellized target with O(N)O(N) cost. Validation covers Ising and 4-state Potts models at three temperatures, a target-variance and ablation study, and one non-additive Potts reference.

Critical strengths
  1. An exact, lower-variance controller target at no extra network cost. Thm 4.2 and Eq. (15) show that Rtx→y(A)R^{x\to y}_t(A) is unbiased and removes both auxiliary randomness sources. Fig. 2 shows roughly 1–2 orders of magnitude variance reduction at early bridge times. Eq. (16) uses the same local energy ratios and the same single corrector forward pass as DASBS, so it is a drop-in change with the same training time (Table 2).

  2. Correct and transparent theory. Thms 4.1–4.2, A.3 and C.1–C.4 are short and check out. Coupling-independence (C.4) and the reduction to factorized kernels (C.3) are stated precisely, which makes the scope of the result easy to audit.

  3. Strong sample quality on the standard benchmarks. TAM is best or near-best among uniform-based samplers in most cells of Tables 2–3. At criticality it improves even on the values reported in the DASBS paper (e.g., critical Ising EW2 2.3 vs 12.1; critical Potts EW2 1.0 vs 20.0). Fig. 1 shows that this agreement holds at the distribution level, not just for scalar metrics.

Critical weaknesses
  1. Removing the cyclic structure seems incremental, and the transport framing adds limited value. Extending the DASBS adjoint-matching construction to other compatible group actions or kernels appears straightforward. Generalizing beyond cyclic maps therefore seems incremental, and I am not convinced that the transport framing provides a substantial conceptual advance or that the proposed kernels offer a clear practical advantage.

  2. The practical advantage over DASBS is not convincingly established. The DASBS row in Tables 2–3 differs from its original paper, while every other baseline row is unchanged. Examples include critical Ising ΔMag\Delta\mathrm{Mag} (5.7×10−15.7\times10^{-1} here vs 5.7×10−25.7\times10^{-2} originally), EW2EW_2 (53.7 vs 12.1), and critical Potts ΔMag\Delta\mathrm{Mag} (4.2×10−24.2\times10^{-2} vs 4.1×10−34.1\times10^{-3}). The paper does not clarify whether DASBS was rerun, its training budget (3.0k vs 3.75k steps), or its seed count. Against the published DASBS results, some gains shrink or reverse: high-temperature Ising ΔMag\Delta\mathrm{Mag} is 2.7×10−32.7\times10^{-3} for DASBS versus 3.4×10−33.4\times10^{-3} for TAM, while critical Potts is 4.1×10−34.1\times10^{-3} versus 3.4×10−33.4\times10^{-3}. A direct comparison between TAM and the DASBS deterministic-coupling target under the same pipeline, budget, and multiple seeds is also missing; Table 4 compares only against independent coupling, a weaker reference point. Finally, variance reduction should improve training efficiency, but no convergence curves or step-matched comparisons establish this benefit.

  3. The experimental scale and baseline coverage are limited. The experiments reuse DASBS’s main benchmarks: Ising L=24L=24 (D=576D=576, N=2N=2) and Potts L=16L=16 (D=256D=256, N=4N=4), with approximately 144k parameters and no scaling study in lattice or alphabet size. Since the fiberwise target averages over NN symbols, larger alphabets are particularly relevant, yet neither its variance gains nor its O(N)O(N) evaluation cost is tested beyond N=4N=4. All non-DASBS baseline results—LEAPS, UDNS, DNFS, MDNS, and MH—are copied from DASBS without reruns or variability estimates. MDNS still performs better on high-temperature Ising ΔCorr\Delta\mathrm{Corr} (7.4×10−47.4\times10^{-4} vs 1.0×10−31.0\times10^{-3}) and EW2EW_2 (0.1 vs 1.9), low-temperature Ising ΔMag\Delta\mathrm{Mag} (9.0×10−39.0\times10^{-3} vs 1.5×10−21.5\times10^{-2}), and low-temperature Potts ΔMag\Delta\mathrm{Mag} (8.4×10−48.4\times10^{-4} vs 9.9×10−49.9\times10^{-4}). MH is better on all three high-temperature Ising metrics, so the “best” claims depend on restricting comparisons to uniform-based neural samplers. Newer uniform-diffusion samplers cited by DASBS, such as PDNS (Guo et al., ICLR 2026), are absent. Moreover, all tested references remain coordinate-factorized, including the non-additive example, leaving the broader transport claim insufficiently explored.

I would be happy to increase my score if the authors convincingly demonstrate that the proposed generalization to broader group actions or kernels is practically useful. For example, by enabling useful new reference dynamics, improving scalability or computational efficiency, or showing a clear training advantage over DASBS under matched conditions.

Questions for authors
  1. Novelty and practical usefulness. Could the authors clarify what TAM adds beyond extending DASBS adjoint matching to other compatible group actions or kernels? Which useful reference dynamics does this generalization enable, and can they improve scalability, computational efficiency, or training performance? In particular, what new capability does the non-additive experiment in Table 5 demonstrate compared with existing DASBS constructions?

  2. Fair comparison and training speed. Was DASBS rerun, and with which hyperparameters, training budget, and random seeds? Why do its results differ from the original paper—for example, critical Ising ΔMag\Delta\mathrm{Mag} of 5.7×10−15.7\times10^{-1} versus 5.7×10−25.7\times10^{-2}? Please compare TAM with the original DASBS deterministic-coupling target under the same pipeline and training budget, report the mean and standard deviation across seeds, and include training curves such as EW2EW_2 versus gradient steps. Does TAM converge faster, or does it only achieve slightly better results at a fixed budget?

  3. Scaling and stronger baselines. Do TAM's gains over DASBS persist, grow, or disappear with larger lattices (e.g., Ising L≥48L\geq48) and larger alphabets (e.g., Potts N≥8N\geq8)? How does TAM compare with MDNS at the same training cost in these settings? These results would show whether the method offers a useful advantage beyond the small DASBS benchmarks or whether its benefits are limited to those settings.

3 comments
AuthorsOpenReview ↗

Response to Reviewer VCAh

We thank the reviewer for the constructive assessment. We clarify TAM's relationship to DASBS and add matched controller-target comparisons, training curves, larger-scale benchmarks, and non-additive references.

1. Generalization beyond deterministic cyclic shifts

The DASBS deterministic cyclic-shift target is a special case of TAM: choosing the coupling induced by that shift recovers its endpoint correspondence. TAM generalizes this construction through the marginal compatibility condition KxCtx→y=KyK_x C_t^{x\to y}=K_y. Theorem 4.1 shows that any Markov coupling satisfying this condition, deterministic or stochastic, yields the exact population identity for the optimal-control potential ratio. This identity requires neither a group structure nor coordinate-local transitions.

For coordinate-local transitions whose endpoint kernels share a context marginal, we further derive a coupling-free, fiberwise Rao--Blackwellized target. It marginalizes both the auxiliary transport variable and the local endpoint symbol, preserves the population identity, and has O(N)O(N) evaluation cost without constructing a transport matrix.

The non-additive Potts experiment demonstrates the practical value of this generalization. With the nonuniform refresh distribution ρ=(0.1,0.2,0.3,0.4)\rho=(0.1,0.2,0.3,0.4), endpoint probabilities depend on the absolute symbol, so cyclic translation no longer aligns the endpoint kernels. TAM remains computable from local endpoint probabilities and corrector ratios. Its energy Wasserstein distance is 1.4±0.21.4\pm0.2, compared with 220.79±1.64220.79\pm1.64 for DASBS-DM. We also evaluate a 64-site ordinal chain with five states per site and an Ehrenfest birth--death reference, whose state-dependent adjacent-symbol transitions do not admit the original cyclic-shift construction. With matched reference dynamics, architecture, and budget, TAM obtains EW2=0.34±0.02E\mathcal W_2=0.34\pm0.02, compared with 24.58±0.3024.58\pm0.30 for DASBS-DM.

2. Matched DASBS comparison and training behavior

The discrepancy in the critical-point DASBS values arises from different versions of that work. The reviewer quotes the arXiv results (EW2=12.1E\mathcal W_2=12.1 for critical Ising and 20.020.0 for critical Potts), whereas Tables 2--3 of our submission use the published ICML 2026 values (53.753.7 and 14.614.6, respectively). Other baseline results in these tables are unchanged.

We reproduced the DASBS cyclic-shift target under the same environment, architecture, training configuration, and three seeds as TAM; the results are reported in the Appendix. The direct controller-target ablation further isolates the target construction. On critical Ising, DASBS obtains EW2=54.68±54.35E\mathcal W_2=54.68\pm54.35, compared with 2.3±0.12.3\pm0.1 for fiberwise TAM; on critical Potts, the corresponding values are 16.94±9.7516.94\pm9.75 and 1.0±0.041.0\pm0.04.

We added five-stage training curves. On critical Ising (L=48L=48), TAM's EW2E\mathcal W_2 decreases from 21.5321.53 to 12.4512.45, while DASBS remains above 600600. On eight-state Potts, TAM reaches 3.273.27 after the second stage and remains below 3.33.3, whereas DASBS decreases from 341.45341.45 to 31.4231.42 over five stages. End-to-end training times are comparable: 41.56±0.1241.56\pm0.12 versus 44.04±0.0644.04\pm0.06 minutes on Ising, and 7.79±0.027.79\pm0.02 versus 7.82±0.067.82\pm0.06 minutes on Potts.

3. Scaling and target-variance evidence

We extended the matched-budget comparison to Ising (L=48L=48) and eight-state Potts (N=8N=8). Over three seeds, TAM reduces EW2E\mathcal W_2 from 727.92±97.13727.92\pm97.13 to 12.45±1.5812.45\pm1.58 on Ising, and from 31.42±13.4731.42\pm13.47 to 2.45±1.012.45\pm1.01 on Potts, with comparable training times.

The variance plot compares the sampled-transport, transport-marginalized, and fiberwise targets on critical Ising and Potts. Each marginalization further reduces target variance at the evaluated bridge times. In controlled log-potential tests, we extend the alphabet-size study to N=2,4,8,16N=2,4,8,16. Relative to the DASBS cyclic-shift target, the fiberwise target reduces variance by 87.9887.98--88.95%88.95\% at t=0.2t=0.2 and 47.0947.09--51.86%51.86\% at t=0.8t=0.8. With learned eight-state Potts correctors, the reductions at t=0.2,0.5,0.8t=0.2,0.5,0.8 are 98.14±0.03%98.14\pm0.03\%, 94.44±0.03%94.44\pm0.03\%, and 81.26±0.13%81.26\pm0.13\%, respectively. TAM target-construction time remains nearly constant as NN increases from 22 to 1616 (approximately 0.1090.109--0.1130.113 ms per call).

We also cite PDNS (Guo et al., 2026). Our best-performance claims refer only to uniform-based discrete neural samplers, not all classes of sampling methods.

We hope these clarifications and additional results address the concerns about novelty, practical usefulness, comparison fairness, and experimental scale, and respectfully ask the reviewer to reconsider the rating.

Thank you for the clarification and additional experiments. The Ehrenfest example clarifies which reference dynamics TAM can support, and the matched DASBS comparison shows the practical benefit of the fiberwise training target. These results address my main concerns, and I am raising my score to 3.

We sincerely thank the reviewer for their thoughtful assessment, constructive feedback, and time devoted to reviewing our work. We greatly appreciate the reviewer's recognition of our contributions and willingness to raise the rating.

We are grateful for the reviewer's careful consideration and support.

Reviewer_oYqXOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies training a neural sampler for a given unnormalized probability distribution on a discrete domain. The proposed method is a generalization of the DASBS, enabling more general reference processes and a variance reduction technique for the training loss. The experiments on multiple benchmark distributions show promising results.

Critical strengths
  • The paper proposes a generalization of the DASBS method which are very novel from my understanding
  • Experiments on multiple high-dimensional discrete benchmark distributions show promising results
Critical weaknesses
  • The current writing seems to assume the readers are familiar with DASBS. Schrödinger bridge is a very math-heavy topic and lots of the important results and notations are required for general readers to understand the basic setup. I suggest the authors to polish the preliminary knowledge section to make it more accessible. It's OK not to present the proofs of this preliminary knowledge in the appendix, but at least should point to the references (e.g., DASBS paper). Also, the DASBS corrector-matching objective should be clearly explained in the main text because it's an important part of the algorithm
  • I feel many of the newly introduced notations are actually not quite necessary, for instance replacing p1∣tr(z∣x)p^r_{1|t}(z|x) with Kt,x(z)K_{t,x}(z) doesn't make the presentation simpler, and using hh instead of φ1\varphi_1 is totally unnecessary
  • If I understand correctly, like DASBS, the proposed method also requires the first-order oracle of the target distribution, i.e., x↦(ν(xd←n)ν(x))d∈[D],n∈[N]x\mapsto\left(\frac{\nu(x^{d\gets n})}{\nu(x)}\right)_{d\in[D],n\in[N]}. For complicated target distributions where the first-order oracle does not have a simple close form, this requires O(DN)O(DN) evaluations of the potential, which is quite expensive
Questions for authors
  • I don't quite get the motivation of 4.2 Fiberwise Rao-Blackwellization. This paragraph looks heavily AI generated and not well organized (also fiberwise seems to be an AI invented term that I've never seen in the literature). Without doing this kind of variance reduction, is it still possible to get a feasible training loss, or are we introducing this training objective simply because we don't want to parameterize the full [N]D×[N]D[N]^D \times [N]^D transport matrix (i.e., restricting transitions to only Hamming distance 1)? Also, is similar technique possible for DASBS?
  • Does the choice of anchor symbol j0j_0 matter if the target distribution is not symmetric? I hope the authors can provide some ablation experiments on this
  • One of the reasons DASBS chose the uniform reference process is for the required additive reference endpoint kernel structure, and the authors also agreed with this. I'm wondering, under the current more general setting, is it possible to use a masking / absorbing reference process? Also, is a similar construction possible for the continuous ASBS?
  • Minor comments: (1) the reference of the adjoint matching paper should be (Domingo-Enrich et al., 2025) instead of (Domingo i Enrich et al., 2025). (2) The font size in table 4 is too small, you should make ±\pm standard deviation below the mean value. (3) AI usage statement should be before the references
4 comments
AuthorsOpenReview ↗

Response to Reviewer oYqX (Part I: Method and Theory)

We thank the reviewer for the positive assessment of our work's novelty and experimental results, and for the constructive suggestions regarding presentation, methodological motivation, and applicability. In this first part, we address the preliminary material, the Corrector Matching Objective, and the motivation and scope of Fiberwise Rao--Blackwellization.

1. Preliminaries, Notation, and Corrector Matching Objective

We reorganized Section 3 to make the basic framework accessible without assuming prior familiarity with DASBS. The revised preliminaries start from the discrete Schrödinger Bridge problem and introduce the Doob transform, Schrödinger potentials, reference endpoint distributions, and optimal conditional endpoint distributions. We then derive the cyclic-shift adjoint identity used by DASBS. The relevant background is supported by references to Léonard (2014) and Guo et al. (2026b), making the origins of the foundational results explicit.

We also added a detailed explanation of the DASBS Corrector Matching Objective in Section 4.3. The Corrector estimates the ratio of the second Schrödinger potential and is trained through a Bregman regression objective. In the Denoising Matching formulation, the regression target is the reference endpoint-kernel ratio Kt,Xt(z′)/Kt,Xt(z)K_{t,X_t}(z')/K_{t,X_t}(z), which is used to train the inverse Corrector ratio. For full-support uniform reference processes, DASBS also adopts Adjoint Matching, regressing the inverse Corrector ratio against a stop-gradient Controller prediction. For non-additive reference dynamics, we directly use the Denoising Matching objective based on the reference endpoint-kernel ratio, without relying on the cyclic-shift construction. Algorithm 1 now explicitly presents the corresponding alternating Controller and Corrector updates.

Regarding notation, we retain Kt,xK_{t,x} and hh as concise notations for the reference endpoint kernel and terminal Schrödinger potential, respectively, and explicitly define them at their first appearance in Section 3.1. These quantities recur throughout the subsequent derivations, and the notation helps maintain concise and consistent expressions.

2. Motivation and Scope of Fiberwise Rao--Blackwellization

It is important to distinguish the general transport adjoint identity from its efficient local implementation.

First, Theorem 4.1 establishes that any Markov coupling satisfying the transport-compatibility condition yields a population-unbiased Controller Target. Therefore, Rao--Blackwellization is not necessary for the validity of the training target: one can directly use a regression target sampled from a compatible coupling. Rather, Rao--Blackwellization further improves its statistical efficiency.

For high-dimensional discrete systems, explicitly representing or sampling a general transport coupling may involve a dense, high-dimensional transition kernel. For coordinate-local reference dynamics whose endpoint kernels share the same marginal over unchanged coordinates, we can fix these unchanged coordinates and perform conditional marginalization only over the corresponding local state set. This eliminates the need to construct an explicit transport matrix and reduces the target-evaluation complexity to O(N)O(N), where NN is the local alphabet size.

Here, a fiber refers to the preimage of the coordinate projection z↦z−dz\mapsto z_{-d}. That is, after fixing the remaining coordinates aa, the corresponding local state set is {(a,j):j∈[N]}\{(a,j):j\in[N]\}. Thus, Fiberwise Rao--Blackwellization describes conditional marginalization within a fixed context, rather than introducing an additional state-space structure.

We also clarify the distinct roles of the two marginalizations in Section 4.2. The first eliminates the randomness introduced by the auxiliary transport symbol J′J', while the second further averages over the local endpoint symbol JJ, leaving only the randomness associated with the context AA. The resulting variance relation is established in Eq. (15).

The same construction also applies to DASBS. Its additive reference endpoint kernel satisfies the shared-fiber-marginal condition, so the same conditional marginalization is applicable. For the deterministic cyclic-shift coupling in DASBS, the first marginalization is degenerate because the transport symbol is uniquely determined by the original symbol; the second marginalization over the local endpoint symbol remains effective. The revised Section 4.2 also establishes the conditional-expectation relationship between the Fiberwise Target and the general Denoising Matching Target under the shared-context condition.

AuthorsOpenReview ↗

Response to Reviewer oYqX (Part II: Computational Cost, Experiments, and Extensions)

We continue with the computational-cost clarification, the asymmetric-target anchor ablation, and the discussion of more general reference dynamics and other revisions.

3. Computational Cost of the First-Order Oracle

We further clarify the first-order oracle required by TAM in Section 4.3. As in DASBS, TAM requires evaluating target-energy differences under local state changes. For a single coordinate-local transition, the target requires O(N)O(N) local energy differences and Corrector ratios. Computing all single-coordinate energy ratios for a given state may require up to O(DN)O(DN) potential evaluations in the worst case when no efficient local-update structure is available.

For the Ising and Potts models studied in this paper, these energy differences can be computed directly from local interactions, without reevaluating the energy of the entire system each time. Thus, TAM retains the target-query requirement of DASBS while avoiding the computation and storage of an explicit transport matrix through its local target construction.

4. Anchor-Symbol Ablation under an Asymmetric Target

We added an anchor ablation on an asymmetric four-state Potts model in Appendix Table 9. The experiment uses external-field coefficients (0,0.3,0.7,1.2)(0,0.3,0.7,1.2) and a nonuniform reference distribution ρ=(0.1,0.2,0.3,0.4)\rho=(0.1,0.2,0.3,0.4). We test four fixed anchors, j0∈{0,1,2,3}j_0\in\{0,1,2,3\}, while keeping the network architecture, reference dynamics, and training budget unchanged.

All four anchors achieve low sampling errors. The energy Wasserstein distances range from 0.39170.3917 to 0.52240.5224, the magnetization errors are all below 6.3×10−46.3\times10^{-4}, and the correlation errors are all below 1.23×10−31.23\times10^{-3}. We also numerically verify the exact targets: the maximum difference between targets obtained with different anchors is only 4.44×10−164.44\times10^{-16}, consistent with the exact anchor-invariance result established in Appendix Section C.1.

These results show that the anchor choice does not change the theoretical target, while all four choices maintain good sampling performance with learned Correctors.

5. Masking/Absorbing References and Continuous ASBS

We further discuss the applicability of TAM to more general reference dynamics in Appendix Section C.3.

For independent-coordinate masking/absorbing references that introduce a mask symbol into an expanded alphabet, the endpoint kernel retains a coordinate-factorized structure and therefore automatically satisfies the shared-context-marginal condition. When the support of the reference bridge is compatible with the prescribed target and the positivity condition on the terminal potential required by Theorem 4.1 holds, the Fiberwise Target can be constructed directly from local endpoint probabilities.

For continuous ASBS, the transport-compatibility identity can similarly be extended to continuous endpoint distributions by replacing discrete sums with integrals. Further differentiating the potential ratio with respect to the initial state establishes a connection to Score-based Control. Conditional Rao--Blackwellization can also be considered when the relevant conditional expectations are tractable. We present this as a theoretical extension and a direction for constructing continuous counterparts.

6. Other Revisions

We have also made the following changes:

  • Corrected the author name in the Adjoint Matching reference to Domingo-Enrich et al. (2025).

  • The original Table 4 mentioned by the reviewer corresponds to Table 5 (Controller-Target Comparison) in the revised manuscript. We adjusted its font size and layout to improve readability.

  • Moved the AI Use Statement to before the references, without changing its content.

7. Summary

In summary, following the reviewer's suggestions, we have systematically improved the theoretical exposition, methodological explanations, and experimental validation. The revised manuscript reduces the reliance on prior familiarity with DASBS and provides a clearer explanation of the theoretical motivation and statistical role of Fiberwise Rao--Blackwellization, as well as its relationship to DASBS. Meanwhile, the new asymmetric-anchor ablation and the theoretical discussion of more general reference dynamics further strengthen the evidence for the applicability of our method.

We again thank the reviewer for recognizing the novelty and experimental potential of our work, as well as for providing specific and valuable suggestions. We hope that these revisions adequately address the concerns raised and respectfully ask the reviewer to reconsider the rating. We sincerely appreciate the time and effort devoted to reviewing our work.

I thank the authors for the detailed reply and feel all concerns have been addressed. The presentation has been greatly improved. I'll keep my positive assessment to the paper.

We sincerely thank the reviewer for the encouraging feedback and for acknowledging that all concerns have been addressed. We are pleased that the revisions have substantially improved the presentation of the paper.

Given the theoretical clarifications and additional experimental evidence provided in the revised manuscript, we would be grateful if the reviewer would consider whether these improvements warrant a higher rating.

We greatly appreciate the reviewer's time and constructive feedback.

Reviewer_MnGmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies neural samplers for unnormalized distributions on discrete space [N]D[N]^D based on continuous-time Markov chains (CTMCs). It works within the discrete Schrödinger bridge (SB) framework of the discrete adjoint Schrödinger bridge sampler (DASBS; Guo et al., 2026). DASBS learns the optimal CTMC rates ut⋆(y,x)=rt(y,x)φt(y)/φt(x)u^\star_t(y,x)=r_t(y,x)\varphi_t(y)/\varphi_t(x) by alternating between a controller and a corrector. Its adjoint-matching controller target relies on a deterministic cyclic translation that maps the reference endpoint kernel of a state xx onto that of a neighboring state yy. This requires additive reference kernels.

The proposed Transport Adjoint Matching (TAM) replaces this translation by an arbitrary Markov kernel that transports one endpoint kernel into the other (“transport-compatible coupling”, Thm. 4.1). For coordinate-local references satisfying a shared-fiber marginal condition, it then analytically marginalizes the auxiliary transport variable and the endpoint symbol of the updated coordinate (Sec. 4.2). This yields a “fiberwise Rao–Blackwellized” controller target that can be computed locally. The rest of the pipeline (corrector objective, alternating updates, architecture, evaluation protocol) follows DASBS.

Experiments use 24×2424\times 24 Ising and 16×1616\times 16 four-state Potts models at three temperatures. They report the magnetization error (ΔMag), the two-point correlation error (ΔCorr) and the energy Wasserstein-2 distance (EW$_2$), and show improvements over DASBS and other discrete neural samplers. They also include an ablation of the marginalization steps at criticality and a run with a non-uniform (“non-additive”) refresh reference on critical Potts.

Critical strengths
  1. The paper poses a clear question (whether the DASBS controller construction intrinsically requires a cyclic translation) and gives a simple and correct answer. The identities in Thms. 4.1–4.2 and the variance ordering (15) are easy to verify.
  2. The fiberwise target is practical. It is a drop-in replacement for the DASBS controller target, costs O(N)O(N) per transition without constructing transport matrices, and is anchor-invariant if the corrector is exact.
  3. The results on the standard Ising and Potts benchmarks are strong, including the critical regimes, at a small training cost (0.1–0.3 h on two consumer GPUs). Fig. 1 complements the scalar metrics with distributional comparisons.
  4. The experiments test both main claims of the paper. The ablation of the marginalization steps (Table 4, Fig. 2) tests the claimed benefit of the fiberwise Rao–Blackwellization. The non-additive experiment (Table 5) tests the claimed applicability beyond additive references.
Critical weaknesses
  1. Contribution relative to DASBS and its presentation. As far as I can see, TAM targets the same discrete SB formulation as DASBS and keeps its controller-corrector parameterization, corrector objectives, alternating scheme, architecture and evaluation protocol. The only methodological change seems to be the controller regression target, plus a different reference in Sec. 5.5.

    • The "transport" in the name seems to refer to an auxiliary coupling between the endpoint kernels Kt,xK_{t,x} and Kt,yK_{t,y}, used to derive this target, rather than to a different transport problem.
    • The paper mentions the relation to DASBS (e.g., in Secs. 2, 4.3 and 5.1, and in Table 1), but only in passing. The abstract and introduction do not make clear how much of DASBS is retained, so the paper reads like a new method. This should be stated explicitly and early.
    • The DASBS corrector objective, on which TAM relies (Algorithm 1, line 4), should also be stated.
  2. Unclear sense of "generalization".

    • (a) DASBS already provides a denoising-matching controller identity that holds for general references, φt(y)/φt(x)=Ep1∣t⋆(x1∣x)[Kt,y(x1)/Kt,x(x1)]\varphi_t(y)/\varphi_t(x)=\mathbb E_{p^\star_{1|t}(x_1|x)}\big[K_{t,y}(x_1)/K_{t,x}(x_1)\big]. Under the shared-fiber condition, this ratio equals qJ(A)/pJ(A)q_J(A)/p_J(A) for x1=(A,J)x_1=(A,J). Since P(J=j∣A=a)∝pj(a)h(a,j)\mathbb P(J=j\mid A=a)\propto p_j(a)h(a,j) under the optimal law, its conditional expectation given AA is Hq(A)/Hp(A)=Rtx→y(A)H_q(A)/H_p(A)=R^{x\to y}_t(A) (for pj>0p_j>0). The fiberwise target thus seems to be obtainable directly from the DASBS denoising identity, without the transport-coupling construction.
    • (b) On the additive benchmarks (Tables 2-4), eq. (14) with the deterministic shift as coupling shows that the fiberwise target is the conditional expectation (under the optimal law) of the DASBS cyclic-shift target given the context. There, the comparison with DASBS therefore tests the Rao-Blackwellization rather than the generalization.
    • (c) For non-additive references, the practical construction covers references satisfying the shared-fiber condition (correlated references are left to future work), and only the controller. At high temperature, the paper still uses DASBS's adjoint-matching corrector, which relies on the cyclic structure.
  3. Role of the Rao-Blackwellization. Under the optimal law (with exact hh), R^C\widehat R_C, TCT_C and RR have the same conditional mean. The marginalization therefore seems to be a variance-reduction device rather than a requirement: R^C\widehat R_C with the independent coupling already applies to factorized references. The two steps also differ in nature, which the presentation does not make clear:

    • Averaging over J′J' is an exact Rao-Blackwellization under any sampling law, but it only removes noise introduced by the coupling; DASBS's deterministic shift has no such noise.
    • Averaging over JJ uses the hh-tilted conditional (with hh estimated via the learned corrector) instead of the conditional law of JJ under the training distribution. In general, it is therefore an exact Rao-Blackwellization only when the current process is optimal.

    Empirically, this second step gives no significant change on critical Ising despite the further variance reduction in Fig. 2 (Table 4: EW$_2$ 2.42±0.21 vs. 2.36±0.12), and a modest improvement on critical Potts (EW$_2$ 1.36±0.29 vs. 1.04±0.04). The natural baseline without marginalization, DASBS's cyclic-shift target in the same pipeline, is missing from Table 4.

  4. Fixed-point iteration and convergence. Thms. 4.1-4.2 hold under the optimal endpoint law, whereas in (18) the endpoint pairs come from the current controlled process and XtX_t from the reference bridge. The theorems thus seem to show only that the optimal control is a fixed point. DASBS, following the adjoint Schrödinger bridge sampler (ASBS; Liu et al., 2025), gives an iterative proportional fitting (IPF) argument for convergence of the idealized alternating scheme. That argument relies on a unique fixed point of the controller objective. The paper neither restates the argument for the new target nor discusses uniqueness of the fixed point.

  5. Experimental comparison with DASBS.

    • (a) The rows for LEAPS, the masked diffusion neural sampler (MDNS) and its uniform-diffusion variant (UDNS), the discrete neural flow sampler (DNFS) and Metropolis-Hastings (MH) are identical to those reported in the DASBS paper, but the DASBS row is not. For example, on critical Ising it reports ΔMag 5.7e-1 and EW$_2$ 53.7, whereas the DASBS paper reports 5.7e-2 and 12.1, and it lists 3.0k instead of 3.75k training steps. Please state whether DASBS was rerun, and with which settings, budget and seeds.
    • (b) TAM's hyperparameters also differ from those of DASBS: γ=0.25 instead of γ=1 at low temperature, fewer controller/corrector steps per stage for high/critical Ising and critical Potts, different buffer sizes and refresh intervals for critical/low Ising, and different hardware. Depending on the rerun settings, Tables 2-3 may change the target and the hyperparameters at the same time.
    • (c) R^C\widehat R_C in Table 4 differs from the DASBS cyclic-shift target only in the coupling (independent instead of deterministic). Yet it reaches EW$_2$ 3.37 on critical Ising and 1.68 on critical Potts, far better than the DASBS rows (53.7, 14.6) and the published DASBS values (12.1, 20.0). This should be explained, since it determines how much of the gain can be attributed to the proposed target.
    • (d) In Table 2, training budgets (3k vs. 30-50k steps) and hardware differ across methods, so the runtimes are not directly comparable. The highlighting also covers only uniform-based neural samplers, although MH or MDNS are better on several metrics. For example, on high-temperature Ising, MH reaches ΔMag 8.9e-4 and ΔCorr 2.9e-4, vs. 3.4e-3 and 1.0e-3 for TAM.
    • (e) For critical Ising, the ΔMag of TAM is 1.47e-2 in Table 4 but 6.8e-3 in Tables 2 and 8. With the Table 4 value, MDNS (1.1e-2) would be better on this metric, and the unqualified claim in Sec. 5.2 of best results on all three metrics at criticality would not hold.
  6. Limited experimental scope. The evaluation uses only the two benchmark families of DASBS (Ising with N=2N=2, D=576D=576; four-state Potts with D=256D=256). This makes it hard to judge how challenging the tasks are and how TAM scales, e.g., to larger alphabets. The motivating non-additive case is tested in a single setting (critical Potts with a non-uniform refresh distribution), without a competing baseline, and performs slightly worse than the additive reference (Table 5). A more convincing test would use a setting where non-additive references are natural, e.g. ordinal alphabets with birth-death references such as the Ehrenfest process (Winkler et al., 2024), and compare against DASBS's denoising-matching controller objective, which applies to general references.

  7. Related work. Citations are sparse. Relevant works that are not cited include:

    • Campbell et al., "A continuous time framework for discrete denoising models" (NeurIPS 2022);
    • Lou, Meng & Ermon, "Discrete diffusion modeling by estimating the ratios of the data distribution" (ICML 2024);
    • Kholkin, Vargas & Korotin, "Sampling from energy distributions with target concrete score identity" (arXiv:2510.23106, 2025);
    • Sanokowski et al., "Scalable discrete diffusion samplers: combinatorial optimization and statistical physics" (ICLR 2025);
    • Ksenofontov & Korotin, "Categorical Schrödinger bridge matching" (ICML 2025);
    • Winkler, Richter & Opper, "Bridging discrete and continuous state spaces: exploring the Ehrenfest process in time-continuous diffusion models" (ICML 2024), whose birth-death process is a natural non-additive reference and is named as future work in DASBS;
    • further diffusion-based samplers for the continuous case, e.g., Richter & Berner, "Improved sampling via learned diffusions" (ICLR 2024), and Vargas et al., "Transport meets variational inference: controlled Monte Carlo diffusions" (ICLR 2024).

    The relation to the bridge matching sampler (Blessing et al., 2026), which recently generalized adjoint-based methods for sampling, should also be discussed. Finally, the sentence "Adjoint matching turns their optimality conditions into tractable regression objectives" may suggest that earlier approaches lacked tractable objectives, which is not the case.

  8. Presentation (minor).

    • The colored equations and background colors are distracting and look unprofessional.
    • More intuition during the derivations would help; Sec. 4.2 is hard to read.
    • Some shorthand notation is confusing. In (9), the right-hand side depends on tt, while this dependence is suppressed in the other terms. Kx(a,j)K_x(a,j) with two arguments is only defined implicitly via z=(a,j)z=(a,j).
    • I would call the Swendsen-Wang samples "reference" rather than "ground truth", since they are also an approximation.
    • The subsections in Sec. 5 are quite short; maybe use paragraph headings instead?
Questions for authors
  1. Is it correct that TAM is DASBS with a different controller regression target (plus a different reference in Sec. 5.5)? If not, what else differs?
  2. In what sense does TAM generalize DASBS?
    • Is the fiberwise target the conditional expectation of the DASBS denoising-matching controller target given the context? If so, is the transport-coupling construction needed?
    • Is the shared-fiber condition needed for the identity in Thm. 4.2, or only for computational convenience?
    • How would the corrector be trained for a non-additive reference at high temperature, where the paper uses DASBS's adjoint-matching corrector, which relies on the cyclic structure?
  3. Is the Rao-Blackwellization essential, or mainly a variance-reduction device? Please add the DASBS cyclic-shift target, trained in the same pipeline, to Table 4, report the ablation for all temperatures and for the non-additive reference, and show training curves.
  4. Were the DASBS results in Tables 2-3 rerun? If so, with which hyperparameters, training budget and seeds? Why do they differ from the DASBS paper (e.g., critical Ising EW$_2$ 53.7 vs. 12.1), and why is DASBS so much worse at criticality than at low temperature in Table 2 (EW$_2$ 53.7 vs. 2.5)? Can you compare both methods under identical settings on all six tasks?
  5. Does the alternating fixed-point iteration converge? Does the IPF argument of DASBS carry over to the new target, in particular the uniqueness of the fixed point of the controller objective?
  6. Why does the critical-Ising ΔMag of TAM differ between Table 4 (1.47e-2) and Tables 2 and 8 (6.8e-3)?
  7. Were the runtimes in Table 2 measured on the same hardware? The baseline values coincide with those in the DASBS paper, where they were measured on a single A6000, whereas TAM was trained on two RTX 4080 GPUs. Why is the highlighting restricted to uniform-based neural samplers, given that MH or MDNS are better on several metrics?
  8. Which practical applications motivate non-additive references, and how does TAM perform there compared with the DASBS denoising-matching controller objective?
4 comments
AuthorsOpenReview ↗

Response to Reviewer MnGm (Part I: Method and Theory)

We thank the reviewer for the positive assessment of Theorems 4.1–4.2 and Eq. (15). We clarify TAM's relationship to DASBS and the two Rao--Blackwellization steps.

1. Relationship to DASBS

TAM retains the discrete Schrödinger Bridge objective and Controller--Corrector framework of DASBS, while generalizing its Controller Target. DASBS derives its adjoint identity from the deterministic cyclic-shift correspondence of additive reference endpoint kernels. TAM replaces this requirement with the transport-compatibility condition

KxCtx→y=Ky,K_x C_t^{x\to y}=K_y,

where Ctx→yC_t^{x\to y} is a Markov coupling between endpoint kernels. Any coupling satisfying this marginal constraint yields the exact population-level adjoint identity; the deterministic cyclic shift of DASBS is a special case.

For coordinate-local dynamics, the shared-fiber condition permits analytical marginalization of the transport variable and local endpoint symbol. The resulting coupling-free Fiberwise Target uses local endpoint probabilities, energy differences, and Corrector-estimated potential ratios, without an explicit transport matrix.

2. Relationship to Denoising Matching

Under the shared-fiber condition and corresponding support assumptions, the local endpoint-kernel ratio is

Ky(a,j)Kx(a,j)=qj(a)pj(a).\frac{K_y(a,j)}{K_x(a,j)}=\frac{q_j(a)}{p_j(a)}.

Under the optimal endpoint distribution, its conditional expectation given A=aA=a is

Rtx→y(a)=∑jqj(a)h(a,j)∑jpj(a)h(a,j),R_t^{x\to y}(a) =\frac{\sum_j q_j(a)h(a,j)}{\sum_j p_j(a)h(a,j)},

which is precisely the Fiberwise Target.

This relation is consistent with Denoising Matching. TAM provides a unified transport-compatible adjoint formulation: Theorem 4.1 establishes the exact population identity for general compatible Markov couplings, while Theorem 4.2 derives the coupling-free local form under the shared-fiber condition. The latter enables an O(N)O(N) local implementation without explicitly constructing a transport matrix.

3. The Two Rao--Blackwellization Steps

The two marginalizations remove distinct sources of randomness. First, conditioning on (A,J)(A,J) eliminates the auxiliary transport variable J′J':

TC(A,J)=E[R^C(A,J,J′)∣A,J].T_C(A,J)=\mathbb E[\widehat R_C(A,J,J')\mid A,J].

For a deterministic cyclic shift, J′J' is determined by JJ, so this step is degenerate.

Second, conditioning on AA eliminates the local endpoint symbol JJ:

Rtx→y(A)=E[TC(A,J)∣A],P(J=j∣A=a)=pj(a)h(a,j)Hp(a).R_t^{x\to y}(A)=\mathbb E[T_C(A,J)\mid A], \quad \mathbb P(J=j\mid A=a)=\frac{p_j(a)h(a,j)}{H_p(a)}.

Under this optimal endpoint conditional law,

Var⁡[Rtx→y(A)]≤Var⁡[TC(A,J)]≤Var⁡[R^C(A,J,J′)].\operatorname{Var}[R_t^{x\to y}(A)] \leq \operatorname{Var}[T_C(A,J)] \leq \operatorname{Var}[\widehat R_C(A,J,J')].

Figure 2 evaluates the three targets under the corresponding hh-tilted endpoint law and same context distribution, providing an empirical diagnostic consistent with Eq. (15). Practical training uses samples from the current controlled process and learned Corrector, distinct from the optimal endpoint law underlying the variance ordering.

4. Corrector Objectives and Non-additive References

TAM retains the DASBS Controller--Corrector framework and selects the Corrector objective according to the reference. For standard high-temperature tasks, we use the product-uniform source and Adjoint Matching Corrector; for critical- and low-temperature tasks, we use the corresponding zero-temperature source and Denoising Matching Corrector.

For the non-additive reference experiment, we use a Denoising Matching Corrector based directly on the reference endpoint-kernel ratio, without cyclic-shift correspondence. Thus, TAM supports non-additive references in both its Controller construction and Corrector objective.

5. Fixed-Point Uniqueness and Alternating Convergence

We distinguish uniqueness of the population Controller objective from uniqueness of the full coupled Controller--Corrector fixed point. The appendix's Bregman regression analysis establishes a unique minimizer of the population Controller objective under the optimal endpoint-context distribution. Together with Theorems 4.1–4.2, this characterizes the optimal control objective.

These results do not by themselves establish global uniqueness of the coupled fixed point or convergence of practical alternating updates. DASBS's IPF argument relies on idealized updates corresponding to forward and backward Schrödinger half-bridge projections. TAM's practical updates use samples from the current controlled process and a learned Corrector; convergence therefore cannot be inferred from the adjoint identity alone.

AuthorsOpenReview ↗

# Response to Reviewer MnGm (Part II: Experiments)

We thank the reviewer for the questions about experimental fairness, baselines, and TAM's scope. We verified the published DASBS version and added matched reproductions, Controller Target ablations, large-scale tests, and non-additive reference experiments.

1. DASBS Version and Matched Reproduction

Tables 2–3 use the official ICML 2026 version of DASBS (PMLR 306), whereas the comparison you consulted used the earlier arXiv v1. For critical Ising, the earlier version reports ΔMag.=5.7×10−2\Delta\mathrm{Mag.}=5.7\times10^{-2} and EW2 =12.1=12.1, versus 5.7×10−15.7\times10^{-1} and 53.753.7 in the ICML version. Critical-Potts EW2 changed from 20.0 to 14.6. Published Ising training steps (high/critical/low) are 3.0k/3.0k/3.75k.

We also reran the original DASBS cyclic-shift Controller on all six standard tasks, using the same environment, architecture, configurations, and three seeds as TAM.

2. Hyperparameters and Computational Budget

TAM Appendix Table 7 and Table 4 of the published DASBS paper have consistent main configurations across the six tasks, including GPU count, batch size, buffer size, Controller/Corrector updates, refresh interval, sampling steps, and learning rate. Both use γ=1\gamma=1 for high/critical temperatures and γ=0.25\gamma=0.25 for low temperature.

Published DASBS runtimes were measured on two RTX A6000 GPUs, while TAM used two RTX 4080 GPUs. We therefore use same-environment reproductions and large-scale experiments, rather than cross-hardware runtimes, for direct efficiency comparisons.

3. Critical- versus Low-Temperature DASBS Results

Matched reproductions show different DASBS performance across temperatures:

  • Ising EW2: 54.677±54.35054.677\pm54.350 (critical) vs. 2.499±0.1992.499\pm0.199 (low).
  • Potts EW2: 16.939±9.75116.939\pm9.751 (critical) vs. 1.124±0.4971.124\pm0.497 (low).

We attribute this mainly to target structure: criticality involves long-range correlations and fluctuations, whereas low-temperature distributions concentrate around ordered phases, closer to the zero-temperature source. The published DASBS setup also uses a lower noise rate at low temperature.

4. Controller Target Ablation and Seed Consistency

The revision compares Sampled Transport, Transport-Marginalized, Fiberwise TAM, and DASBS Cyclic-Shift. Their EW2 values on critical Ising are 3.4±0.23.4\pm0.2, 2.4±0.22.4\pm0.2, 2.3±0.12.3\pm0.1, and 54.68±54.3554.68\pm54.35; on critical Potts, they are 1.7±0.71.7\pm0.7, 1.4±0.31.4\pm0.3, 1.0±0.041.0\pm0.04, and 16.94±9.7516.94\pm9.75.

The first marginalization improves all three metrics on both tasks. Further Fiberwise marginalization gives Ising the lowest magnetization error and EW2, and Potts the best results on all three metrics.

The critical-Ising magnetization discrepancy between the original Table 4 and Tables 2 and 8 came from different seed sets. We unified the seeds and updated the results; Fiberwise TAM is now consistently (6.8±1.8)×10−3(6.8\pm1.8)\times10^{-3}.

5. Large-Scale and Non-additive Experiments

We added critical Ising (L=48L=48) and eight-state Potts (N=8N=8). Under the same architecture and budget, TAM reduces EW2 from 727.92±97.13727.92\pm97.13 to 12.45±1.5812.45\pm1.58 on Ising, and from 31.42±13.4731.42\pm13.47 to 2.45±1.012.45\pm1.01 on Potts; all three errors improve. The training times of TAM and DASBS are 41.56 ± 0.12 and 44.04 ± 0.06 minutes on Ising L48, and 7.79 ± 0.02 and 7.82 ± 0.06 minutes on Potts q8, respectively. Five-stage curves show lower TAM errors throughout training.

The alphabet-size study extends to N=16N=16. For N=2,4,8,16N=2,4,8,16, Fiberwise TAM reduces variance over DASBS cyclic shift by 87.98%–88.95% at t=0.2t=0.2 and 47.09%–51.86% at t=0.8t=0.8, with target construction taking 0.109–0.113 ms.

For non-additive four-state Potts, we use ρ=(0.1,0.2,0.3,0.4)\rho=(0.1,0.2,0.3,0.4) and compare with DASBS Denoising Matching under the same reference. TAM achieves EW2 1.4±0.21.4\pm0.2, versus 220.79±1.64220.79\pm1.64 for DASBS-DM, with lower magnetization and correlation errors. Thus, TAM constructs effective targets from local endpoint probabilities when cyclic shifts do not apply.

We also add a 64-site, five-state ordinal chain with an Ehrenfest birth–death reference. Under the same dynamics, architecture, and budget, TAM achieves EW2 0.34±0.020.34\pm0.02, versus 24.58±0.3024.58\pm0.30 for DASBS-DM, improving all three metrics.

6. Other Baselines and Related Work

Best/second-best markers in Tables 2–3 apply only to uniform-based discrete neural samplers; MH and MDNS remain included as other sampler categories. We do not imply global superiority across all methods and metrics.

We added the related works suggested by the reviewer and clarified the relation between Adjoint Matching and existing tractable objectives.

Summary

The revised experiments provide matched DASBS comparisons, demonstrate TAM's large-scale advantages, and establish its applicability to non-additive references. We thank the reviewer for the suggestions and hope these additions address the concerns about fairness, generality, and practical effectiveness. We respectfully invite the reviewer to reconsider the current rating.

Dear Authors,

Thank you for the detailed rebuttal and the extensive updates to your paper. I appreciate the significant effort involved. Apologies for not being able to finish reviewing all of the new material before the author-reviewer discussion period closes. Rest assured, I will go through your revisions carefully and take them into full account when finalizing my rating and discussing the submission with the other reviewers and the AC.

We sincerely thank the reviewer for the thoughtful response and for acknowledging the extensive revisions and efforts we have made. We greatly appreciate the reviewer's willingness to carefully consider our updated manuscript and responses in the final assessment.

Thank you again for your time, careful consideration, and constructive feedback.

Comments on the paper

1 comment
Area_Chair_L87UOpenReview ↗

Author-Reviewer Discussion ends in 1 day, 12 hours

Dear authors and reviewers,

Thank you to the authors for their detailed responses of Nov 15 and the updated manuscript. The discussion period closes in about a day and a half, so this is the last chance for a direct exchange before the reviewer-AC discussion starts on Nov 19.

Reviewers: I haven’t yet seen reply from any of you. Please read the authors’ responses and the revision, which address several of your points directly. Please let us know whether your concerns have been addressed, and update your recommendation if your assessment has changed. Even a short acknowledgment helps.

Authors: please be ready to answer any follow-up questions in the time left, focusing on the points most likely to affect the decision.

Thank you all for keeping the discussion constructive and timely.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.