- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Tilted Schrödinger Bridge Matching (TSBM), a method for reward adaptation of pretrained Schrödinger Bridges. The goal is to adjust the target marginal from to while preserving the source marginal . The authors use the pretrained bridge as the reference process and derive a relative formulation based on the new and original Schrödinger potentials, yielding a control objective that does not require explicitly evaluating the target marginal density or score during post-training. They further develop an algorithm that alternates between controller and corrector updates. Experiments on Colored MNIST and CelebA demonstrate the effectiveness of TSBM in reward steering while preserving source attributes.
Critical strengths
- A key strength of this work is its novel perspective on global marginal reward tilting for pretrained Schrödinger Bridges with non-memoryless source distributions. Built on Adjoint Matching and alternating controller–corrector updates, the formulation avoids explicitly evaluating the original target density or its score, providing a distinctive and well-motivated approach to this problem.
Critical weaknesses
- Insufficient training-based baselines. TSBM requires additional post-training, while the main experiments primarily compare it with training-free methods, making it difficult to assess its advantages over other post-training approaches. Although the authors note that suitable post-training methods for SBs are currently lacking, comparisons with Flow/Diffusion post-training methods on the same benchmarks are still needed to adequately evaluate the incremental benefits and practical value of the proposed method.
Questions for authors
-
In Figure 3(c), DPS achieves a higher reward than TSBM. However, the qualitative results in Figure 5 suggest that TSBM produces digits with more consistently red colors. Does this discrepancy indicate that the current Red-Chroma reward may not fully capture the intended color transformation?
-
Given the similarities between their Controller–Corrector Matching frameworks, could the authors clarify the key innovations of TSBM?
-
For general reinforcement learning or reward optimization tasks, what advantages does TSBM offer over flow-based post-training methods?
3 comments
Rebuttal
3. Key innovations in Controller–Corrector Matching
Reviewer
Given the similarities between their Controller–Corrector Matching frameworks, could the authors clarify the key innovations of TSBM?
Answer
To our knowledge the prior Controller-Corrector Matching and related energy-based Schrödinger bridge frameworks [3, 4, 5] were primarily designed for sampling from Boltzmann distributions or bridging to an explicitly known energy prior. If the reviewer had some other work in mind, we kindly as him to share it. While TSBM shares the high-level alternating Controller-Corrector Matching template, adapting it to reward post-training introduces three key innovations in formulation, algorithm, and problem setting:
- Relative formulation for arbitrary marginals without known densities: Our core methodological innovation is the relative formulation for adapting a pretrained bridge between arbitrary endpoint marginals without known densities (Propositions 1-3). Taking the difference between the new and pretrained Schrödinger potentials cancels the unknown target density (whereas prior Controller-Corrector Matching frameworks assume an explicit unnormalized density or energy) and yields the relative terminal corrector , while preserving the arbitrary source marginal .
- General reference processes and full adjoint ODE computation: Prior Controller-Corrector Matching frameworks are restricted to simple analytical reference priors (such as memoryless or Brownian priors where the adjoint ODE simplifies analytically). By contrast, TSBM operates relative to a general learned reference bridge with nonlinear neural drift, requiring integration of the full backward lean adjoint ODE via vector-Jacobian products through the frozen drift network. In this respect, our algorithmic framework is strictly more general.
- Post-training for unpaired domain translation: Practically, prior Controller-Corrector Matching frameworks train a model from scratch against an explicitly evaluable energy function (e.g., Boltzmann sampling), whereas TSBM post-trains a pretrained Schrödinger bridge for unpaired domain translation where is only implicitly represented by the pretrained forward and backward drifts, resulting in a substantially different training pipeline.
In summary, TSBM's key innovations over prior Controller-Corrector Matching frameworks are (i) the potential-difference relative formulation that eliminates dependence on explicit endpoint densities, (ii) generalization of the adjoint matching step to arbitrary learned reference drifts via full backward ODE integration, and (iii) warm-started post-training of pretrained bridges for unpaired translation.
We hope that our clarifications and supporting experiments address your questions and reservations, and if so, we kindly ask you to consider updating your score accordingly. Please let us know if any questions remain.
References
[1] Domingo-Enrich et al., Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control, ICLR 2025.
[2] Clark et al., Directly Fine-Tuning Diffusion Models on Differentiable Rewards, ICLR 2024.
[3] Liu et al., Adjoint Schrödinger Bridge Sampler, NeurIPS, 2026.
[4] Tamogashev & Malkin, Data-to-Energy Stochastic Dynamics, ICLR 2026.
[5] Shin et al., Efficient Generative Modeling beyond Memoryless Diffusion via Adjoint Schrödinger Bridge Matching, ICML 2026.
Response
Thank you for the detailed response.
The authors' explanation regarding the Red-Chroma reward sounds reasonable, and reward-tilting also preserves the thickness of the original numbers.
Although the main framework is built upon ASBS, developing a reward post-training method for pretrained Schrödinger bridges is still a meaningful contribution.
However, my main concern regarding the lack of training-based baselines remains:
-
I could not find other training-based studies using the same experimental settings, making it difficult to assess how much TSBM improves over existing post-training methods. Comparisons with training-free methods alone are insufficient to answer this question.
-
I believe various flow-based post-training methods could also be applied to MNIST image translation and CelebA semantic attribute editing, since flow models can also be trained as data-to-data models. However, the authors do not provide sufficient comparisons, making it difficult to determine whether flow-based methods or TSBM are preferable for general reinforcement learning or reward optimization tasks.
Overall, I believe TSBM is an effective approach for Schrödinger bridges. However, the current experiments are insufficient to fully assess its advantages and practical applicability. I therefore still lean toward acceptance, but cannot give a stronger recommendation at this stage.
Rebuttal
We thank Reviewer AQcj for the thoughtful review and constructive feedback. Below, we address each of your comments and questions in detail. We have also uploaded a revised version of the manuscript with all changes highlighted in blue.
1. Comparison with Flow/Diffusion post-training
Reviewer
Answer
We thank the reviewer for raising this point. Existing flow and diffusion post-training methods do not directly apply to reward-tilting pretrained Schrödinger bridges applied for unpaired trasnfer:
To our knowledge, there are currently no fine-tuning methods, other than TSBM, for Schrödinger bridges that account for non-memorylessness, so we do not have other suitable post-training baselines beyond this static-corrector (regular Adjoint Matching) variant.
2. Red-Chroma reward
Reviewer
Answer
Indeed, in In Figure 3(c), DPS (γ = 2) achieves a higher reward than TSBM and produces digits with less red color. As one can see in the Appendix D.2 of the original submission, the red-chroma reward favors red color per each pixel, while blue and green channels are penalized, also per pixel. This reward doesn't provide any normalization per digit size, so full image consisting only of red pixel would maximize the reward. In that light, the DPS is maximizing the reward by rather raising the digit size than making it red, which can be see as a reward hacking. In the Figure3(c) we also plot the "thickness error" and one can see that DPS (γ = 2) has significantly thicker digits than TSBM, which allows it to make up for the noted red color loss.
As a result, one can see that TSBM provides better reward maximization-faithfullness trade off, since almost all the digits are red, while their thickness is closer to the original thickness.
Although, as the reviewer suggested this also can be seen as reward capturing not only the color transformation, but also the size of a digit.