acceptodds
Under review as a conference paper at ICLR 2027

There and Back Again: Bidirectional Diffusion Bridges for Multimodality Translation

12 views

Abstract

Multimodality translation (e.g., text-to-image) is a core generative AI task. However, existing approaches (1) have semantically meaningless generative paths, greatly limiting the flexibility of sampling algorithms; (2) are unidirectional, preventing inversion (e.g., image-to-text). We propose BIT: Bidirectional Image-Text Diffusion Bridges. In contrast to previous approaches, BIT starts directly from text and interpolates into images, providing (1) a semantically rich generative path that enables diverse and flexible sampling algorithms; (2) inherent reversibility of the model from image to text, providing a unified, bidirectional generative framework. BIT is derived through rigorous stochastic calculus, yielding SDE forms friendly to simulation and tractable loss functions that scale to high dimensions. Our theoretical and empirical results confirm competitiveness, and often advantage over denoising diffusion and deterministic flow model baselines; not only in vision-language, but also in natural science tasks.

Primary area:
generative models
Keywords:
diffusion bridge, diffusion language model, text-to-image, invertible generative model

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_xYk4OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes BIT, a bidirectional diffusion-bridge framework for text-to-image and image-to-text generation. Fixed continuous token embeddings and a lightweight decoder place text and image latents in a common representation space. The authors derive endpoint-conditioned forward and reverse SDEs with tractable training objectives, together with state-only drifts that match the bridge’s one-time marginals at the population level. Experiments evaluate conditional generation, cross-modal round-trip variations, and cell-fate modeling, with additional demonstrations of large-scale shared bidirectional modeling and qualitative image editing.

Critical strengths

The paper derives forward and reverse SDEs with tractable training objectives for text–image translation. Continuous token embeddings allow both modalities to use the same DiT backbone. The theoretical results clearly distinguish population-level guarantees from marginal matching and finite-step sampling.

The experiments show that starting from text representations can achieve T2I quality comparable to the controlled noise-to-data baseline, while reducing caption generative perplexity from 178.2 to 123.4 (Table 1). The billion-parameter shared model and the variation and editing examples further demonstrate that the approach works beyond the basic generation setting.

Critical weaknesses
  • Incomplete coverage and comparison of closely related bridge methods. The paper omits BBDM (CVPR 2023) and BiBBDM (TPAMI 2025). BBDM already studies direct domain-to-domain translation through Brownian bridges, and BiBBDM explicitly learns both translation directions with one model. The paper should discuss and compare these methods to clarify BIT’s specific advances in cross-modal modeling beyond direct data-to-data transport and bidirectionality.

  • The empirical tradeoff of shared bidirectional modeling remains unclear. The controlled results in Table 1 use separate models for the two directions The shared model in Table 2 also increases model size and training examples and changes several training settings. These results demonstrate feasibility but do not isolate the effect of joint training. A controlled shared-versus-separate comparison would clarify the quality–compute tradeoff and potential interference between the two tasks.

Questions for authors

Please address the weaknesses raised above, particularly the comparison with BBDM and BiBBDM and the empirical tradeoff between shared and separate directional models.

Reviewer_4qvBOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

Traditional text-to-image (T2I) models start from Gaussian noise and use a deterministic or conditional diffusion process to generate images. The authors argue that this noise-to-data paradigm discards valuable semantic information present in the text prompt during the early stages of generation. Instead, they propose BIT (Bidirectional Image-Text Diffusion Bridges), a unified framework that treats text and images as two endpoints of a data-to-data stochastic process. By directly interpolating from a continuous text representation to an image (and vice versa), the model retains source information along the entire generative trajectory, enabling high-fidelity generation, cross-modal inversion, and round-trip sampling.

The contributions are listed below:

  1. Paradigm Shift in T2I: Challenges the assumption that T2I must start from Gaussian noise. BIT starts directly from the text representation, creating a semantically richer path.
  2. Unified Bidirectional Framework: A single mathematical framework (based on diffusion bridges and Doob's h-transform) that handles both Text-to-Image (T2I) and Image-to-Text (I2T) generation, allowing reversible paths at any point in the trajectory.
  3. Discrete-Continuous Unification: Solves the problem of applying continuous SDEs to discrete text by creating a fixed, near-invertible lookup table of continuous text embeddings (using Qwen3-Embedding), matching the dimensionality of image latents.
  4. Theoretical Guarantees: Proves that the forward and reverse SDEs describe the same process in opposite time directions, and mathematically demonstrates that data-to-data bridges have a local mutual-information advantage over noise-to-data paths (Theorem 3).
  5. Novel Capabilities: Enables cross-modal stochastic variation (generating diverse but semantically faithful variations of an input) and image editing by inverting an image to text, editing the text, and generating a new image.
Critical strengths

Critical Strength 1: A clean, theoretically grounded reconceptualization of cross-modal generation as a data-to-data bridge, enabling capabilities that noise-to-data diffusion structurally cannot provide. This is the consideration that most strongly determines my recommendation for acceptance.

What it is: Rather than conditioning a noise-to-data score network on a text prompt (leaving the evolving state semantically empty), BIT treats text and image as two coupled endpoints of a single stochastic bridge. The forward and reverse SDEs are derived rigorously via finite-horizon h-transforms and analytic time-reversal (Theorems 1–2), and the paper proves the two directions describe the same process traversed in opposite time.

Why it is consequential: The reconceptualization is not cosmetic—it unlocks behaviors that are architecturally impossible for standard conditional diffusion:

(1) Cross-modal inversion (image → text → edited text → image) using the same inferred noise increments ("noise backslowing," Alg. 2), yielding structurally aligned edit spectrums (Fig. 5).

(2) Graceful degradation under corruption (Fig. 3): because the state carries source information, BIT retains semantic fidelity at high corruption fractions where noise-to-data diffusion collapses.

(3) Theoretical backing for the intuition (Theorem 3, strengthened to τ\tau=1 for positively correlated Gaussians in App. E).

Critical Strength 2: The discrete-to-continuous unification is a clean, reusable engineering recipe that makes the continuous SDE framework actually operational for text.

What it is: §4 and App. F construct context-independent token embeddings from Qwen3-Embedding, reshape a 64×64 token matrix row-major into the 4×32×32 image latent shape, and share a single DiT backbone across both modalities—with a shared MLP decoder achieving >99% token accuracy even under noisy embeddings.

Why it is consequential: The elegant continuous-space theory of §3 would be inert if text remained discrete. The paper's solution is not just a workaround—it is a design that preserves invertibility (fixed vocabulary lookup table), matches dimensionality exactly (no modality-specific projections that would confound comparisons), and keeps text order legible to the transformer via row-major packing. This is the bridge between the math and the working system, and the care taken here (e.g., explicitly avoiding contextual embeddings to keep the lookup table tractable) reflects real understanding of the failure modes. It also makes the framework transferable to other modalities beyond text-image (as the cell-fate experiments demonstrate), which broadens the paper's relevance.

Critical weaknesses

The headline comparison to the standard T2I paradigm is not actually established by the theory or by a controlled experiment — the central "paradigm shift" claim outruns the evidence.

What the paper claims: The abstract and §1 frame BIT as a "paradigm shift" that "challenges the assumption that T2I generation needs to start from a Gaussian noise distribution," and the intro implies the data-to-data path is superior to noise-to-data for T2I.

What the paper actually establishes:

(1) Theorem 3 compares a data-to-data bridge against a state-only noise-to-data bridge (Eq. 13). The paper itself states: "It does not compare against a standard conditional T2I sampler that retains the text prompt as side information throughout the trajectory, and with retained side information the theorem gives no ordering." Standard T2I models (SD, Flux) do retain the prompt as cross-attention side information. So the theorem does not support the headline claim against the standard paradigm — it supports a narrower statement about state-carried information in the round-trip setting.

(2) Appendix E strengthens this to τ\tau=1, but only for positively correlated jointly Gaussian scalar endpoints — a regime the paper admits is "unlikely" for real text-image data.

(3) The empirical comparison in the controlled ablation (Table 1, which is absent from the provided excerpt) is the only like-for-like test. The foundation-model comparison (Table 2) is explicitly not like-for-like ("models differ in training data, compute, architecture, and inference settings"), and BIT actually loses to Qwen-3-VL-4B on I2T and to SD 1.5 on CLIP T2I.

Why it is consequential: The paper's primary intellectual contribution is framed as overturning the dominant T2I assumption. If the theorem only covers the state-only / round-trip case, and the only controlled experiment is a parameter-matched ablation (not a comparison to real conditional T2I systems), then the central claim is not validated at the level the framing demands. This is not a minor scoping issue — it is the thesis of the paper. A reader who accepts the framing would believe the paper shows data-to-data beats standard conditional diffusion for T2I; the paper does not show that.

Questions for authors

Q1. What exactly does the controlled ablation (Table 1) show for T2I and I2T against a parameter-matched conditional diffusion baseline?

The paper's central "paradigm shift" claim rests on data-to-data beating noise-to-data. Theorem 3 only covers the state-only case. The controlled, like-for-like comparison is Table 1 — which is not included in the provided text. Specifically: is the noise-to-data baseline in Table 1 conditioned on the text prompt via cross-attention (i.e., a genuine standard conditional T2I sampler), or is it a state-only noise-to-data bridge? If it is the former, the empirical case is much stronger than Theorem 3 alone suggests, and my Weakness 1 narrows considerably. If it is the latter, the paper has no controlled comparison against the standard paradigm at all, and I would raise the severity of that concern.

Q2. Does the round-trip / variation capability still work under the analytically justified drift (i.e., without CFG), and how much does it degrade?

Table 2 shows CFG materially improves FID (4.39 → 2.54). Figure 3 and the image-editing results (Alg. 2, Fig. 5) are run with CFG scale zero per App. M — good. But can you confirm that the Fig. 3 fidelity/diversity curves and the editing pipeline are all ω\omega=0, i.e., inside the theoretically analyzed process, and report a matched ω\omega>0 comparison? This directly tests whether the capability is a property of the bridge or of the heuristic.

Q3. On the LARRY cell-fate benchmark, how do the baselines fare on the point-estimate metrics where deterministic models look "wildly inconsistent"?

Table 3 reports aggregate ranks favoring BIT, but the text says deterministic models do well on point-estimate but poorly on distributional metrics. Can you show the per-metric breakdown (forward/reverse MSE, RBF-MMD, fate/clone kNN, cycle consistency) so I can judge whether BIT's advantage is broad or driven by one or two columns? This affects how strongly I weigh the scientific-domain validation as evidence of general framework value.

Reviewer_6E8UOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces BIT, a diffusion-bridge framework for bidirectional image-text generation that directly transforms between text and image representations. It uses stochastic diffusion bridges to model the transport between modalities, enabling both text-to-image and image-to-text generation.

Critical strengths
  • Exploring direct transport between text and image distributions is an interesting direction for multimodal generative modeling, even if the general idea has been explored before.
  • Using diffusion bridges to model cross-modal transport is an intuitive and theoretically motivated alternative to deterministic flow matching, particularly for capturing the one-to-many nature of text-to-image generation.
Critical weaknesses
  • Directly transforming text representations into images rather than generating images from noise is not a novel direction, as works such as CrossFlow [1] and FlowTok [2] have already investigated this. Similarly, BDBM [3] has previously explored bidirectional stochastic bridges. BIT's contribution is therefore closer to applying and extending diffusion-bridge theory to cross-modal text-image generation rather than introducing an entirely new framework. Overall, I find the methodological novelty and contribution somewhat limited.
  • According to Appendix H.1.1, the authors train separate models for T2I and I2T. Despite describing BIT as a unified bidirectional framework, they do not actually demonstrate a single jointly trained image-text model performing both tasks. This somewhat weakens the claim of a unified bidirectional model.
  • Regarding the claimed image-editing capability, at least based on Figure 4, the method appears to generate largely new images rather than preserving the original image while applying targeted edits. The results seem closer to image regeneration than actual image editing, which raises questions about the practical editing capability of the proposed framework.
  • I would expect quantitative comparisons against closely related prior works such as CrossFlow [1] and FlowTok [2] to better benchmark the proposed method. These ODE-based approaches have already demonstrated strong image generation quality, and their qualitative results appear competitive, if not better, than those presented here. While a direct visual comparison across papers is not necessarily fair due to differences in training settings, architectures, and datasets, the lack of comparisons with these closely related methods makes it difficult to assess the actual benefits of the proposed stochastic formulation.

[1] Flowing from Words to Pixels: A Noise-Free Framework for Cross-Modality Evolution

[2] FlowTok: Flowing Seamlessly Across Text and Image Tokens

[3] Bidirectional Diffusion Bridge Models

Questions for authors

None

1 comment

I thank the authors for providing additional results and conducting the experiments within such a short timeframe. Most of my concerns have been resolved. In particular, the inclusion of CrossFlow and FlowTok as additional baselines was important and helped strengthen the evaluation. One remaining concern is that the low FID appears to be largely attributable to the in-distribution data on which BIT was trained, as acknowledged by the authors. Evaluating on a separate dataset that none of the compared methods were trained on would further strengthen the work and provide a more convincing assessment of generalization. Nevertheless, since most of my concerns have been addressed, I will raise my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.