acceptodds
Under review as a conference paper at ICLR 2027

Adaptively Pivoted Flow Matching for Composition-Consistent Translation

Abstract

Controlled image translation requires jointly achieving target fidelity, source equivariance under appearance interventions, and compositional consistency across sequential paths. Factoring conditional transport through a shared latent pivot () naturally induces a translation groupoid structure: this algebraic formulation inherently ensures compositional consistency under exact integration, guarantees bi-Lipschitz source sensitivity against information collapse, and introduces a structural inductive bias for source equivariance. While this architecture is coupled with a standard Gaussian prior, overlapping interpolation distributions associate the same state, time, and condition with different target velocities, producing conditional velocity ambiguity. To reduce this ambiguity, we introduce Adaptively Pivoted Flow Matching (AP-FM), which replaces the single Gaussian with an adaptive prior comprising trainable multi-Gaussian modes optimized with a prior mode adaptation loss defined using a frozen reference field. At inference, the observed source determines the latent pivot; neither identity labels nor mode lookup is required. On SRN Cars and DeepFashion, AP-FM preserves source appearance under interventions and composed translations, with improved target fidelity over a standard Gaussian prior.

Primary area:
generative models
Keywords:
flow matching, image-to-image translation, adaptive priors, compositional consistency, source equivariance

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aUAAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AP-FM addresses appearance-preserving image translation across camera viewpoints and human poses. Building on GP-FM, it transports a source image latent through a shared pivot before decoding it under the target condition; this structure yields compositional consistency for exact reversible flows. The paper identifies conditional velocity ambiguity from the standard Gaussian training coupling as a potential obstacle to preserving source appearance. To address it, the authors introduce identity-grouped Gaussian prior modes, fit their means and scales against a frozen GP-FM reference field, and subsequently adapt the velocity field to the resulting coupling. Experiments show substantial gains in target fidelity, appearance interventions, and multi-step translation on SRN Cars. Results on DeepFashion provide more limited, metric-dependent evidence of improvement.

Critical strengths
  1. The paper presents a compelling motivation for addressing conflicting velocity targets in shared-pivot flow matching. Its structural analysis establishes compositional consistency under the stated flow assumptions, and its variance analysis provides a plausible mechanism by which adaptive prior modes can reduce conditional velocity ambiguity.

  2. The SRN Cars experiments provide substantial empirical support for the method. AP-FM improves PSNR from 15.39 for GP-FM to 24.23 and shows favorable results on source-appearance interventions and multi-step composition (Table 1).

Critical weaknesses
  1. The connection between the proposed optimization objective and the claimed mechanism remains incomplete. The Bridge objective minimizes the frozen reference field’s total regression error, which combines irreducible conditional velocity ambiguity with field approximation error. The theoretical comparison establishes a reduction at the prior endpoint under its assumptions, but does not show that Bridge fitting reduces ambiguity across flow time. The reported geometric diagnostics are informative proxies rather than direct measurements of this quantity.

  2. The current experiments leave important questions about generalization. The strongest SRN result uses 300 identity-specific modes with Bridge-MDS, whereas DeepFashion uses eight grouped modes with a different fitting procedure and yields smaller, mixed improvements (Tables 1–2 and supplementary Tables 5–6). Since the evaluations use identities present in training, the results do not establish performance on unseen identities or provide a clear rule for selecting the mode structure and fitting method on a new task.

Questions for authors
  1. Configuration selection. Bridge-MDS is strongest on SRN Cars, whereas Bridge performs better on DeepFashion. Beyond the explanations offered in the appendix, is there a training-set diagnostic or principled criterion for selecting the fitting method, mode count, and grouping for a new task without relying on test performance?

  2. Contribution of each component. The prior uses learnable Gaussian modes with fixed identity-to-mode assignments. Can the authors isolate the gains attributable to this parameterization, MDS initialization, and optimization against the frozen reference field? In particular, how would MDS-initialized modes perform without subsequent Bridge fitting under the same field-training budget?

  3. Evidence for the proposed mechanism. The analysis distinguishes conditional velocity ambiguity from field approximation error, while the reported field residual combines them. Could the authors estimate their respective contributions across flow time, or provide a more direct test that the improvement is driven by reduced ambiguity?

  4. Unseen-identity generalization. Since the main evaluations use training identities, could the authors report results on held-out identities? This seems especially relevant to the SRN Cars configuration, which assigns a separate prior mode to each training identity.

4 comments

We thank the reviewer for finding the motivation compelling and the variance analysis a plausible mechanism. This comment covers W1/Q3 and W2/Q4; Q1–Q2 follow in a second comment. Revised text is in blue.

In short: (W1/Q3) on SRN, where the split is exact, about half of AP-FM's residual reduction is ambiguity and half approximation, and the ambiguity part sits where GP-FM is ambiguous; (W2/Q4) on unseen identities the shared pivot keeps the best multi-step fidelity and AP-FM's gain over GP-FM persists; both tasks now use one configuration. (Scope: general response.)

W1 / Q3. Bridge minimizes total risk: is the gain driven by reduced ambiguity?

Short answer: partly. We agree that Bridge minimizes the total residual R=Cvel(0)+AθR=C^{(0)}_{\rm vel}+A_\theta (Eq. 20), not the ambiguity alone, and that the original geometric diagnostics are proxies. The revision splits RR directly on SRN, where the finite training posterior is exact: both terms fall, and the ambiguity term is removed where GP-FM is ambiguous.

1. Time-averaged split (Table 5).

R/ER/\mathcal ECvel(0)/EC^{(0)}_{\rm vel}/\mathcal EAθ/EA_\theta/\mathcal E
GP-FM.0440.0112.0329
AP-FM, Online-MLE.0316≈0.0316
AP-FM, Bridge-MDS.0225≈0.0225

From GP-FM to Bridge-MDS the residual halves, about half through each term. With identity modes the ambiguity term is numerically negligible (the posterior concentrates on the true component), so the fits differ in AθA_\theta, lowest for Bridge-MDS.

2. Split across flow time (Table 3, Appendix D.4).

τ\tauGP-FM RR / CvelC_{\rm vel}Online-MLE RRBridge-MDS RR
0.5.025 / 0.025.005
0.8.039 / .0002.020.013
0.9.069 / .032.022.028
1.0.081 / .079.024.153
  • GP-FM's ambiguity appears only for τ≥0.8\tau\ge0.8 and is almost all of its residual at τ=1\tau=1.
  • Online-MLE, with transport energy close to GP-FM's, lowers the residual only over that final stretch: the improvement sits where the ambiguity is.
  • Bridge-MDS also lowers approximation error along the path, so its larger gain (+8.84 vs +3.78 dB) also reflects a better-fitted field; its error concentrates at τ=1\tau=1 (.153, ambiguity ≈0), an approximation cost of mapping each mode to its identity, and is the lowest of the three for τ≤0.9\tau\le0.9 (Appendix D.4).

3. Across autoencoders (Figure 6a, Table 6). The ambiguity profile (Eq. 43) rises only for τ>.98\tau>.98 under RAE-DINO, against .84 under SD-VAE, and with the same U-Net the gain is +3.17 vs +8.84 dB. On DeepFashion the pose-neighborhood proxy estimates the ambiguity but does not split the residual.

W2 / Q4. Unseen identities and the different SRN/DeepFashion configurations

Short answer: yes. We evaluated 15 unseen SRN identities in three representations and 128 unseen DeepFashion identities in two autoencoders (Tables 7, 9; paired statistics in Table 11; Appendix F.1). Clean PSNR, and four-hop SSIM on SRN:

Unseen identitiesSC-FMD2D-FMGP-FMAP-FM
SRN, SD-VAE15.86 / .84016.46 / .83214.46 / .83016.04 / .855
SRN, RAE-DINO / U-Net15.02 / .786–14.17 / .81715.70 / .841
SRN, RAE-DINO / DiT16.41 / .82916.99 / .84514.12 / .82416.56 / .851
DeepFashion, RAE-DINO15.5315.6614.7615.57
DeepFashion, SD-VAE13.7812.1812.3012.52
  • The shared pivot holds out of sample. AP-FM has the best four-hop fidelity on unseen SRN identities in all three representations, ahead of SC-FM on 14–15 of 15 identities; D2D-FM is best at a single step but drifts (−.030 SSIM over four hops vs −.001 for AP-FM, SD-VAE).
  • The gain over GP-FM transfers: +1.5 to +2.4 dB on 15 of 15 unseen SRN identities per representation, +0.8 dB on DeepFashion; with SD-VAE on DeepFashion, SC-FM is best out of sample.
  • The train→unseen drop is not specific to modes: SC-FM, which has none, drops 7.3 dB vs 8.2 for AP-FM on SRN/SD-VAE; on DeepFashion AP-FM drops less than GP-FM (6.4 vs 7.0 dB).

On configurations: the reviewer is right that the submission used 300 identity modes on SRN but 8 shared modes on DeepFashion. Both tasks now use one rule, one mode per training identity with Bridge-MDS (Q1): the best DeepFashion training fit (21.94 dB; 28.26 with SD-VAE) and +0.81 dB on unseen identities, where the 8-mode configuration was on par with GP-FM (14.75 dB).

Summary: about half of the gain is ambiguity reduction, located where GP-FM is ambiguous; out of sample the pivot keeps the best multi-step fidelity and the gain over GP-FM persists; one configuration now serves both tasks.

(Continued: Q1 and Q2.)

Q1. A training-set criterion for fitting method, mode count and grouping

Short answer: yes. A procedure on training identities only (Appendix D.5) checks the three conditions of §4.4 and selects one mode per training identity fitted by Bridge-MDS; unseen identities (Appendix F.1) serve only as a check, and there it is the best or tied-best fit.

  1. Before training. The ambiguity profile (Eq. 43) shows where GP-FM is ambiguous along the flow (Figure 6a); its start τ∗\tau^* explains a shortfall together with steps 2 and 4, rather than predicting the gain alone (Table 6). The removable share (Eq. 44), one minus the within-mode variance (Eq. 27) relative to GP-FM, bounds what a grouping can remove (Appendix F.2); only one mode per identity can remove all of it, zeroing the within-mode variance and leaving the mode uncertainty (Eq. 28) for mode fitting to reduce.
  2. Reference models. Train GP-FM and SC-FM and read the gap GP−SC (Table 6); without it, AP-FM stays close to GP-FM on training identities.
  3. After mode optimization: coupling geometry (Table 4) shows, before field training, whether the modes separate the interpolation components.
  4. After field training: pivot identifiability (Eq. 45) divides the gap DpairD_{\rm pair} between an identity's encoded views by the gap DnnD_{\rm nn} to the nearest other identity (Figure 6b,c). Below 1 requires MDS-initialized identity modes, and Bridge-MDS has the lowest ratio wherever alternatives were measured (SRN .19, .73 with DiT; DeepFashion .15, .11). With RAE-DINO/U-Net it stays at 7.0 yet gains +3.2 dB, so the ratio is read with the other conditions.

The training-set diagnostic ranks the fits out of sample (tables in Q2). On SRN the training ratio orders the four fits exactly as unseen PSNR does; training PSNR does not, ranking Fixed-MDS last although it is second on unseen identities. On DeepFashion (M=256M=256) it splits them the same way: the two MDS fits fall below 1 and gain about 0.8 dB on unseen identities, the two Gaussian-initialized fits stay above 2 and gain at most 0.2 dB. Measured on unseen identities (Figure 8), the ratio orders the SRN fits as before.

Resulting rule: one mode per training identity (M=KM=K) with Bridge-MDS: it zeroes the within-mode variance, makes the mode uncertainty numerically negligible on the finite SRN training posterior (Table 5), has the lowest pivot ratio, and is the best training fit wherever alternatives were run (Tables 8, 10). MM is set by the data; on DeepFashion this rule replaces the submission's 8-mode Bridge (Table 2).

Confirmed on disjoint identities (Table 11, paired per identity):

  • SRN, SD-VAE: Bridge-MDS beats Online-MLE (+0.93 dB, 14/15 identities) and Bridge (+1.79 dB, 15/15); it ties Fixed-MDS in single-step PSNR but keeps higher four-hop SSIM (12/15).
  • SRN, RAE-DINO/DiT: it beats Fixed-MDS (+0.65 dB, 13/15).
  • DeepFashion, M=256M=256: 15.57 dB, tied with Fixed-MDS (15.63) and 0.81 dB above GP-FM.

Q2. Contribution of each component

Short answer: the components play different roles. MDS initialization provides cross-identity geometry that transfers to unseen identities; Bridge fitting against the frozen reference field is needed for identity-resolved pivots on SRN and gives the large training gain; on DeepFashion both MDS variants already resolve the pivots and transfer similarly. The new Fixed-MDS control keeps the MDS-initialized modes fixed, i.e. without Bridge fitting (Tables 8, 10; Appendix F.3):

SRN, M=300M=300Online-MLEBridgeFixed-MDSBridge-MDS
Train PSNR (GP-FM 15.39)19.1718.2017.9324.23
Unseen PSNR (GP-FM 14.46)15.1114.2516.0116.04
Dpair/DnnD_{\rm pair}/D_{\rm nn}19.921.14.970.19
DeepFashion, RAE-DINO, M=256M=256Online-MLEBridgeFixed-MDSBridge-MDS
Train PSNR (GP-FM 21.80)21.7521.8021.9121.94
Unseen PSNR (GP-FM 14.76)14.7414.9915.6315.57
Dpair/DnnD_{\rm pair}/D_{\rm nn}2.262.800.440.15
  • Parameterization alone (Gaussian-initialized modes): +2.8 to +3.8 dB on SRN, but pivots unresolved (ratio ≈ 20) and little transfer.
  • MDS initialization alone: +2.5 dB on training identities and already transfers (16.01 vs 14.46 unseen), but SRN pivots stay unresolved (ratio ≈ 5).
  • MDS + Bridge fitting: +8.84 dB and ratio 0.19 on SRN; out of sample it adds 0.65 dB over Fixed-MDS with DiT (13/15) and higher four-hop fidelity with SD-VAE.

Summary: the configuration is chosen from training identities alone and holds on unseen ones; MDS initialization gives transferable geometry, and Bridge fitting resolves the pivots on SRN and adds the large training gain.

Follow-up on W2/Q4: one mode per identity is not required, on training or unseen identities.

Regarding the concern that the SRN configuration assigns one prior mode to each training identity, we additionally evaluated the shared-mode variants (M=64M=64 and 128128 for 300 identities) on the 15 unseen identities, with the protocol of Table 7 (now added as two rows there). SRN, SD-VAE, Bridge-MDS; training / unseen identities:

GP-FMM=64M=64M=128M=128M=300M=300
Removable share (Eq. 44)0.52.691.00
Fidelity: clean PSNR15.39 / 14.4616.93 / 15.5319.00 / 15.7224.23 / 16.04
Equivariance: hue− response cosine CrgbC_{\rm rgb}.000 / .117.162 / .349.373 / .425.554 / .473
Equivariance: hue− SSIM vs edited target.802 / .776.816 / .791.834 / .795.881 / .796
Composition: four-hop SSIM.834 / .830.819 / .815.872 / .843.927 / .855

The removable share (Eq. 44) is one minus the spread among identities that share a mode, relative to GP-FM's velocity ambiguity at the prior endpoint. It reads as the largest fraction of that ambiguity a grouping into MM modes can remove, is computed before training, and equals 1 only for one mode per identity.

  • Fidelity and edit response rise monotonically with the number of modes on both splits, following the removable share; the response to hue+ and RGB→GBR rises monotonically as well (Tables 7–8). A fifth of the identities as modes already gains +1.1 dB over GP-FM on unseen identities and triples its hue− response.
  • Composition benefits once the modes separate identities: from M=128M=128, four-hop fidelity exceeds GP-FM on both splits; at M=64M=64 it is slightly below GP-FM, whose composition is flat but at much lower fidelity.
  • Inference uses no modes, so unseen identities need none; one mode per identity is the best configuration, not a requirement for generalization.
Reviewer_aUAAOpenReview ↗

Update rating

Thanks for the authors' clarification. I've already updated my score.

Reviewer_jAFqOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Adaptively Pivoted Flow Matching (AP-FM), a framework for conditional image translation. The core idea is based on two modules: Translation Groupoid Structure: Formulates conditional transformations as a groupoid, algebraically guaranteeing compositional consistency (Ts→t=Dt∘EsT_{s \to t} = D_t \circ E_s) through the precise cancellation of intermediate encoding-decoding flow maps, thereby avoiding error accumulation during multi-step latent traversals. Adaptively Pivoted Multi-Gaussian Priors: Replaces traditional single or static Gaussian priors with MM trainable Gaussian modes to eliminate "conditional velocity ambiguity," resolving trajectory cross-contamination that typically causes output blurring.The method is validated on standard benchmarks spanning rigid 3D geometries (SRN Cars using Stable Diffusion VAE) and non-rigid semantic manipulations (DeepFashion using DINOv2-based RAE).

Critical strengths

The application of translation groupoid theory provides a clean algebraic guarantee for compositional consistency without requiring heuristic regularizers. The property that intermediate latent states (zrz_r) cancel out cleanly across multi-step paths (Er∘Dr=idE_r \circ D_r = id) is a conceptually elegant solution to multi-hop drift.

By introducing multi-Gaussian modes to uncouple overlapping trajectories in the shared pivot space, the model directly addresses a fundamental limitation of standard flow matching. This leads to tangible qualitative improvements, successfully recovering high-frequency textures and sharper structures (such as facial features and apparel details) where traditional baselines yield averaged, blurry outputs.

Critical weaknesses

The framework relies on completely different pre-trained image encoder-decoder depending on the task (Stable Diffusion VAE for rigid 3D vs. DINOv2 RAE for dense semantics). It remains unclear how the framework generalizes to a unified encoder-decoder or handles complex, multi-condition entanglement (e.g., simultaneously editing pose, facial identity, and clothing) where feature capacity and inter-condition cross-talk could break the current pivot assumptions.

AP-FM framework relies on a discrete, manually tuned number of Gaussian modes (MM) to cluster identity and category distributions (e.g., M=8M=8 in DeepFashion). However, in real-world open-world scenarios characterized by a virtually infinite or massive number of distinct identities, the static choice of MM becomes a severe limitation. If MM is too small, it fails to capture long-tail variations, resulting in severe trajectory collision; if MM is scaled indefinitely to accommodate thousands of identities, it introduces severe training instability, clustering overlap, and lacks a dynamic mechanism for open-ended adaptation.

Questions for authors

All primary experiments are constrained to 256×256256 \times 256 pixels. The paper does not thoroughly demonstrate whether the multi-Gaussian pivot allocation and high-dimensional latent mapping (768768-channel DINOv2 RAE) can scale smoothly to higher-resolution generation without exploding memory costs or encountering representation bottlenecks.

How do the MM adaptive Gaussian modes behave when scaling from single-condition control (e.g., pure camera ray or skeleton pose) to joint multi-condition editing? Do the modes suffer from capacity overload or cross-contamination when multiple conflicting control signals are introduced simultaneously?

Given that SRN Cars and DeepFashion employ entirely distinct underlying encoder-decoder, how sensitive is the AP-FM performance gain to the choice of the foundational encoder-decoder? Could a sub-optimal encoder-decoder, choice negate the benefits of the adaptively pivoted prior?

2 comments

We thank the reviewer for recognizing the clean algebraic guarantee for composition and the recovered detail from the adaptive modes. Revised text is in blue.

In short: the gain over GP-FM holds across autoencoders and backbones, with a size set by how much the Gaussian coupling limits GP-FM (§4.4); the representation bottleneck belongs to the autoencoder, not to AP-FM. Modes are set by the data, stay separated, help even when fewer than identities, and are not used at inference. (Scope: general response.)

W1 / Q3. Dependence on the encoder–decoder

Short answer: the encoder changes the size of the gain, not its sign, including with one encoder (RAE-DINO) for both tasks (Table 6; Appendix F.4):

Clean PSNR (dB)GP−SC (train)AP−GP, trainAP−GP, unseen
SRN, SD-VAE / U-Net−7.78+8.84+1.58
SRN, RAE-DINO / U-Net−2.40+3.17+1.53
SRN, RAE-DINO / DiT−4.43+5.86+2.44
DeepFashion, RAE-DINO / XMDPT+0.20+0.14+0.81
DeepFashion, SD-VAE / U-Net+1.84+0.23+0.22

An encoder that already separates identities along most of the flow leaves the prior less to remove (RAE-DINO: ambiguity only for τ>.98\tau>.98, vs .84 for SD-VAE; Figure 6a); the gain then tracks the GP−SC gap.

W1 / Q2. Multi-condition entanglement

Short answer: not tested here; future work. Structurally, Props. 1–2 hold for any (composite) condition, which enters only the field; modes group identities, not control signals, so cross-talk is a matter of field capacity, as for GP-FM and SC-FM.

W2. Number of modes for open-world identity counts

Short answer: MM is set by the data (one mode per training identity: 300 on SRN, 256 on DeepFashion); modes serve only training and are not used at inference. Each scenario the reviewer raises, measured:

  • Fewer modes than identities. The collision a grouping leaves is its unremovable ambiguity, measured before training by the removable share (Eq. 44). Fidelity follows it and stays above GP-FM:
SRN, SD-VAE (300 identities)GP-FMM=64M=64M=128M=128M=300M=300
Removable share0.52.691.00
Pivot ratio Dpair/DnnD_{\rm pair}/D_{\rm nn}20.94.181.210.19
Clean PSNR (dB)15.3916.9319.0024.23

A fifth of the identities as modes already gives +1.5 dB: initial evidence, beyond our claims, that AP-FM scales without a mode per identity (Appendix F.2).

  • One mode per identity at scale. Trains stably at MM = 256 and 300 under every autoencoder and backbone; modes stay separated, with pivot ratio below 1 (SRN .19, .73 with DiT; DeepFashion .15, .11; Figure 6b,c). Each step needs only its batch's modes.
  • New identities. Unseen identities need no mode, and AP-FM still beats GP-FM on them: +1.5 to +2.4 dB on SRN (15/15 identities) and +0.8 dB on DeepFashion (Tables 7, 9, 11). MDS modes are closed-form in identity latents (Eqs. 46–48), so a new identity's mode can be computed from its views, a natural route to a dynamic mechanism (future work).

Q1. Higher resolution and the autoencoder bottleneck

Short answer: the bottleneck lies in the autoencoder, not in AP-FM: RAE-DINO limits in-sample fidelity for every method, a larger backbone does not lift it, and AP-FM keeps its gain and stable composition under both autoencoders (Tables 7–10). Clean PSNR, training (unseen) identities:

SRNSC-FMD2D-FMGP-FMAP-FM
SD-VAE / U-Net23.17 (15.86)19.40 (16.46)15.39 (14.46)24.23 (16.04)
RAE-DINO / U-Net16.84 (15.02)17.97 (–)14.44 (14.17)17.61 (15.70)
RAE-DINO / DiT18.51 (16.41)18.57 (16.99)14.08 (14.12)19.94 (16.56)
DeepFashionSC-FMD2D-FMGP-FMAP-FM
SD-VAE / U-Net26.19 (13.78)11.87 (12.18)28.03 (12.30)28.26 (12.52)
RAE-DINO / XMDPT21.60 (15.53)20.94 (15.66)21.80 (14.76)21.94 (15.57)
  • Capacity does not remove it: with RAE-DINO, DiT changes each method by at most 2.3 dB; SC-FM and AP-FM stay 4–5 dB below SD-VAE.
  • The pivot structure is unaffected: AP-FM's four-hop SSIM stays within .007 of single-step on SRN (.931→.927, .863→.863, .890→.883).
  • Transfer can favour RAE-DINO: it beats SD-VAE for every method on unseen DeepFashion identities (Appendix F.4).

Inference costs what GP-FM costs; training adds a mode table linear in M⋅DM\cdot D (0.4 GB for RAE-DINO at 256×256). Training above 256×256 is future work.

Summary: the gain survives encoder changes and tracks GP−SC; modes are set by the data, stay separated, are unused at inference, and help even when fewer than identities; the resolution bottleneck is the autoencoder's.

Thanks for the authors' answer and experiments, though the authors acknowledge the limitation of not being able to edit multi-properties at the same time, the rebuttal addressed most of my concerns, I will keep my score.

Reviewer_yMsWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AP-FM for image translation. The method first extracts the noise as shared pivot from conditional image, then generate the target image with that initial noise (shared pivot). The author proposes to use shared adaptive gaussian modes to reduce ambiguity caused by standard noise.

Critical strengths
  1. The paper is easy to follow and understand
  2. The adaptive prior improves flow matching by modifying training coupling
Critical weaknesses
  1. Limited novelty: The core method is too simple. Just simply inverse from image to noise then generate target under new condition with initial noise

  2. Weak empirical improvement across tasks: AP-FM only gains marginal PSNR and SSIM on DeepFashion. Compositional consistency and appearance metrics are worse than GP-FM, leading to concern about the general performance of technique

  3. Can the author please provide the comparison against DDIB, MM-FM and CSFM ? I think the technique is simple combination of these works but lack of comparison and motivation.

DDIB: Dual Diffusion Implicit Bridges for Image-to-Image Translation

MM-FM: Flow Matching for Multimodal Distributions

CSFM: Better Source, Better Flow: Learning Condition-Dependent Source Distribution for Flow Matching

Questions for authors

see the weaknesses

1 comment

In short: as the reviewer observes, AP-FM's inference follows the DDIB-style shared pivot, which we keep deliberately because it makes composition exact. Our contribution is a principled account of this translation structure and of how to train it: what the structure guarantees, why its Gaussian training coupling costs fidelity, and an adaptive coupling that reduces this cost with inference unchanged. On DeepFashion this cost is small, which §4.4 now explains (scope: general response).

W1. Novelty: "invert to noise, then regenerate"

Short answer: the shared-pivot inference is common with DDIB by design; the paper contributes a theory of this translation structure and a training method for it:

  1. Structure: a translation-groupoid formulation with exact composition, a source-sensitivity bound and an equivariance criterion (Props. 1–2).
  2. Learning problem: a Gaussian coupling maps one state, time and condition to many target velocities (Eq. 24), which costs fidelity.
  3. Method: identity-shared modes, fitted against a frozen reference field, change only the training coupling; evaluated on interventions, composition and diagnostics (§4.4).

Table 1 separates the roles of structure and coupling:

  • Removing the pivot (SC-FM) attenuates source interventions: CrgbC_{\rm rgb} .481 vs .554 for AP-FM.
  • Removing the shared latent (D2D-FM) breaks composition: SSIM .888 → .849 over four hops.
  • Keeping the pivot but using a Gaussian coupling (GP-FM) collapses fidelity: 15.39 vs 24.23 dB.

The design trade-off: a shared Gaussian pivot buys exact composition, but its coupling averages the velocities of all identities that share a condition (Eq. 24). AP-FM reduces this cost; the price is that identity grouping is needed during training (not at inference).

W2. Gains on DeepFashion; composition and appearance vs GP-FM

Short answer: DeepFashion marks the boundary of the method rather than contradicting it. GP-FM already performs strongly there, so the coupling has little fidelity deficit to recover; AP-FM stays comparable in composition, slightly improves fidelity and appearance response, and gains more on unseen identities (+0.81 dB, Table 9).

Fidelity follows how far GP-FM trails source conditioning (clean PSNR, training identities):

Task, autoencoderSC-FMGP-FMAP-FMGP−SCAP−GP
SRN, SD-VAE23.1715.3924.23−7.78+8.84
SRN, RAE-DINO / DiT18.5114.0819.94−4.43+5.86
DeepFashion, RAE-DINO21.6021.8021.94+0.20+0.14
DeepFashion, SD-VAE26.1928.0328.26+1.84+0.23

§4.4 explains why: large gains need ambiguity across the flow, a prior that separates identities, and GP-FM trailing SC-FM. On DeepFashion GP-FM does not trail SC-FM, and its ambiguity appears only in the last 1% of the flow (Figure 6a).

Composition is shared by design; appearance gains are empirical. GP-FM and AP-FM use the same shared pivot, so both inherit composition and source sensitivity, and AP-FM is not expected to beat GP-FM on composition. What AP-FM adds is a better-fitted field, which improves the measured appearance response:

  • Table 2 updates. The AP-FM row now uses the SRN rule; hue columns now use the garment-region edit.
  • Hue shifts. AP-FM slightly improves over GP-FM (SSIM/LPIPS .7830/.0905 vs .7816/.0925); both agree far better with the ground-truth response than source conditioning (CappC_{\rm app} .383 / .369 vs .140).
  • Garment substitution. AP-FM is on par with GP-FM in SSIM and better in LPIPS (.7147/.1840 vs .7145/.1902), with higher CappC_{\rm app} (.543 vs .529).
  • Four-hop composition. Flat for both (SSIM .971 GP, .967 AP), while D2D-FM drops to .730.

W3. Comparison with DDIB, MM-FM and CSFM

Short answer: GP-FM is our controlled DDIB-style baseline; AP-FM changes only its training coupling. MM-FM and CSFM change the prior for other purposes and do not translate through a shared pivot.

MethodRelation to AP-FM
DDIBClosest inference structure (shared pivot); GP-FM is its flow-matching analogue (Appendix B.1), so AP-FM vs GP-FM is the controlled comparison.
MM-FMMixture prior for generation. Adapted to translation as source-conditioned FM, with the component chosen from source features (Appendix B.1): no shared pivot.
CSFMCondition-dependent source for generation. Grouping modes by condition would split an identity's views across modes; AP-FM shares identity modes across conditions and gives the condition only to the field (Appendix H; conceptual, not run).
Clean PSNRMM-FM (adapted)SC-FM + C²OTAP-FM
SRN20.7422.0424.23
DeepFashion21.8021.8221.94

Summary: the novelty is the theory and training of shared-pivot translation; DeepFashion is its boundary; GP-FM is the DDIB-style baseline, and adapted MM-FM and C²OT are below AP-FM on both tasks.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Scope of the Paper and Summary of the Revision

We thank all reviewers for their careful and constructive reviews. The revised paper is uploaded, with all changes in blue.

In short: the shared latent pivot (GP-FM and AP-FM) guarantees compositional consistency and source sensitivity; AP-FM's adaptive coupling improves fidelity and the response to source edits where the Gaussian prior's velocity ambiguity limits GP-FM.

Problem setting

Controlled translation renders a source observation under a new condition, such as a camera view or a body pose. A useful model must do three things at once:

  • Target fidelity: reproduce the target.
  • Source equivariance: carry appearance edits of the source into the output.
  • Compositional consistency: give the same result whether a translation is made directly or through intermediate conditions.

Existing designs trade these off. Source-conditioned FM attenuates source edits, and unpivoted pairwise FM drifts under composition (Tables 1–2). The paper asks how one model can deliver all three.

Contributions

  1. Shared-pivot structure. Translating through a shared latent pivot, Ts→t=Dt∘EsT_{s\to t}=D_t\circ E_s, makes composition exact for exact flows, guarantees source sensitivity, and gives a condition for equivariance (Props. 1–2). GP-FM, the standard-Gaussian instance, shares this structure.
  2. Conditional velocity ambiguity. With an independent Gaussian prior, the same state, time and condition can correspond to different target velocities (Eq. 24), so the regression must average them. How much this matters depends on the representation and the conditioning.
  3. AP-FM. Identity-shared Gaussian modes, obtained by a mode optimization against a frozen reference field, change only the training coupling. Inference is unchanged and needs no identity labels.

Scope: two questions

The paper does not claim that AP-FM wins every metric everywhere; every setting in the revision answers one of two questions.

Q1. What does translation through a shared latent pivot provide, and what learning problem remains?

  • Compositional consistency: Ts→t=Dt∘EsT_{s\to t}=D_t\circ E_s guarantees exact composition for exact flows, since intermediate maps cancel (Prop. 1). SRN four-hop fidelity stays stable for AP-FM (.931→.927), while unpivoted D2D-FM drifts (.888→.849).
  • Source transport and equivariance: transporting the source state gives a bi-Lipschitz bound against latent collapse (Eq. 4), and Prop. 2 characterizes when edits are equivariant. In image space, AP-FM's response to hue edits agrees better in direction and errs less in magnitude than SC-FM's (CrgbC_{\rm rgb} .554 vs .481; DrgbD_{\rm rgb} 1.38 vs 2.31).
  • Remaining learning problem: under a standard Gaussian prior, the same state, time and condition can correspond to conflicting target velocities (Eq. 24), forcing conditional-mean regression. On SRN, GP-FM correspondingly exhibits lower fidelity and weaker edit response (§4.1); Q2 asks when adaptive coupling improves it.

Q2. When does adaptive prior coupling improve over GP-FM? When this ambiguity limits GP-FM. §4.4 checks three conditions on training identities: ambiguity spans much of the flow (Eq. 43), the prior separates identities at the pivot (Eq. 45), and GP-FM trails SC-FM. All hold on SRN (+8.8 dB); on DeepFashion, GP-FM already matches SC-FM, so the gain is small (+0.1 dB training, +0.8 dB unseen).

What the revision adds

Within scope: new analysis of Q1 and Q2

  • §4.4 explains when adaptive coupling helps across tasks, autoencoders and backbones (Figure 6, Table 6).
  • A direct test of the mechanism (Tables 3, 5): on SRN, the residual split into ambiguity and approximation, overall and per flow time.
  • One configuration for both tasks, chosen from training identities (Table 2, Appendix F.3): one mode per training identity with Bridge-MDS; best or tied-best on unseen identities.
  • The submitted mode-count ablation, now explained and extended to unseen identities (Tables 7–8, Appendix F.2): on SRN, PSNR rises from GP-FM to MM = 64, 128 and 300 on training identities (15.39 → 16.93, 19.00, 24.23) and on unseen ones (14.46 → 15.53, 15.72, 16.04), with the edit response rising likewise, following the removable share (Eq. 44). Fewer modes than identities already beat GP-FM: initial evidence, beyond our scope, that AP-FM scales without a mode per identity.

New experiments at the reviewers' request

  • The gain holds on unseen identities (Appendix F.1, Tables 7, 9, 11): AP-FM beats GP-FM on every unseen SRN identity (+1.5 to +2.4 dB) and keeps the best four-hop fidelity; +0.8 dB on DeepFashion.
  • New baselines and controls (Tables 8, 10; Appendices B.1, H): adapted MM-FM and C²OT, a CSFM discussion, a Fixed-MDS control, and an autoencoder × backbone matrix.

Summary: the shared pivot guarantees composition and source sensitivity; adaptive coupling improves fidelity and edit response where the Gaussian coupling's ambiguity limits GP-FM.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.