- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
AP-FM addresses appearance-preserving image translation across camera viewpoints and human poses. Building on GP-FM, it transports a source image latent through a shared pivot before decoding it under the target condition; this structure yields compositional consistency for exact reversible flows. The paper identifies conditional velocity ambiguity from the standard Gaussian training coupling as a potential obstacle to preserving source appearance. To address it, the authors introduce identity-grouped Gaussian prior modes, fit their means and scales against a frozen GP-FM reference field, and subsequently adapt the velocity field to the resulting coupling. Experiments show substantial gains in target fidelity, appearance interventions, and multi-step translation on SRN Cars. Results on DeepFashion provide more limited, metric-dependent evidence of improvement.
Critical strengths
-
The paper presents a compelling motivation for addressing conflicting velocity targets in shared-pivot flow matching. Its structural analysis establishes compositional consistency under the stated flow assumptions, and its variance analysis provides a plausible mechanism by which adaptive prior modes can reduce conditional velocity ambiguity.
-
The SRN Cars experiments provide substantial empirical support for the method. AP-FM improves PSNR from 15.39 for GP-FM to 24.23 and shows favorable results on source-appearance interventions and multi-step composition (Table 1).
Critical weaknesses
-
The connection between the proposed optimization objective and the claimed mechanism remains incomplete. The Bridge objective minimizes the frozen reference field’s total regression error, which combines irreducible conditional velocity ambiguity with field approximation error. The theoretical comparison establishes a reduction at the prior endpoint under its assumptions, but does not show that Bridge fitting reduces ambiguity across flow time. The reported geometric diagnostics are informative proxies rather than direct measurements of this quantity.
-
The current experiments leave important questions about generalization. The strongest SRN result uses 300 identity-specific modes with Bridge-MDS, whereas DeepFashion uses eight grouped modes with a different fitting procedure and yields smaller, mixed improvements (Tables 1–2 and supplementary Tables 5–6). Since the evaluations use identities present in training, the results do not establish performance on unseen identities or provide a clear rule for selecting the mode structure and fitting method on a new task.
Questions for authors
-
Configuration selection. Bridge-MDS is strongest on SRN Cars, whereas Bridge performs better on DeepFashion. Beyond the explanations offered in the appendix, is there a training-set diagnostic or principled criterion for selecting the fitting method, mode count, and grouping for a new task without relying on test performance?
-
Contribution of each component. The prior uses learnable Gaussian modes with fixed identity-to-mode assignments. Can the authors isolate the gains attributable to this parameterization, MDS initialization, and optimization against the frozen reference field? In particular, how would MDS-initialized modes perform without subsequent Bridge fitting under the same field-training budget?
-
Evidence for the proposed mechanism. The analysis distinguishes conditional velocity ambiguity from field approximation error, while the reported field residual combines them. Could the authors estimate their respective contributions across flow time, or provide a more direct test that the improvement is driven by reduced ambiguity?
-
Unseen-identity generalization. Since the main evaluations use training identities, could the authors report results on held-out identities? This seems especially relevant to the SRN Cars configuration, which assigns a separate prior mode to each training identity.
4 comments
(Continued: Q1 and Q2.)
Q1. A training-set criterion for fitting method, mode count and grouping
Short answer: yes. A procedure on training identities only (Appendix D.5) checks the three conditions of §4.4 and selects one mode per training identity fitted by Bridge-MDS; unseen identities (Appendix F.1) serve only as a check, and there it is the best or tied-best fit.
- Before training. The ambiguity profile (Eq. 43) shows where GP-FM is ambiguous along the flow (Figure 6a); its start explains a shortfall together with steps 2 and 4, rather than predicting the gain alone (Table 6). The removable share (Eq. 44), one minus the within-mode variance (Eq. 27) relative to GP-FM, bounds what a grouping can remove (Appendix F.2); only one mode per identity can remove all of it, zeroing the within-mode variance and leaving the mode uncertainty (Eq. 28) for mode fitting to reduce.
- Reference models. Train GP-FM and SC-FM and read the gap GP−SC (Table 6); without it, AP-FM stays close to GP-FM on training identities.
- After mode optimization: coupling geometry (Table 4) shows, before field training, whether the modes separate the interpolation components.
- After field training: pivot identifiability (Eq. 45) divides the gap between an identity's encoded views by the gap to the nearest other identity (Figure 6b,c). Below 1 requires MDS-initialized identity modes, and Bridge-MDS has the lowest ratio wherever alternatives were measured (SRN .19, .73 with DiT; DeepFashion .15, .11). With RAE-DINO/U-Net it stays at 7.0 yet gains +3.2 dB, so the ratio is read with the other conditions.
The training-set diagnostic ranks the fits out of sample (tables in Q2). On SRN the training ratio orders the four fits exactly as unseen PSNR does; training PSNR does not, ranking Fixed-MDS last although it is second on unseen identities. On DeepFashion () it splits them the same way: the two MDS fits fall below 1 and gain about 0.8 dB on unseen identities, the two Gaussian-initialized fits stay above 2 and gain at most 0.2 dB. Measured on unseen identities (Figure 8), the ratio orders the SRN fits as before.
Resulting rule: one mode per training identity () with Bridge-MDS: it zeroes the within-mode variance, makes the mode uncertainty numerically negligible on the finite SRN training posterior (Table 5), has the lowest pivot ratio, and is the best training fit wherever alternatives were run (Tables 8, 10). is set by the data; on DeepFashion this rule replaces the submission's 8-mode Bridge (Table 2).
Confirmed on disjoint identities (Table 11, paired per identity):
- SRN, SD-VAE: Bridge-MDS beats Online-MLE (+0.93 dB, 14/15 identities) and Bridge (+1.79 dB, 15/15); it ties Fixed-MDS in single-step PSNR but keeps higher four-hop SSIM (12/15).
- SRN, RAE-DINO/DiT: it beats Fixed-MDS (+0.65 dB, 13/15).
- DeepFashion, : 15.57 dB, tied with Fixed-MDS (15.63) and 0.81 dB above GP-FM.
Q2. Contribution of each component
Short answer: the components play different roles. MDS initialization provides cross-identity geometry that transfers to unseen identities; Bridge fitting against the frozen reference field is needed for identity-resolved pivots on SRN and gives the large training gain; on DeepFashion both MDS variants already resolve the pivots and transfer similarly. The new Fixed-MDS control keeps the MDS-initialized modes fixed, i.e. without Bridge fitting (Tables 8, 10; Appendix F.3):
| SRN, | Online-MLE | Bridge | Fixed-MDS | Bridge-MDS |
|---|---|---|---|---|
| Train PSNR (GP-FM 15.39) | 19.17 | 18.20 | 17.93 | 24.23 |
| Unseen PSNR (GP-FM 14.46) | 15.11 | 14.25 | 16.01 | 16.04 |
| 19.9 | 21.1 | 4.97 | 0.19 |
| DeepFashion, RAE-DINO, | Online-MLE | Bridge | Fixed-MDS | Bridge-MDS |
|---|---|---|---|---|
| Train PSNR (GP-FM 21.80) | 21.75 | 21.80 | 21.91 | 21.94 |
| Unseen PSNR (GP-FM 14.76) | 14.74 | 14.99 | 15.63 | 15.57 |
| 2.26 | 2.80 | 0.44 | 0.15 |
- Parameterization alone (Gaussian-initialized modes): +2.8 to +3.8 dB on SRN, but pivots unresolved (ratio ≈ 20) and little transfer.
- MDS initialization alone: +2.5 dB on training identities and already transfers (16.01 vs 14.46 unseen), but SRN pivots stay unresolved (ratio ≈ 5).
- MDS + Bridge fitting: +8.84 dB and ratio 0.19 on SRN; out of sample it adds 0.65 dB over Fixed-MDS with DiT (13/15) and higher four-hop fidelity with SD-VAE.
Summary: the configuration is chosen from training identities alone and holds on unseen ones; MDS initialization gives transferable geometry, and Bridge fitting resolves the pivots on SRN and adds the large training gain.
Follow-up on W2/Q4: one mode per identity is not required, on training or unseen identities.
Regarding the concern that the SRN configuration assigns one prior mode to each training identity, we additionally evaluated the shared-mode variants ( and for 300 identities) on the 15 unseen identities, with the protocol of Table 7 (now added as two rows there). SRN, SD-VAE, Bridge-MDS; training / unseen identities:
| GP-FM | ||||
|---|---|---|---|---|
| Removable share (Eq. 44) | 0 | .52 | .69 | 1.00 |
| Fidelity: clean PSNR | 15.39 / 14.46 | 16.93 / 15.53 | 19.00 / 15.72 | 24.23 / 16.04 |
| Equivariance: hue− response cosine | .000 / .117 | .162 / .349 | .373 / .425 | .554 / .473 |
| Equivariance: hue− SSIM vs edited target | .802 / .776 | .816 / .791 | .834 / .795 | .881 / .796 |
| Composition: four-hop SSIM | .834 / .830 | .819 / .815 | .872 / .843 | .927 / .855 |
The removable share (Eq. 44) is one minus the spread among identities that share a mode, relative to GP-FM's velocity ambiguity at the prior endpoint. It reads as the largest fraction of that ambiguity a grouping into modes can remove, is computed before training, and equals 1 only for one mode per identity.
- Fidelity and edit response rise monotonically with the number of modes on both splits, following the removable share; the response to hue+ and RGB→GBR rises monotonically as well (Tables 7–8). A fifth of the identities as modes already gains +1.1 dB over GP-FM on unseen identities and triples its hue− response.
- Composition benefits once the modes separate identities: from , four-hop fidelity exceeds GP-FM on both splits; at it is slightly below GP-FM, whose composition is flat but at much lower fidelity.
- Inference uses no modes, so unseen identities need none; one mode per identity is the best configuration, not a requirement for generalization.
Update rating
Thanks for the authors' clarification. I've already updated my score.
We thank the reviewer for finding the motivation compelling and the variance analysis a plausible mechanism. This comment covers W1/Q3 and W2/Q4; Q1–Q2 follow in a second comment. Revised text is in blue.
In short: (W1/Q3) on SRN, where the split is exact, about half of AP-FM's residual reduction is ambiguity and half approximation, and the ambiguity part sits where GP-FM is ambiguous; (W2/Q4) on unseen identities the shared pivot keeps the best multi-step fidelity and AP-FM's gain over GP-FM persists; both tasks now use one configuration. (Scope: general response.)
W1 / Q3. Bridge minimizes total risk: is the gain driven by reduced ambiguity?
Short answer: partly. We agree that Bridge minimizes the total residual R=Cvel(0)+Aθ (Eq. 20), not the ambiguity alone, and that the original geometric diagnostics are proxies. The revision splits R directly on SRN, where the finite training posterior is exact: both terms fall, and the ambiguity term is removed where GP-FM is ambiguous.
1. Time-averaged split (Table 5).
From GP-FM to Bridge-MDS the residual halves, about half through each term. With identity modes the ambiguity term is numerically negligible (the posterior concentrates on the true component), so the fits differ in Aθ, lowest for Bridge-MDS.
2. Split across flow time (Table 3, Appendix D.4).
3. Across autoencoders (Figure 6a, Table 6). The ambiguity profile (Eq. 43) rises only for τ>.98 under RAE-DINO, against .84 under SD-VAE, and with the same U-Net the gain is +3.17 vs +8.84 dB. On DeepFashion the pose-neighborhood proxy estimates the ambiguity but does not split the residual.
W2 / Q4. Unseen identities and the different SRN/DeepFashion configurations
Short answer: yes. We evaluated 15 unseen SRN identities in three representations and 128 unseen DeepFashion identities in two autoencoders (Tables 7, 9; paired statistics in Table 11; Appendix F.1). Clean PSNR, and four-hop SSIM on SRN:
On configurations: the reviewer is right that the submission used 300 identity modes on SRN but 8 shared modes on DeepFashion. Both tasks now use one rule, one mode per training identity with Bridge-MDS (Q1): the best DeepFashion training fit (21.94 dB; 28.26 with SD-VAE) and +0.81 dB on unseen identities, where the 8-mode configuration was on par with GP-FM (14.75 dB).
Summary: about half of the gain is ambiguity reduction, located where GP-FM is ambiguous; out of sample the pivot keeps the best multi-step fidelity and the gain over GP-FM persists; one configuration now serves both tasks.