- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SWIFT, a flow matching training framework for conditional audio generation. It replaces the frame-level Euclidean cost in minibatch OT with a new ground metric, SWSD. SWSD prepends a time coordinate to each frame to encode temporal order. It then samples slicing directions from a von Mises Fisher distribution centered on the projected conditioning vector to compute a sliced Wasserstein distance. The Sinkhorn algorithm uses this distance to couple audio and noise. The paper evaluates SWIFT on text-to-audio generation with AudioCaps and video-to-audio generation with VGGSound. SWIFT outperforms existing coupling methods on most metrics, with larger gains at low NFE, and provides a better initialization for reflow.
Critical strengths
- The paper addresses the failure of Euclidean OT coupling in conditional audio generation. Data-noise coupling for flow matching in this setting has received little systematic attention.
- Time augmentation and vMF condition-directed slicing are combined into a single ground cost. This adds only training-time cost with no inference overhead, which makes the method practical.
- Results cover text-to-audio and video-to-audio, with fairly consistent FAD gains at low NFE. The ablations cover the main components, and SWIFT also gives a better initialization for reflow.
Critical weaknesses
Major weaknesses:
-
L258–L261 state that "the resulting interpolation paths are straighter." This reads as an empirical conclusion and needs further verification. From the description of the method, SWSD projects each frame, sorts the projected values, and then compares them, so frame correspondence is determined by sorting rather than by time index. It is therefore unclear why an SWSD-based coupling should straighten the Euclidean interpolation paths. Moreover, Table 2 shows that SWIFT does not achieve better straightness than MinibatchOT or LOOM-CFM, and the gap widens at NFE = 2. This challenges the core theoretical claim of SWIFT.
-
According to Sec. 3.1 and 3.2, SWSD's robustness to temporal shifts comes from the sorting and transport in sliced Wasserstein, and plain SW without the time coordinate is already fully invariant to frame permutations. The time coordinate binds frames back to their positions to guarantee injectivity, which makes the distance more sensitive to shifts. Preserving temporal order and tolerating temporal shifts are therefore a trade-off rather than complementary properties, as the paper claims. Their joint benefit should depend on the scale of the time coordinate, but the paper provides no corresponding analysis. I found this part confusing and difficult to follow.
-
The conclusion at L418 (Table 3) does not fully match the authors' claim. With SWIFT, IS is higher in absolute terms, but its drop after reflow is larger. I suggest discussing this in the Limitations section of the appendix to help readers better understand SWIFT. In addition, the paper should include subjective evaluation by human experts to improve the credibility of the generation results. Audio and video generation ultimately serves users, not just metric improvements.
Minor weaknesses:
Some defensive writing in the paper weakens the presentation. The authors could reorganize the logic of certain sections to better highlight the contributions of SWIFT.
Questions for authors
Please review the content in the Critical Weaknesses; if it clears up most of my confusion, I'm willing to raise my score.