acceptodds
Withdrawn from ICLR 2027

FMS: Unified Flow Matching for Segmentation and Synthesis of Thin Structures

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Segmenting thin structures such as infrastructure cracks and anatomical vessels is hampered by fragile connectivity, expensive annotations, and poor cross-domain generalization. We propose FMS, a flow-matching framework that addresses all three with two modules, SegFlow for imagemask segmentation and SynFlow for maskimage synthesis. SegFlow is an encoder–decoder that casts segmentation as deterministic transport, regressing a velocity field on interpolated image–mask states and integrating from the image itself. Two exact identities describe this sampler: the velocity objective is equivalent to time-weighted endpoint regression on interpolated reference states, and the -step output equals the one-step prediction plus a sum of signed corrections. Together with margin and residual assumptions, these identities characterize possible pixel-label changes and give sufficient conditions for reconnecting two components or recovering the decoded binary mask exactly. SynFlow renders mask-conditioned images using multi-scale mask injection and boundary gating, while class-conditional mask generation and controlled propagation vary sparsity, width, and branching. The same endpoint view applies to SynFlow, where extrapolation drifts by time-weighted trajectory acceleration, giving a conditional stability criterion for a fixed structural image readout already aligned with the conditioning mask. Across five crack and vessel benchmarks, SegFlow improves both overlap and topological metrics over nine baselines: mean IoU versus (+10.1%) and Betti matching error versus (-30.2%), each against the strongest baseline on that metric. Synthetic augmentation improves SegFlow when it receives 25% of the real training labels, recovering most of the full-label performance on two crack subsets and increasing source-only cross-domain IoU by on average. We release code, model checkpoints, and 10k crack and 1k vessel image–mask pairs.

Primary area:
generative models
Keywords:
Flow Matching, Topology-Preserving Segmentation, Mask-Conditioned Image Synthesis, Label-Efficient Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eP6kOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes FMS², a flow-matching framework for thin-structure segmentation and synthesis, targeting connectivity preservation, annotation scarcity, and cross-domain generalization in crack and vessel segmentation. It has two modules: SegFlow, which formulates segmentation as deterministic pixel-space transport from an image to its binary mask by learning velocity fields along image–mask interpolation paths, and SynFlow, which performs mask-conditioned image synthesis through multi-scale mask injection and boundary gating. Experiments on three crack and two vessel datasets show strong overlap and topology performance compared with nine baselines, while synthetic data generated by SynFlow improves partial-label training.

Critical strengths

The paper evaluates SegFlow against nine baselines on five distinct datasets spanning two different domains (infrastructure cracks and anatomical vessels), showing strong empirical performance for the thin-structure segmentation task.

Critical weaknesses
  • The core motivation for formulating segmentation as a multi-step flow matching process is not sufficiently justified. SegFlow employs linear interpolation and uses a constant target velocity along the interpolation path during training. The paper also notes that "an exactly fitted compatible straight path reaches the target in one step". It remains unclear what substantive advantages this flow matching formulation offers over direct image-to-mask regression. Furthermore, as shown in Figure 2, the mIoU on CRACK500 and CrackLS315 at 20 integration steps falls below the corresponding one-step result. Thus, the experiments do not convincingly demonstrate that multi-step flow inference is the primary driver of performance improvement.
  • The paper presents extensive experimental results, but their interpretation remains limited. Moreover, the experimental design does not adequately disentangle the contribution of each design component. The paper also notes that the common-backbone experiment in Table 2 cannot distinguish the effects of supervision and inference, while the SynFlow ablation in Table 4 lacks a baseline variant using neither LSGN nor BGM, making it difficult to quantify the benefit of the full conditioning design.
  • The presentation and formatting are insufficiently polished and below the standard expected for an ICLR paper. For instance, there are layout issues with the caption of Table 5; the frequent use of long sentences and semicolons in the main text hinders readability; and there are obvious proofreading errors in the references, such as "Trans. Mach. Learn. Res., 2025, 2024".
Questions for authors

See Weaknesses.

1 comment

W1. Thank you for raising the attribution question. At t=0t=0, Eq. (5) gives D∗(x0,0)=E[x1∣x0]D^{\ast}(x_0,0)=\mathbb{E}[x_1\mid x_0], the same target as direct endpoint regression, and Eq. (9) shows that later evaluations only add signed endpoint corrections from the learned velocity field. We therefore narrow our claim: flow matching does not guarantee a topological advantage, and multi-step inference is not assumed to drive mIoU. Its value must be shown under matched controls. Fig. 2 already shows step count is non-monotonic (20 steps degrade CRACK500 and CrackLS315), so we now select K on validation (K=10).

We add a same-backbone ladder. All rows share the splits, 2.96M-parameter U-Net, optimizer/schedule, 42k iterations, seed and checkpoint selection (best test step, as in the paper). Values are five-dataset means.

VariantNFEmIoU↑clDice↑Betti↓
Direct regression (MSE)10.5600.74260.08
Direct regression (BCE+Dice)10.5580.73966.87
MSE + TTA100.5650.73860.32
BCE+Dice + TTA100.5660.74359.41
Non-flow iterative refinement + SC100.5750.75361.16
SegFlow, no SC, K=110.5800.75560.38
SegFlow, no SC†100.5850.75961.21
SegFlow, full, K=110.5720.74659.81
SegFlow, full†100.5910.76357.00

TTA averages 8 flips/rotations and 2 shifts. †Retrained here with seed 1 and a fixed K = 10 for every dataset instead of the best K from {5, 10, 20} used in Table 1, so means differ from Tables 1 and 3.

At one evaluation without SC, the FM objective improves mIoU over direct regression by +0.020 (MSE) and +0.022 (BCE+Dice). It beats BCE+Dice on Betti error (60.38 vs. 66.87) but not MSE (60.08). At 10 evaluations, non-flow refinement reaches 0.575/61.16 and the best TTA 0.566/59.41, versus 0.591/57.00 for SegFlow. Integration (K=1 to 10) adds +0.019 mIoU/−2.81 Betti with SC but only +0.005/+0.83 without, and SC adds +0.006/−4.21 at K=10, consistent with Table 3. We also revise the Fig. 2 discussion: K has an interior optimum, excessive integration can accumulate rollout error, and Eq. (18) does not imply its bound decreases with K.

W2. Thank you for your meaningful feedback. Table 2 (mean Betti error 71.348 clCE, 73.738 Skeleton Recall, 51.524 SegFlow) mixes supervision and inference. The W1 ladder now separates them: direct regression vs. SegFlow (no SC, K=1) isolates the objective, K=1 vs. K=10 isolates integration, and Table 3 isolates SC. For Table 4, we trained the missing "neither" variant under the same protocol, giving a full 2×2 ablation:

\hline \lower{1.4ex}{\textbf{LSGN}}&\lower{1.4ex}{\textbf{BGM}}&&\hphantom{0.000}\mathclap{\textbf{C500}}&&&&\hphantom{0.000}\mathclap{\textbf{CT260}}&&\cr &&{\rm mIoU}\uparrow&{\rm F1}\uparrow&{\rm clDice}\uparrow&{\rm Betti}\downarrow&{\rm mIoU}\uparrow&{\rm F1}\uparrow&{\rm clDice}\uparrow&{\rm Betti}\downarrow\cr \hline \times&\times&0.575&0.722&0.789&37.4&0.573&0.714&0.715&98.3\cr \times&\checkmark&0.593&0.737&0.818&30.2&0.567&0.710&0.710&97.2\cr \checkmark&\times&0.581&0.727&0.803&34.2&0.562&0.706&0.707&94.3\cr \checkmark&\checkmark&0.613&0.754&0.835&28.0&0.603&0.739&0.740&88.0\cr \hline \end{array}$$ The full model is best on all eight metrics. Versus "neither", it improves mIoU/F1/clDice by +0.038/+0.032/+0.046 on CRACK500 and +0.030/+0.025/+0.025 on CrackTree260, and lowers Betti error by 25% and 10%. Each component alone lowers Betti error in all four cases, but on CrackTree260 both fall slightly below "neither" in mIoU (−0.006, −0.011). The joint mIoU gain exceeds the sum of the single-component gains (+0.038 vs. +0.024; +0.030 vs. −0.017), so the components are complementary. We therefore replace "together they give the best overlap, topology, and FID" with three supported statements: (i) the full design improves on "neither" on all metrics; (ii) the gain comes from combining both, not either alone; (iii) encoder+decoder mask injection lowers FID vs. decoder-only on all five datasets (Supp. Table S3). **W3.** Thank you for your suggestion and are revising the manuscript. An audit of the main-text prose gives a mean of 19.4 words per sentence and 31.0 for the longest quartile. We will paraphrase long, multi-clause sentences throughout the main text into shorter statements and reduce semicolon use, without changing the claims. Table 5 now has a full-width caption above it and a study-protocol note. We corrected the Boffi et al. TMLR year and audited all references for venue/year/capitalization/URL consistency. **AI-use statement.** We will add the required statement. AI tools were used only for language editing (grammar, phrasing, clarity, LaTeX formatting). The synthetic image–mask pairs are outputs of our SynFlow model, not third-party generative-AI content.
Reviewer_jgTSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

〖FMS〗^2 combines SegFlow, a deterministic image-to-mask flow, with SynFlow, a separately trained mask-conditioned image renderer. SegFlow learns velocities on image-mask interpolations and uses Euler integration with self-conditioning. SynFlow combines multi-scale mask injection, spatial normalization, and boundary gating with generated and propagated masks. Across five crack/vessel benchmarks, SegFlow achieves the best reported mean IoU and Betti matching error. Synthetic augmentation also improves partial-label learning and source-only transfer.

Critical strengths
  1. Coherent formulation for thin structures. Initializing segmentation from the observed image gives a clear alternative to random-mask or latent-space generation. The synthesis branch complements segmentation by varying both geometry and appearance through mask-conditioned training pairs.
  2. Encouraging empirical breadth and gains. The comparison includes nine baselines across three crack and two vessel datasets. Mean IoU improves from the strongest baseline value of 0.546 to 0.601. Reporting overlap, centerline, and spatially matched topology metrics is appropriate.
  3. Useful downstream synthesis evidence. The paper goes beyond visual examples, it includes manual mask-image alignment, conditioning ablations, partial-label experiments, and three source-only transfers. The mean absolute transfer IoU gain of approximately 0.109 is substantial within the reported protocol.
  4. Careful qualification of the theory. The endpoint-correction identities are consistent with the implemented Euler update. The supplement distinguishes algebraic identities from conditional recovery guarantees and provides a counterexample showing why low average error or a straight trajectory alone cannot guarantee correct topology.
Critical weaknesses
  1. The total annotation budget remains unclear. The 25%-label result concerns labels supplied directly to SegFlow. The paper acknowledges additional supervision for SynFlow and the mask generator, but does not state their exact labeled-data budgets in this experiment. End-to-end annotation savings therefore remain unestablished. The transfer gains also combine geometric diversification, image synthesis, and additional training data; matched-volume controls would help identify which source of augmentation produces the benefit.
  2. The theory remains conditional and largely unvalidated empirically. The connectivity certificate uses future trajectory variation, and exact recovery requires samplewise residual and rollout bounds. SynFlow stability additionally assumes an already aligned structural readout and a positive margin. These limitations are stated appropriately, but the relevant quantities are not estimated on the experiments. The analysis provides an interpretation rather than an operational topology guarantee or evidence that its sufficient conditions usually hold.
  3. Formulation and inference advantages are not fully isolated. The common-backbone loss comparison and integration ablations are useful, but do not match supervision, iterative refinement, self-conditioning, and total inference computation simultaneously. A plain endpoint-regression baseline with comparable capacity and evaluation budget would sharpen attribution.
  4. The synthesis-alignment evaluation needs a clearer protocol. Table 5 supports improved alignment over the evaluated diffusion renderers, but does not specify sample count, annotator expertise, blinding, inter-annotator agreement, or uncertainty. The compared renderers also need explicit training-budget and conditioning details. A larger, documented annotation study would make the mask-image fidelity claim more convincing, particularly for one-pixel structures.
Questions for authors

I would suggest to include clearer data and annotation accounting, repeated-run uncertainty, matched-computation controls, and a documented synthesis-alignment study.

1 comment

W1. Thanks and we agree with the concern. In the 25% experiment, SynFlow and the mask generator are trained on the same 25% subset as SegFlow (100/100/25 labeled pairs for CRACK500/CrackTree260/XCAD), so the end-to-end budget is 25%; we now state this. Diversity comes from controlled propagation of those masks. For the transfer gains, please see the mask-statistics table in our response to Reviewer wfmh (W2): propagation moves density and width to the target's regime (e.g., width set to 1 px for CrackTree260), while appearance stays source-only. We did not run matched-volume controls: a new mask needs a newly rendered image, so volume, geometry and synthesis cannot be fully separated.

W2. Thanks for pointing it out. We now estimate these quantities on all five test sets (243 images). They support your reading: the image-level sufficient conditions are not met on real images. SegFlow rows use the seed-1 models of our eP6k response at K = 10 (the no-SC model for Eq. (18)); for Eq. (17), SynFlow generates one image per test mask and the SegFlow trained on that dataset is the readout. Contested pixels are annotated or predicted foreground at some step; a gap closing joins two or more one-step components.

QuantityFive-dataset mean (range)
Eq. (9): pixels changed, K = 1 → 100.38% (0.15–0.87)
Changes ending on the annotated label52.9% (45.4–60.3)
Additions: share of changes / on the label62.6% / 49.3%
Gap closings / openings per image11.72 / 1.62
Closings mostly on annotated foreground62.0% (55.8–66.6)
Eq. (11): contested pixels certified, t = 0 / 0.563.7% / 97.9%
Calibrated margin, t = 0.5: contested certified89.4% (73.6–96.1)
Eq. (18): images meeting the bound; median B0 of 243; 4.79–5.89
Eq. (17): samples with γ > 00 of 243
Readout labels changed after t = 0.50.70% (0.23–1.80)

What holds. Eq. (11) holds by construction; in retrospect it certifies most contested pixels only by t = 0.5. A margin threshold calibrated on held-out images needs no future variation, but it is an empirical test, not a guarantee: it is violated on 8.3% of DRIVE and 7.7% of XCAD test images. The Eq. (9) corrections are nearly balanced pixel by pixel, yet mIoU rises from 0.572 to 0.591: most changes are additions, and because IoU ignores true negatives, additions raise it once more than IoU/(1 + IoU) of them are correct. They also reconnect structures: closings outnumber openings about seven to one.

What does not hold. Eq. (18) is met on no test image (B < 1 required), as expected: it implies exact recovery, and rollout mIoU is 0.585. For Eq. (17), the readout never agrees with the mask at every pixel, so the condition never applies as stated, and late drift correlates with alignment inconsistently (ρ from −0.40 to +0.67).

We will reframe Eqs. (11), (17) and (18) from guarantees to conditional diagnostics, add these estimates to the appendix, and state that the sufficient conditions are not met on real images.

W3. We agree and added the requested controls on the same 2.96M-parameter U-Net, splits, schedule and seed (full ladder in our response to Reviewer eP6k; five-dataset means, mIoU / Betti error). Plain endpoint regression at one evaluation gives 0.560 / 60.08 (MSE) and 0.558 / 66.87 (BCE+Dice), versus 0.580 / 60.38 for SegFlow without SC at K = 1. To match iterative refinement, SC and compute together, we unroll the endpoint regressor for 10 evaluations with SC: it reaches 0.575 / 61.16, versus 0.591 / 57.00 for full SegFlow at K = 10. Regression with 10-view TTA reaches at best 0.566 / 59.41. Within SegFlow, integration from K = 1 to 10 changes Betti error by −2.81 with SC but +0.83 without. So the FM objective improves overlap already at one evaluation (+0.020 mIoU), while the topology gain needs integration with SC, and we no longer attribute it to flow matching alone.

W4. We agree. Table 5 is a small pilot and we will present it as one: 3 masks from real crack photographs, 2 images per mask per renderer (6 per renderer, 24 in total), and one unblinded manual annotation per image, so no inter-annotator agreement or meaningful interval can be reported. Re-scoring the archived annotations reproduces the LDM and SynFlow rows, but gives 0.422/0.586/0.694/3.833 for ControlNet (submitted 0.467/0.632/0.738/8.667) and 0.407/0.568/0.670/5.167 for T2I-Adapter (submitted 0.413/0.574/0.676/5.917) in mIoU/F1/clDice/Betti error. SynFlow (0.561/0.717/0.852/4.583) keeps the highest mIoU, F1 and clDice, and its mIoU is higher on 6/6 images against LDM and ControlNet and 5/6 against T2I-Adapter. ControlNet has the lower Betti error, so "best on all four metrics" was not precise; we will correct the table and text.

Reviewer_wfmhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FMS², a flow-matching framework for thin-structure segmentation and synthesis. SegFlow formulates image-to-mask segmentation as deterministic transport, while SynFlow generates mask-conditioned images using multi-scale mask injection and boundary gating. The authors evaluate the framework on five crack and vessel datasets, reporting improvements in segmentation overlap and topological metrics. They further show that synthetic image–mask pairs can improve label efficiency and cross-domain generalization.

Critical strengths

(1) The paper addresses thin-structure segmentation, where small pixel-level errors can break connectivity. Evaluating both overlap and topological metrics is appropriate and provides a more informative assessment than conventional segmentation metrics alone.

(2) Promising empirical results and a useful synthesis pipeline. SegFlow performs competitively across five crack and vessel datasets, while SynFlow shows potential for improving label efficiency and cross-domain transfer. The combination of segmentation, mask-conditioned synthesis, and topology-focused evaluation could be useful for practical applications.

Critical weaknesses

(1) The contribution of flow matching is not sufficiently isolated. The reported gains may come from a combination of the flow-matching objective, multi-step integration, and self-conditioning. The current ablations do not fully disentangle these factors, and the theoretical results do not establish that flow matching itself provides a general advantage for topology preservation.

(2) The benefits of synthetic data are not fully explained. SynFlow improves label efficiency and cross-domain performance, but the experiments do not clearly separate the effects of additional training data, structural diversity, and appearance diversity. The added training and generation costs also make it difficult to assess the practical advantage over simpler augmentation methods.

Questions for authors

(1) Contribution of flow matching: Under a matched backbone and inference budget, how much of SegFlow's improvement comes from the flow-matching objective itself, rather than multi-step integration and self-conditioning? Which theoretical results support a topology-related advantage beyond the stated endpoint identities and assumptions?

(2) Source of synthetic-data gains: Can the authors disentangle the effects of additional training data, structural diversity, and appearance diversity in SynFlow's label-efficiency and cross-domain results? How does it compare with simpler augmentation methods under a comparable training budget?

1 comment

W1. Thank you for your suggestion. We agree that the submitted ablations isolate integration and self-conditioning (SC) individually but do not fully isolate the flow-matching (FM) objective from a direct endpoint predictor. We now make that separation explicit. Existing Table 3 already quantifies the SC contribution (mean mIoU 0.595 → 0.601; Betti error 55.874 → 51.524), while Fig. 2 isolates the effect of the integration budget and shows it is non-monotonic.

We therefore add matched direct-regression and iterative-refinement controls (the same runs as the ladder in our response to Reviewer eP6k). All rows share the 2.96M-parameter U-Net, splits, optimizer/schedule and 42k iterations; they use seed 1, report means over the five datasets, and select checkpoints as in the paper (best test step). The key comparisons are direct MSE regression vs. SegFlow without SC at K = 1 (the objective at one evaluation) and matched-NFE non-flow refinement vs. SegFlow at K* (benefit under an equal evaluation count).

\begin{array}{lll} \hline \textbf{Comparison} & \textbf{Evidence} & \textbf{Result (mean of five datasets)} \cr \hline \text{Self-conditioning effect} & \text{Submitted Table 3} & \text{mIoU +0.006; Betti −4.350} \cr \text{Integration effect} & \text{Submitted Fig. 2; new runs} & \text{non-monotonic; validation-selected }K^{\ast}=10 \cr \text{FM objective at one evaluation} & \text{direct MSE vs. SegFlow without SC, }K=1 & \text{mIoU 0.560 vs. 0.580; Betti 60.08 vs. 60.38} \cr \text{Matched evaluation budget} & \text{non-flow refinement vs. SegFlow, }K^{\ast}=10 & \text{mIoU 0.575 vs. 0.591; Betti 61.16 vs. 57.00} \cr \hline \end{array}$$ SegFlow is retrained here with seed 1 and a fixed K = 10 for every dataset instead of the best K from {5, 10, 20} used in Table 1, so its means (0.591 / 57.00) differ from Table 1 (0.601 / 51.524). Under a matched backbone and evaluation budget, the FM objective accounts for most of the overlap gain. At one evaluation without SC, it raises mean mIoU from 0.560 (direct MSE) to 0.580, about two-thirds of the way to the full model (0.591). Integration (K = 1 → 10) then adds +0.005 mIoU without SC but +0.019 with SC, and SC adds +0.006 at K = 10. Topology behaves differently: the objective alone does not reduce Betti error (60.38 vs. 60.08), and the gain appears mainly when integration and SC are combined (−2.81 from integration with SC but +0.83 without it; −4.21 from SC at K = 10). At a matched 10 evaluations, SegFlow improves on non-flow refinement by +0.016 mIoU and −4.16 Betti error. We therefore do not claim a general topology advantage from flow matching itself. The paper already states that exact recovery requires margin/residual/rollout conditions, and Supplement Sec. A.4 gives a one-pixel counterexample in which a straight trajectory with low average error still has incorrect connectivity. The theory is diagnostic and conditional: Eq. (9) identifies the corrections made beyond the one-step endpoint, Eq. (11) gives a retrospective margin certificate for where labels can change, and Eq. (18) gives sufficient conditions for exact mask recovery. We will revise the wording so these are not presented as an unconditional superiority theorem, and state the empirical conclusion directly from the matched controls: the FM objective improves overlap already at one evaluation, whereas the topological gain requires integration with SC. **W2.** We agree that the paper does not isolate geometry from appearance or volume. A full factorial is impossible: a new mask needs a newly rendered image, and across real crack datasets shape and texture change together. Appearance is fixed by design: SynFlow is trained only on the source set (transfer) or the 25% subset (label efficiency), so it adds no target or held-out appearance. What it adds is mask structure (means per mask): | Mask set | Density (%) | Branch pts | Skeleton wider than 2 px (%) | |---|---|---|---| | C500 real | 2.76 | 5.0 | 45.1 | | C500 generated | 2.83 | 4.1 | 53.8 | | CT260 real | 0.84 | 2.7 | 0.0 | | CT260 generated | 0.90 | 3.1 | 0.0 | | CLS315 real | 0.56 | 1.1 | 0.0 | | C500→CT260 set | 0.80 | 9.5 | 0.0 | | CT260→C500 set | 2.38 | 2.6 | 41.7 | | CLS315→C500 set | 2.29 | 1.6 | 60.9 | Generated masks are new structures: median F1 (2-px tolerance) to the nearest training mask is 0.48 on C500 and 0.28 on CT260, against 0.45 and 0.28 for held-out real masks. Width is the dominant gap (1-px annotation in CT260/CLS315, true width in C500). In all three transfer sets, propagation moves density and width to the target's regime; branch points stay closer to the source. Classic augmentation also adds samples but transforms image and mask together, so it cannot change width: it adds +0.006 mIoU against +0.109 for SynFlow (Table S6). On cost, SynFlow is trained once per dataset (150k iterations, two A100 GPUs), generation is offline and SegFlow inference is unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.