acceptodds
Withdrawn from ICLR 2027

Knowledge-Guided Multi-Task Diffusion with Type-Aware Boundary Modeling for Lesion Synthesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Annotated pathological CT data remain scarce, limiting the generalization of lesion segmentation models. Diffusion-based synthesis offers scalable data augmentation, but mask- or prompt-conditioned generation alone does not enforce measurable radiological properties or realistic lesion–tissue transitions; joint generation and segmentation can also produce conflicting gradients. We propose a knowledge-guided lesion synthesis framework with two complementary components. Knowledge-guided multi-task lesion synthesis (KG-MTS) incorporates CT priors as differentiable constraints on lesion density, tissue composition, and internal heterogeneity, and introduces auxiliary segmentation through asymmetric conflict-aware weighting that attenuates harmful segmentation updates while preserving the generation direction. Type-aware boundary transition modeling (TA-BTM) represents the lesion boundary as a phenotype-dependent intensity transition zone and aligns its signed contrast, effective width, and smoothness with real-lesion statistics. Given a CT region of interest, a prescribed mask, and a structured radiological prompt, the model synthesizes controllable lesions while preserving the surrounding anatomy. We evaluate pulmonary nodule and renal tumor synthesis on LIDC-IDRI and KiTS23 using three downstream segmentation architectures. Compared with the reproduced LeFusion baseline, our method achieves Dice and NSD improvements of 2.80–3.20% and 2.04–2.38% on LIDC-IDRI, and 1.73–3.05% and 1.31–1.99% on KiTS23, respectively. Ablations further show that KG-MTS and TA-BTM provide complementary improvements in volumetric overlap and boundary agreement. Code: https://anonymous.4open.science/r/lesion-synthesis-review-1C7D

Primary area:
generative models
Keywords:
Lesion Synthesis, Diffusion Models, Knowledge-Guided Generation, Multi-Task Learning, Type-Aware Boundary Modeling, Medical Image Segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 5 reviews, above 27% of the others.
Reviewer_MM27OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

To address the high cost of acquiring pathology data with voxel-level annotations, this paper discusses existing synthesis methods and proposes a Knowledge-Guided Multi-Task Lesion Synthesis (KG-MTS) approach that incorporates diverse detailed information. Additionally, it introduces a technique to mitigate the issue of boundary and evaluates the method using three downstream segmentation network architectures.

Critical strengths

None

Critical weaknesses

The paper requires a comparative analysis of failed synthetic cases. Methods like Repaint and LeFusion are sensitive to background distribution; therefore, it is necessary to report synthetic results and showcase failure cases stratified by background intensity, as well as to present a few clear, explicit examples of background selection.

Questions for authors

To further validate the effectiveness of the various modules, the authors are encouraged to provide examples of generated results and failure cases for the ‘LeFusion + KG-MTS + TA-BTM’ configuration. Additionally, it is recommended to include examples of lesions synthesized with varying degrees of intensity against different background conditions (e.g., the specific appearance of lung nodules synthesized within low-density or clear, dark regions of the lung parenchyma).

Reviewer_bV2BOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a diffusion-based framework for synthesizing CT lesions to augment segmentation training data. Starting from a lesion-focused, prompt-conditioned diffusion model, it adds two training-time components. KG-MTS penalizes deviations of the predicted lesion's mean HU, solid-component ratio, and local variance from prompt-specific intervals estimated from real lesions, and adds an auxiliary segmentation branch whose loss is down-weighted when its gradient conflicts with the generation gradient. TA-BTM applies analogous interval penalties to the signed contrast, width, and smoothness of the lesion–tissue boundary transition. On ROI-level pulmonary nodule (LIDC-IDRI) and renal tumor (KiTS23) segmentation with three downstream architectures, the authors report Dice and NSD gains over a reproduced LeFusion baseline and other synthesis methods, and conclude that the two components provide complementary improvements.

Critical strengths

1. Consistent downstream performance improvements across architectures and organs, with an informative ablation.

  • The method improves over the reproduced LeFusion baseline on all three downstream segmenters and both datasets: Dice gains are 2.80–3.20 points on LIDC-IDRI and 1.73–3.05 points on KiTS23 (Tables 1–2).
  • The LIDC-IDRI split is patient-level (Appendix A), and all prior intervals used in training are estimated from the training set only (Sec. 3.4).
  • The module ablation (Table 3) shows an interpretable pattern: KG-MTS mainly improves Dice, TA-BTM mainly improves NSD, and the full model is best on both datasets.

This is consequential because it indicates that the augmentation benefit is not specific to one segmentation architecture or one organ.

2. A physically grounded and well-specified formulation.

  • The descriptors are defined in HU and millimeters, with spacing-aware boundary rings and finite differences (Sec. 3.3, Appendix A).
  • The interval loss penalizes only values outside real-lesion ranges, which in principle preserves intra-class variation (Eq. 2).

This makes the radiological constraints interpretable and transferable across scans with different voxel spacing.

Critical weaknesses

1. The evidence for the main claims lacks statistical support, and the realism evaluation is largely circular.

  • No variance is reported: Each configuration is run three times, but only means are given, and the paper does not state what varies across runs (synthesis seed, segmenter seed, or both) (Appendix A). The KiTS23 test set is an internal partition of only 58 ROIs from 49 cases, small relative to Dice differences of 1.7–3.1 points.
  • The descriptor evaluation measures what the method optimizes: KG-MTS directly penalizes deviations of mean HU, solid ratio, and local variance from real-lesion intervals, and NID (Table 4) then scores the same three descriptors against intervals computed on the held-out test lesions. The same holds for TA-BTM and the boundary descriptors. The reductions (e.g., medical NID from 0.669 to 0.049 on LIDC-IDRI) are therefore expected by construction, and they do not provide independent evidence of realism.
  • Diversity is not measured: NID assigns zero penalty to any value inside the interval, so a generator that collapses toward a typical lesion per prompt group would score perfectly. No diversity or distribution-level measure (e.g., a feature-space distance, a two-sample classifier, or a reader study) is reported.
  • Table 4 is incomplete: It omits the full model and all baselines, including LeFusion. Appendix A describes the descriptor protocol for LIDC-IDRI only, although KiTS23 results are reported.
  • Key augmentation settings are unreported: The number of synthetic ROIs (N) and the synthetic-to-real ratio are not given (Appendix A), nor is the source of the prescribed masks and prompts used for synthetic samples.

2. The attribution of performance gains to the proposed mechanisms is incomplete, and relevant baselines are missing.

  • KG-MTS bundles three mechanisms that are never separated: The medical prior loss, the auxiliary segmentation branch, and the conflict-aware weighting are ablated only as a single module (Table 3). The claim that asymmetric weighting "attenuates harmful segmentation updates" is therefore untested. The paper also does not report how often the gradient cosine is negative, or compare against a fixed segmentation weight or gradient projection (PCGrad).
  • The base model is weaker than LeFusion: The prompt-conditioned base model (Sec. 3.1), on which both components are built, reaches 80.96/89.09 Dice/NSD with MedNeXt on LIDC-IDRI, against 81.43/92.01 for LeFusion, and 80.34/86.60 vs. 80.60/89.34 on KiTS23 (Tables 1–3). It remains unclear whether adding KG-MTS and TA-BTM directly to LeFusion would yield the same or larger gains.
  • Relevant attribute-controlled baselines are absent: The paper cites related conditioning strategies, namely text-controlled synthesis (LesionDiffusion; Lei et al., 2025), radiomics-conditioned synthesis (Kim et al., 2025), and phenotype-prompted synthesis (LesionLab; Lyu et al., 2026), but compares against none of them.
  • Baseline adaptations are not described: Hand-Crafted (designed for liver tumors) and Copy-Paste fall below Real Only for all three architectures on LIDC-IDRI. DiffTumor (designed for abdominal tumors) is below Real Only in NSD for every architecture on both datasets. The paper does not describe how these methods were adapted to 32×64×64 lesion ROIs, so their weakness may reflect the adaptation rather than the methods.
  • The evaluation is restricted to lesion ROIs: Downstream segmentation operates on ROIs extracted around known lesions (centered on the lesion for KiTS23), which removes the localization problem. The paper acknowledges this limitation, but it narrows the practical significance of the reported performance gains.
Questions for authors
  1. Variance: What varies across the three runs? Please report standard deviations and a paired test on per-ROI Dice and NSD against LeFusion, particularly for the 58-ROI KiTS23 test set.
  2. Independent realism and diversity: Can the authors report realism and diversity measures that are not optimized during training, such as a feature-space distance between synthetic and real lesions, a two-sample classifier, intra-group diversity, or a radiologist reader study? Please also add the full model and LeFusion to Table 4, and describe the KiTS23 descriptor protocol.
  3. Components of KG-MTS: What are the results with the medical loss alone, with the segmentation branch at a fixed weight, and with PCGrad in place of the asymmetric weighting? How often is the gradient cosine negative during training, and what is the added cost of computing both gradients?
  4. Augmentation setup: What are N and the synthetic-to-real ratio? Where do the prescribed masks and prompts for synthetic samples come from?
  5. Base model and baselines: Why does the prompt-conditioned base model underperform LeFusion, and what is the result of adding KG-MTS and TA-BTM to LeFusion directly? How were Hand-Crafted and DiffTumor adapted to the lesion ROIs?
  6. Attribute-controlled methods: How does the method compare with LesionDiffusion, radiomics-conditioned synthesis, or LesionLab under the same downstream protocol?
Reviewer_HRtgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a knowledge-guided diffusion framework for synthesizing controllable pulmonary nodules and renal tumors in CT images to improve lesion segmentation under limited annotated data. It introduces two complementary components: Knowledge-Guided Multi-Task Synthesis (KG-MTS), which incorporates differentiable constraints on lesion density, tissue composition, and internal heterogeneity while using gradient conflict-aware weighting for auxiliary segmentation supervision; and Type-Aware Boundary Transition Modeling (TA-BTM), which regularizes lesion–tissue interfaces using phenotype-specific contrast, transition width, and smoothness statistics. The framework is evaluated on LIDC-IDRI and KiTS23 against several synthesis baselines using nnU-Net, SwinUNETR, and MedNeXt. Results show consistent improvements over LeFusion in downstream Dice and normalized surface Dice across both datasets, while ablation studies and descriptor-based evaluations support the complementary contributions of the two components. The evaluation is limited to ROI-based segmentation with prescribed lesion masks and does not establish full-volume localization or zero-shot cross-organ generalization.

Critical strengths
  • Consistent downstream segmentation improvements across datasets and architectures. Tables 1–2 report improvements over LeFusion on both LIDC-IDRI and KiTS23 using nnU-Net, SwinUNETR, and MedNeXt. This cross-dataset and cross-architecture consistency provides meaningful empirical evidence that the proposed synthesis framework may benefit downstream segmentation, rather than improving results only under a single evaluation configuration.

  • A coherent integration of radiological priors and boundary-aware synthesis. Sections 3.2–3.3 introduce differentiable constraints on lesion appearance and phenotype-specific boundary transitions, while Table 3 provides ablations suggesting complementary contributions from KG-MTS and TA-BTM. This design addresses both internal lesion characteristics and lesion–tissue interfaces. However, the independent contribution of the conflict-aware weighting mechanism and the full model's descriptor-level realism remain insufficiently validated (Table 4).

Critical weaknesses
  • Insufficient validation of the central lesion-realism claims. Table 4 evaluates Prompt-only and the two modules independently but omits descriptor-level results for the full model. Moreover, the NID metric in Appendix A (pp. 13–14) penalizes values outside percentile-based intervals but does not establish distributional fidelity, preservation of lesion diversity, or joint consistency among descriptors. Consequently, the evidence does not adequately substantiate the claim that the combined framework produces more realistic lesions, which is central to the paper’s motivation.

  • Insufficient experimental isolation and reproducibility of the claimed methodological contributions. Table 3 ablates KG-MTS and TA-BTM as complete modules but does not isolate the contribution of KG-MTS’s conflict-aware gradient weighting (Eqs. 4–5, Sec. 3.2.2). Appendix A (p. 13) also denotes the number of synthetic training ROIs as NN without specifying its actual value, while Tables 1–3 report means over three runs without variability estimates. These omissions prevent clear attribution of the reported gains to the proposed mechanism and make the experimental results difficult to independently reproduce and assess for robustness.

Questions for authors

Clarification questions

  • Diffusion loss implementation: In Eq. (1), the mask is applied to the target noise but not explicitly to the predicted noise or the prediction error, although Sec. 3.1 states that denoising is optimized only inside the lesion mask. Is this equation consistent with the actual implementation? If the implemented loss differs, could the authors provide the exact formula? This clarification materially affects the interpretation of the generation objective.

  • Descriptor-level validation of the full model: Why does Table 4 omit the full model while reporting Prompt-only, KG-MTS-only, and TA-BTM-only variants? Could the authors report all six descriptor-level NID values for the full model on both datasets, using the same evaluation protocol? These results could materially affect the assessment of whether the two components jointly improve lesion realism.

  • Reproducibility and data separation: Appendix A (p. 13) denotes the number of synthetic training ROIs as NN without specifying its value. Could the authors report the exact synthetic sample counts and clarify whether all normal background ROIs used for synthesis are restricted to training cases, particularly for KiTS23? These details are necessary to verify experimental reproducibility and exclude potential overlap between training backgrounds and held-out cases.

  • Independent contribution of conflict-aware weighting: Can the authors provide an ablation comparing the proposed adaptive weighting with fixed auxiliary segmentation weights, while keeping the remaining KG-MTS components and training conditions unchanged? Table 3 evaluates KG-MTS as a whole, so the current evidence does not isolate the contribution of the mechanism defined in Eqs. (4)–(5). A controlled comparison could materially change the assessment of the method's technical novelty.

Suggestions for future work

  • Broader validation of lesion realism: Could future evaluations supplement NID with distribution-level comparisons and independent expert assessment of synthesized lesions? The current interval-based metric in Appendix A (pp. 13–14) does not establish full distributional fidelity or clinical realism. Such validation would help determine whether improvements in the selected descriptors translate into more realistic synthetic CT lesions.
Reviewer_9Dr8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a diffusion framework for CT lesion synthesis using radiological constraints and type-aware boundary modelling. KG-MTS combines intensity, composition, and heterogeneity priors with auxiliary segmentation and gradient-conflict weighting. TA-BTM constrains lesion–tissue transitions using real-lesion statistics. Experiments on LIDC-IDRI and KiTS23 show improved downstream segmentation across three architectures.

Critical strengths
  1. The approach is well motivated, which connects synthesis objectives to measurable radiological properties rather than relying only on text or mask conditioning.
  2. Boundary modelling accounts for physical voxel spacing and lesion phenotype.
  3. Evaluation is comprehensive, which includes two lesion types, three downstream architectures, and matched synthetic sample counts.
  4. The paper clearly acknowledges that its results concern ROI segmentation rather than full-volume localization.
Critical weaknesses
  1. The main components need more detailed ablations. KG-MTS combines medical priors, auxiliary segmentation, and conflict-aware weighting. Separate these contributions and compare the weighting rule with fixed weighting and established gradient-conflict methods.
  2. The qualification on gradient-protection is needed, as attenuating a conflicting auxiliary gradient does not guarantee that the combined update preserves or improves the generation objective. Report gradient norms, conflict frequency, and actual effects on synthesis quality.
  3. Important reproducibility details seem to be missing. Specify the synthetic sample count N, training duration, learning rates, prior intervals, loss coefficients, timestep weights, boundary-ring widths, and the tolerance of NSD.
  4. Some statistical evidences are missing. Tables report means from multiple seeds without standard deviations or confidence intervals. Provide paired comparisons and account for multiple ROIs from the same patient.
  5. The realism evaluation closely follows the training losses. Lower NID demonstrates descriptor compliance but does not establish perceptual realism or diversity. Add independent radiologist assessment, comparisons at distribution level, and the full model’s descriptor results, if possible.
  6. Background and mask provenance needs clarification. For example, confirm that all normal ROIs, prescribed masks, and synthesis inputs come exclusively from training patients, especially the renal backgrounds extracted from non-tumour regions.
  7. The coarse renal resolution needs justification. For instance, resampling to 5.0 × 2.5 × 2.5 mm may obscure small lesions and boundary transitions. Test sensitivity to resolution and explain handling of lesions with empty eroded cores.
  8. Baseline fairness needs more documentation, like explaining how each generator was adapted, tuned, and trained. Include stronger conventional augmentation and report synthesis time and additional training cost.
Questions for authors

N/A

Reviewer_K4mtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies controllable CT lesion synthesis for improving downstream segmentation through synthetic data augmentation. The authors argue that conventional mask- or prompt-conditioned diffusion models do not sufficiently enforce clinically meaningful lesion characteristics or realistic lesion–tissue transitions. The proposed framework introduces two components. KG-MTS incorporates differentiable radiological constraints on lesion density, tissue composition, and internal heterogeneity, together with auxiliary segmentation supervision regulated by conflict-aware gradient weighting. TA-BTM explicitly models phenotype-dependent boundary transitions using signed contrast, effective width, and smoothness statistics. Experiments on LIDC-IDRI and KiTS23 demonstrate consistent improvements over the reproduced LeFusion baseline across three segmentation architectures. Module ablations and descriptor-consistency evaluations further support the complementary roles of the proposed components.

Critical strengths
  1. A meaningful problem formulation with a clear methodological insight. The paper identifies a relevant limitation of mask-conditioned lesion synthesis: spatially accurate generation does not necessarily guarantee realistic internal radiological properties or lesion–tissue interfaces. Explicitly constraining measurable radiological characteristics and modeling phenotype-dependent boundary transitions provides a coherent and clinically motivated approach. The distinction between boundary geometry and boundary intensity transitions is particularly valuable.
  2. Consistent downstream gains supported by informative ablations. The method improves both Dice and NSD over LeFusion on LIDC-IDRI and KiTS23 across nnU-Net, SwinUNETR, and MedNeXt, the ablation results suggest complementary effects.
Critical weaknesses
  1. The contribution of the proposed objectives is not fully isolated from additional training. The method uses a two-stage training procedure, where the proposed objectives are activated after prompt-conditioned pretraining. However, the paper does not clearly establish whether the Prompt-only baseline receives an equivalent amount of continued training. Consequently, part of the observed improvement might be explained by additional optimization rather than the proposed guidance mechanisms.
  2. The effectiveness of conflict-aware segmentation weighting is insufficiently isolated. KG-MTS combines medical knowledge constraints, auxiliary segmentation supervision, and conflict-aware weighting, but the individual effects of these elements are not reported separately. Moreover, the scalar attenuation rule reduces conflicting auxiliary gradients without necessarily eliminating their adverse effects on the primary optimization direction. Thus, the evidence supports the usefulness of KG-MTS as a whole more strongly than the specific necessity of its conflict-aware mechanism.
Questions for authors

Q1. Does the Prompt-only baseline receive the same total number of optimization updates as the full method, including the second training stage? How does the total training budget compare with the reproduced LeFusion baseline? A budget-matched continued-training control would help determine how much improvement comes from the proposed objectives. Q2. Can the authors separate the effects of medical knowledge constraints, standard auxiliary segmentation supervision, and conflict-aware weighting? In particular, does the adaptive weighting provide a meaningful improvement over a fixed segmentation loss coefficient under matched training conditions? Q3. Are all normal/background ROIs used for synthetic training, particularly the 507 normal KiTS23 ROIs, extracted exclusively from training cases? Explicit confirmation of patient/case-level separation would help rule out possible contamination between synthesis training and downstream evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.