acceptodds
Under review as a conference paper at ICLR 2027

Every Sample Speaks Volumes: Steering 3D MRI Generators Under a Measured Contract

Abstract

Controlled experiments in neuroimaging often require volumes that real cohorts cannot provide, such as the same brain with and without a specified degree of Alzheimer's disease (AD) atrophy. Pretrained generators of 3D brain MRI could provide such volumes without retraining. However, inference-time guidance fixes only the direction and strength of a change. It neither specifies the magnitude of the change nor bounds its effect on other structures, and it does not verify the output. We propose a closed-loop controller that obtains from a frozen generator a volume satisfying a verifiable specification, which we call a contract. A contract requests a change in the volume of one anatomical structure and bounds the change of every other planned structure. Both are read by an external segmentation tool, the verifier, in units of its test-retest variability. The controller generates pairs of volumes from the same noise with perturbations of opposite sign. After each pair, it updates a local linear model of the verifier's response with a rank-one correction. It returns a volume only when the contract holds within a fixed budget of pairs. On five released generators from four model families, a single configuration satisfies the contract in 84 to 100% of held-out requests. In contrast, gradient-guidance methods with step sizes fixed in advance satisfy it in at most 28%. An AD classifier built on a second, independent segmentation tool scores volumes steered to the full disease profile above their unsteered counterparts. The procedure also transfers to a frozen physics emulator.

Primary area:
generative models
Keywords:
inference-time steering, closed-loop control, black-box optimization, medical image generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_9gNPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a feedback-based method to control anatomical changes in frozen 3D brain MRI generators. It requests a specific volume change in a target structure while limiting changes to other strucutures.

SynthSeg measures each generated result, and the controller adjusts the intervention until the request is met or the budget is exhausted. Experiments across five generators show improved control accuracy over fixed-strength guidance, although the strict preservation results depending on the segmentation performance.

Critical strengths
  1. The motivation is nice, achieving a specified anatomical change while limiting unwanted changes to other structures.
  2. Using external measurement from segmentation model to provide feedback to correct each sample, making the control more precise and variable.
  3. They evaluate across five released generators thus it supports applicability over different model architectures.
  4. They provide substantial ablations, independent segmentation checks, and anatomical plausibility analysis that help me to understand which component in their model is working.
Critical weaknesses
  1. Their strict preservation results depend heavily on synthSeg. I honestly feel the variation between segmentation model's reliability may be even larger than the request volume change. Table 13 shows that strict preservation transfers poorly to FastSurfer. How should the acceptance thresholds account for disagreement between segmentation tools, and what anatomical precision can reasonably be claimed?
  2. The main baselines use fixed guidance strengths, while their method repeatedly measures and corrects each sample. Please evaluate these guidance methods with comparable per-request feedback and computational budgets to isolate the controller’s contribution.
  3. The image-quality validation is limited, particularly for larger edits. Matching regional volumes does not establish realistic anatomy or biologically valid disease progression. May authors can consider some reader validation or disease target classifiers.
  4. I personally feel this paper's presentation/terminology is unnecessarily difficult to follow, multiple variants, acceptance criteria, also the repeating numbers in the results (such as xxx to 200 examples, xx to xx requests). i feel these are unnecessary repeating in text. Authors can consider keeping more high level analysis in discussion and conclusion.
Questions for authors

Mainly already ask in the weaknesses section. One additional question: What benefit does paired probing provide over the simpler one-sided controller, which achieves similar success with fewer generations?

I currently lean toward borderline, i would reconsider if the authors clarify more on the benefit over simpler controllers or less repeated methods. Also, an improvement over the entire writing.

Reviewer_e9DXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem of quantitatively and reliably steering frozen 3D brain MRI generative models at inference time. While pretrained 3D diffusion and flow-matching models can synthesize anatomical volumes, existing test-time guidance techniques (e.g., DPS, FreeDoM, FlowChef, TFG) merely inject directional gradients scaled by an arbitrary step size. They provide no guarantees on the exact magnitude of the anatomical change, do not constrain off-target anatomical distortion (leakage), and cannot verify the final 3D volume. Retraining or fine-tuning models for specific counterfactual queries is computationally expensive and legally constrained due to medical data-sharing restrictions.

The paper provide a new way to generate synthetic MRI. No model is trained or fine-tuned. Five released generators (MONAI LDM, BrLP, MAISI v2, WaveDiT, MedForj — latent, flow, and pixel-space) are steered at inference time: at each sampler step a small dose is added along a steering direction. The direction is built from the gradients of a small surrogate U-Net (1.46M params, trained per generator on that generator's own samples labeled by SynthSeg) mixed across the six planned structures so predicted side effects cancel. Paired ± probes from the same starting noise measure the true response; a Broyden-corrected local model re-solves the doses; SynthSeg verifies the result.

Critical strengths
  1. the problem is real in the medical image generation.

  2. the method is quite novel and interesting to me.

  3. Seems they find a way to precisely coontrol the generation

Critical weaknesses
  1. The method and the motivation seems not that easy to follow.

  2. The motivation of generate such brain is not valudate in the experiment. What and how you plan to use them in real clinlic cases?

Questions for authors
  1. Seems there is no any radiologist, clinician, or reader study anywhere?

  2. I want to know more about the method details: 1. seems here you train a segmentation model, why do this when we have some exsiting model? 2. How this use to guide the generation? you donot even use the gradient to train the genrator? 3. How can you ensure the other region is not changed by this, how can you precisely control? 4. Seems you treat each region independently? But in practice, how can one region change not cause some (subtle) changes for others?

  3. You get the data, then what? seems you only prove that synthetic brain got the pattern, but never use this for real cases. no experiment trains a model on them, mixes them with real training data, or shows they improve anything on a real task. Can this method used for the real data editing?

  4. Seems this can only handle the visible-pattern?

  5. why put on those six structures, why not put on more fine-grained ROI as getting from the synseg?

  6. different scanner provide quite big domain gap for different data, how the method deal with such issues?

Comments on the paper

1 comment

We thank both reviewers for their careful assessment. We have revised the presentation and added experiments to distinguish the contribution of measured feedback from that of the intervention used to steer the generator. The revisions are marked in blue in the manuscript.

For each request, the controller modifies the sampling trajectory of a frozen generator and uses an external verifier, SynthSeg, to measure the resulting anatomical changes. Measurements on the starting noise of that request update the intervention until the contract is satisfied or the probe budget is exhausted. In the single-structure experiments, the requested change is one test-retest floor, with an error of at most 0.25 floors; changes in the other five planned structures are bounded relative to the measured target change (leak threshold 0.1375, including tolerance). A failed request produces no returned volume and counts as a failure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.