acceptodds
Under review as a conference paper at ICLR 2027

A Fixed-Point Algorithm for Constraining Pre-trained Diffusion Models

Abstract

Three-dimensional diffusion models have emerged as a powerful tool to extract priors from large geometric datasets. However, exploiting these priors in generative high-precision applications (e.g., the design of medical prosthetics) requires a strategy for enforcing hard constraints on each generated shape. Unfortunately, we find that existing algorithms to impose these constraints on pre-trained diffusion models are inadequate for a large class of 3D generative tasks, as they either modify the denoised generation, risking escaping the data distribution, or rely on guiding the noisy reverse diffusion trajectory, very often failing to satisfy the constraints exactly. We introduce a fixed-point strategy for modifying the reverse diffusion trajectory without escaping the data distribution, improving the tradeoff between plausibility and feasibility of the generations. We showcase our method's performance on the 3D generative tasks that motivate its design, on which it outperforms every considered baseline, as well as on common 2D and multi-dimensional benchmarks.

Primary area:
generative models
Keywords:
Generative diffusion models, Constraint satisfaction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_1rbDOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a fixed-point sampler for constraining pretrained diffusion models. It feeds the difference between a denoised prediction and its constraint projection back into the noisy state, then denoises and corrects again to balance constraint adherence with plausible shape. The authors provide a conditional convergence analysis and evaluate the method on point-cloud completion and anatomical reconstruction, among other tasks.

Critical strengths
  1. Feeding constraint corrections into the noisy state is a simple and practical design. Denoising again after each correction lets the pretrained model respond to the change while continuing to supply a shape prior. The update uses the projection residual without retraining or backpropagating through the denoiser, making it convenient to apply existing models to different geometric constraints.

  2. The complete sampler achieves competitive constraint adherence and shape quality. Point-cloud completion and the anatomical reconstruction tasks cover several geometric constraints, with comparisons against multiple families of constrained samplers supporting the empirical advantage. These results establish practical value across different constraint tasks.

Critical weaknesses
  1. The central fixed-point inner loop is not sufficiently justified over a simpler sampler. In Supplementary Tables 13 and 23, removing the loop while retaining the same sampling process and projection at the end of each step preserves much of the performance on point-cloud completion and aorta reconstruction. Computation falls substantially for point-cloud completion. The larger skull improvement is encouraging, but that ablation covers one case. The complete sampler's advantage over other methods therefore does not establish that fixed-point iteration itself provides a substantial advance commensurate with its extra computation.

  2. The geometric feasibility evaluation does not establish the central claim of exact hard-constraint satisfaction. The main text presents exact feasibility as a key advantage over existing methods, but the pass criteria in Appendix A.6.2 use the 95th percentile of observation-point distances and can leave individual coverage violations undetected. Appendix A.3 provides conditional analysis for approximate projections, without establishing that the practical correction operators satisfy those conditions. The results show improved adherence, but do not verify the strict feasibility emphasized in the paper.

Questions for authors
  1. For which constraints does the inner loop provide consistent gains that justify its cost over the version retaining projection at the end of each step? Does the larger benefit observed in the skull case hold across other subjects?

  2. For geometric outputs reported as feasible, are the task's coverage constraints satisfied when all observation points are checked? Do the worst-point residuals or the fraction of violating points support the exact-feasibility claim?

2 comments

We thank the reviewer for their thoughtful feedback. We respond extensively to their comments on hard constraint satisfaction and the need for an inner loop in the general response to all reviewers (see general response above).

Thank the author for the additional comparisons. The equal-denoiser-call aorta experiment gives a concrete justification for the inner loop. It improves reported feasibility and worst-case violation even though K=0 achieves lower Chamfer error. This substantially reduces my concern about its added computational cost. Also, the pointwise-violation and whole-shape feasibility results also clarify what is achieved in practice. They support improved constraint adherence within application tolerances, rather than a guarantee that every generated shape is feasible. These additions address most of my original concerns. I'm now willing to raise my score accordingly.

Reviewer_zKyaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a sampling-time method to impose constraint to a frozen diffusion model, by applying the denoised projection residual to the noisy state, and repeat this process. Authors show empirical results on several tasks that show improved geometric reconstruction and constraint-adherence metrics on point cloud completion and anatomical tasks.

Critical strengths

Design and mechanism is simple and general. The proposed method is not using model specific designs or algorithms, making it generally applicable to a family of generative models. It avoids model training and accomodates different constraint without gradient-based optimization.

Critical weaknesses

Limited experiment scope. Despite showing promising results on the reported fictional aortic graft design task, and some 3D reconstruction results in the appendix, it does not seem to demonstrate experimental results on a broader range of 3D reconstruction. The used cases are mostly simple and not in a way that can be persuasive to the 3D community.

Questions for authors
  1. For the geometric tasks, what fraction of samples satisfies the actual specified constraint? It is a very different perspective to show maximum violations compared to a percentile based metric.
4 comments

We thank the reviewer for the valuable feedback.

Q1: Hard constraint satisfaction

See general response above

Q5: Selected tasks

Our algorithm is not only inspired by anatomical problems; the problems we consider are well-known, common benchmarks in medical computer vision that relate to real-world applications and (in most cases) use real patient data.

Our vertebra reconstruction task is often covered on medical imaging and computer vision venues (some recent examples: MICCAI Wysocki, Magdalena, et al. "Ultron: Ultrasound occupancy networks" 2025, ECCV “UBone3D: Physics-Rectified Conditional Flow Matching for Anatomical 3D Shape Completion from Ultrasound” 2026). On this task we additionally compare with a specialized method (RocoSDF) alongside other constraint diffusion baselines.

Our aorta implant generation comes from a real task setting where the implant needs to be manufactured [Zhai et al., 2026]. This surgery is needed for aortic aneurysm repair that affects around 30 million people worldwide every year. The data we used for training [Bǒsnjak et al., 2022] is also clinically established.

The cranial implant generation is a real world task and implants are being actively generated and produced for surgeries (“Custom-made 3D printing-based cranioplasty using a silicone mould and PMMA”, “Design and mechanical evaluation of a large cranial implant and fixation parts”). The generation of implants is of interest to the medical computer vision community with datasets like SkullFix (the one we used in the paper) and SkullBreak.

We hope that our work inspires others to pursue real-world applicable benchmarks. For a more general 3D generation we used the ShapeNet dataset. Our goal was to isolate the effect of the suggested method on the diffusion model denoising process without adding extra cofounders that the most recent 3D generation models can introduce with more sophisticated architecture. We are very open to add additional experiments that would be persuasive to the 3D community and would be glad to hear the reviewer’s suggestions.

Our revised manuscript will remove the word “fictional” from the caption of Fig. 4. This (admittedly poor) choice of words was meant to indicate that the (real) anatomical data corresponds to healthy patients that would not ordinarily need an aortic repair surgery.

Dear reviewer, we were wondering if you've had a chance to take a look at our new experiments and promised changes based on your feedback. We really think that the main weaknesses have been addressed (as agreed by another reviewer), and we have fixed the unfortunate wording choice referenced in your review. We hope we were able to sway you into agreeing with the acceptance advocated for by the rest of reviewers, but if we haven't, please feel free to suggest any changes or additional questions we can answer in the few discussion hours left.

Thank you for your clarifications and explanations. I lean towards maintaining my original rating, as there is no further concrete evidence to reassess the experimental scope and general significance. The proposed method should have been evaluated on a broader types of tasks, but I imagine this is infeasible due to limited discussion window.

Dear reviewer, thank you for responding. As we said in our response last week, “We are very open to add additional experiments that would be persuasive to the 3D community and would be glad to hear the reviewer’s suggestions.”

Which additional tasks would you like to see, beyond the three anatomical ones, the 3D reconstruction one and the image based and mathematical baselines already included in the paper?

Indeed it would have been easier to include whichever experiment you request if we had had this conversation earlier in the discussion window, but we still want to produce the best possible version of the paper and would love to know what you think is missing.

Reviewer_sRaBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to impose geometric constraints on a pretrained diffusion model without retraining it. It connects projection of the denoised prediction to a fixed-point update of the noisy state, with a conditional contraction argument and extensions to damped and approximate projections. Experiments cover point-cloud completion, anatomical reconstruction, image inverse problems, motion, and synthetic distributions. The supplementary material includes calibration of the noise update, controls separating the corrector components, and detailed baseline configurations. The results support a useful constraint-satisfaction method, although the benefit of the inner loop varies across tasks.

Critical strengths
  • The paper addresses a consequential distinction between satisfying a geometric constraint and merely encouraging it through a guidance loss. The formulation connects a clean-space correction to a simple noisy-state update without requiring backpropagation through the denoiser.

  • The empirical evidence is broader and more diagnostic than the main application tables alone suggest. The geometric results show favorable feasibility/accuracy trade-offs, while Table 20 combines high feasibility with relatively good distributional fidelity on the synthetic tasks. Tables 13 and 22-24 separately examine inner corrections, the final projection, noise settings, and residual handling. These controls substantially strengthen the submission by identifying where the different components matter, rather than attributing every gain to the fixed-point formulation.

Critical weaknesses
  • The incremental value of the inner loop is not consistently compelling relative to its cost. Table 13 already provides the relevant K=0 control: F1 improves from 0.254 to 0.269 and fit from 0.0050 to 0.0043, but runtime increases from 16.0 to 106.9 seconds. Table 23 similarly shows aortic Chamfer of 3.11 without inner updates versus 3.10 for the full method, although the skull screening case shows a much larger gain. The supplementary material therefore answers whether these ablations exist; the remaining concern is the strength and scope of the additional contribution they establish. Section A.7 explicitly states that the comparisons are not compute matched. A budget-controlled comparison would clarify when fixed-point iterations are preferable to the simpler correction scheme, rather than primarily offering a more expensive operating point.

  • The practical feasibility evidence needs to be distinguished from strict satisfaction of the formulated constraints. Appendix A.6.2 carefully discloses that the anatomical correction operators are approximate and that the reported criteria use percentile gaps and tolerances. In vertebral reconstruction, the correction targets a maximum-gap condition, while the reported pass criterion uses a 95th-percentile gap; the appendix explicitly notes that passing does not imply coverage of every observation. The correction tolerance is also set using distances to the known CT surface. These are reasonable benchmark choices, but they limit what a reported feasibility rate of 1.00 establishes about hard constraints or deployment with observations alone. This does not invalidate the reconstruction improvement; it narrows the practical interpretation of the central constraint-satisfaction claim.

Questions for authors
  • At a matched wall-clock budget, how do the full method and K=0 compare on point-cloud completion and the anatomical tasks? Is there a practical rule for selecting the inner-loop budget given the uneven gains in Tables 13 and 23?
  • For vertebral reconstruction, what fraction of samples satisfies the actual maximum-gap target, rather than only the percentile-based evaluation criterion? How would the correction tolerance be specified when the reference CT surface used in the benchmark is unavailable?
1 comment

We thank the reviewer for their thoughtful feedback and proposed experiments.

Q1: Hard constraint satisfaction

See general response above

Q2: Renoising without fixed-point iterations

See general response above

Q3: Budget matching comparison

We agree with the reviewer that this is an important and useful experiment. We report equal-number-of-denoiser-call comparisons with measured runtimes. We will add this table along with a discussion to the paper.

TaskConfigurationDenoiser callsTimeResult
ShapeNet, per shape: F1 ↑ / DCD ↓ / fit ↓K=3, 32 steps (submitted configuration, re-timed)12873 s0.269 / 0.668 / 0.0043
K=0, 32 steps3222 s0.254 / 0.662 / 0.0039
K=0, 128 steps12883 s0.277 / 0.658 / 0.0044
Aorta, per case: Chamfer / reconstructed-segment Chamfer (mm)K=3, 32 steps (submitted configuration, re-run)12832 s3.10 / 6.48
K=0, 32 steps3212 s3.10 / 6.83
K=1, 64 steps12836 s2.82 / 5.18
K=0, 128 steps12846 s2.77 / 5.08
Vertebra, per subject: Chamfer (mm)K=3, 1,000 steps (as submitted)4,0001,216 s1.16
K=3, 250 steps1,000302 s1.34
K=0, 1,000 steps1,000336 s1.26
Skull: Chamfer (mm)K=3, 1,000 steps (as submitted)4,0003.9×2.81
K=0, 1,000 steps1,0001×2.75

In a practical setting, the choice of K can be informed by the amount to which feasibility and plausibility are at odds with one another. If the projection step (enforcing feasibility) greatly disrupts the plausibility of the sample, a higher value of K is appropriate. We will add a sentence to this effect when K is introduced.

Q4: Vertebra reconstruction

The per-subject tolerance requires a reference CT and therefore cannot be selected in the same way when that reference is unavailable. To test sensitivity to this choice, we repeated the experiment with a single tolerance of 3.1 mm for all subjects (median of the benchmark’s CT-derived values). At K=3, Chamfer error was 1.15 mm, compared with 1.16 mm using subject-specific tolerances.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

To all reviewers:

We thank the reviewers for their time. It is heartening to see them appreciate the simplicity and practical value of our method for imposing constraints to a pretrained diffusion model. Their main concern relates to our use of the term “hard constraint satisfaction” in the presence of tolerances; after reading their reviews, we broadly agree with them and propose wording changes and new experiments that will make this clearer in our revision (see below). Further, several reviewers’ comments have inspired us to better disambiguate our contributions and plan to include additional ablations to explore the impact of each individual component of our method (see below). After addressing these general comments, we also respond to each reviewer individually.

Q1: Hard constraint satisfaction (sRaB, zKya, 1rbD)

We will revise the manuscript to stress the real-world motivation behind our work; in particular, that we seek exact constraint satisfaction within the tolerances in each specific application (e.g., manufacture tolerances). For example, our cranial implant generation task uses tolerances taken from the clinically acceptable deviations discussed in cranial implant fabrication studies [Sharma et al., 2023].

This tolerance is used in two places: first, by the projection operator, which only projects samples to be within tolerance of the constraint; secondly, by our reported metrics, which interpret “constraint violation” to mean “constraint violation beyond what is allowed by the tolerance”. With this definition, for example, 1.6% of all points in the vertebra reconstruction task violate the constraint. If one desires even stronger satisfaction, one can reduce the tolerance used in the projection operator (i.e., by making the projection closer to the constraint manifold); in the vertebra case, lowering this tolerance to 2.30mm leads to a much lower 0.01% of violating points. In our revised manuscript, we will include a new figure and metrics showing the effect of this tolerance for this task.

Additionally, we will revise our manuscript to remark how the exact constraint satisfaction imposed in the real world cannot be easily measured in any discretized generative setting. For example, our skull implant generation task produces a point cloud of ~3k points (as proposed by [Friedrich et al., 2023]). This means that the distance between cranial rim and implant is only accurate up to discretization error: for example, even the ground-truth implant (which is exactly a section of the original skull!) does not “exactly satisfy the constraint” for any patient in the dataset.

After modifying the text to state these caveats, we will add a table reporting the percentage of samples (meaning, generated shapes) that fully satisfy the constraint at every generated point. For the vertebra generation, this is 13.5% (at 2.50–5.34 mm tolerance) and 81% (at 2.55–5.39 mm tolerance); for the aorta, this is 13% (at 2.2–2.5 mm tolerance) and 77% (at 2.25–2.55 tolerance).

Despite this difficulty in reporting exact satisfaction, our results show that our method achieves a better balance between constraint satisfaction and reconstruction accuracy. On vertebra reconstruction, LGD-MC and CPS produce fewer violating points (0.1% and 0.4%), but substantially higher Chamfer errors (4.35 and 4.95 mm versus our 1.16 mm). On the skull and aorta tasks, none of the re-evaluated baselines produce any exactly feasible samples.

Q2: Renoising without fixed-point iterations (sRaB, 1rbD)

We are thankful to the two reviewers for noticing this. Indeed, our method contains in it several contributions: not only the titular fixed-point iterations but also a renoising strategy specific to the constrained diffusion setup. As noted by the reviewers, there are cases in which the second of these is enough to vastly outperform all existing baselines; in others, it is from the fixed-point iterations that most performance gains come.

Inspired by this, our revised manuscript will include a larger disambiguation and exploration of these two contributions. For example, a new section in the Supplemental will discuss how when omitting the fixed-point iterations (i.e. K=0), our method can be seen as a generalization of DDNM to non-linear operators ([Wang et al., 2022]’s Limitations section, “DDNM only supports linear operators”).

This new section will also include a comparison at equal computational budget (measured by number of denoiser calls). In our aorta reconstruction task, the fixed-point iterations significantly improve feasibility even when the number of steps is reduced to match the computational budget:

MetricK=0, 128 stepsK=3, 32 steps
Exactly feasible samples ↑25%73%
Largest violation ↓0.22 mm0.04 mm
Chamfer error ↓2.77 mm2.89 mm

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.