- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper integrates iterative Data-Consistent Inversion with normalizing flows to enforce observed distributional constraints through importance-weighted learning of input distributions. The framework supports progressive flow refinement and adaptation of pretrained stochastic models without modifying their forward operators.
Critical strengths
- The framework combines iDCI with normalizing flows through a theoretically motivated weighted maximum-likelihood objective.
- The method updates input distributions while keeping pretrained stochastic operators fixed, reducing the need for full retraining.
- The proposed rebasing strategy progressively improves distributional consistency by learning corrective base distributions.
Critical weaknesses
- Experiments are restricted to 2D and 10D synthetic settings, leaving performance on realistic high-dimensional distributions unverified.
- Rebasing requires training additional normalizing flows and incurs higher runtime than both reported baselines.
- Matching low-dimensional marginals does not guarantee recovery of the joint distribution or its dependency structure.
- The paper lacks real-world applications demonstrating when localized distribution changes justify learning an additional normalizing flow.
- The sensitivity of density-ratio weighting to distribution shift, support mismatch, and effective sample size remains unexplored.
Questions for authors
See Weaknesses and below:
- How does the method perform as dimensionality increases to 100, 500, or 1000, considering accuracy, runtime, memory, and parameter overhead?
- Can the authors demonstrate the method on a pretrained scientific simulator or image normalizing flow with localized distribution shifts?
- How does NF rebasing compare with direct importance resampling or low-dimensional distribution updates under matched computational budgets?
- How does effective sample size change under larger distribution shifts? How do density estimation and weighted NF training scale with sample size, particularly for datasets containing millions of observations?
2 comments
Thank you for the clarification and narrowing the claims. My main concern remains unresolved. The paper still lacks comparisons with simpler but important baselines, such as direct importance reweighting or resampling. Also, the authors have acknowledged that the current work is primarily a proof-of-concept, with limited empirical validation. Overall, I don't believe the current version meets the standard for acceptance at ICLR. I will maintain my original rating.
Thank you for the constructive review. We agree that the current experiments demonstrate feasibility rather than high-dimensional scalability, which is something we plan on emphasizing more clearly as we have stated for the other reviewers.
The intended advantage of the method is that the forward operator can remain fixed while the input probability measure is adapted from output-space distributional constraints via the DCI/iDCI framework. In a revision, we intend to make this scope more explicit and avoid broader scalability claims than the current evidence supports since our goal was to demonstrate the proof-of-concept of utilizing DCI as an adaptation strategy.
We also agree that a conceptual comparison with some simpler alternatives such as direct reweighting or resampling would be useful for framing the contributions more clearly. The current sequential NF baseline controls for staged architecture growth, but does not fully isolate DCI-guided rebasing from generic weighting strategies. In a revision, we intend to add a brief discussion of effective sample size and weight concentration as diagnostics for density-ratio instability and support mismatch, but the theoretical analysis is better suited for a future journal-length article in our opinion. We also intend to emphasize more clearly that enforcing low-dimensional marginal constraints does not, by itself, recover the full joint distribution as we have mentioned in response to other reviewers.
Finally, we intend to revise the computational language to clarify that one benefit is avoiding retraining or modification of the forward map, not universal reduction in wall-clock time.