Decisions that depend on a tail probability or a quantile need the conditional distribution, not a conditional mean. Estimators of it differ in where they obtain normalization: a closed-form family, a Jacobian determinant, a simplex over cells, or a partition function. We present CDF-First, which takes it from a boundary condition: each factor of an autoregressive CDF is a bounded monotone network pinned to 0 and 1 at the ends of a declared domain, and the density is read from an interval probability of half-width δ. At every parameter value, and with no smoothness assumption on the target, the model is an exactly normalized conditional density, and each factor equals the model's own convolved with a uniform kernel of half-width δ; the objective is proper for that smoothed conditional, not the unsmoothed one. The smoothing is therefore a stated parameter, and we test whether it is the one applied. On four synthetic targets with known ground truth it bounds the realized smoothing from above, within 5%, in 253 of the 258 fits for which the smoothing is identifiable, which realize a median of 84% of it; raising δ over two decades improves accuracy on three targets without measurably degrading the fourth. Accuracy is competitive rather than superior: lowest error on three of the four synthetic targets, and the lowest mean held-out negative log-likelihood on four of seven UCI benchmarks, tied on two and second on one.
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.Reviewer_nrzUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary
The paper is about conditional density estimation. A nerual network is parameterized on the CDF-scale in a way that ensures proper normalization of the density. Some experiments check the standardization and performance against other neural networks the authors came up with.
Critical strengths
None
Critical weaknesses
Terrible and confusing writing, it's really hard to understand what is done. The paper reads like 100% AI-generated without much oversight. I could provide a long list of red-flag terminology and phrases, but I'll save us all some time.
No comparison against classical methods for conditional density estimation.
There's no code, so experimental details can't be verified.
Questions for authors
What are these propositions you keep referring to (and prove)? They are nowhere in the paper.
What's the purpose of κδ,αi and piδ,α? They are introduced but never used. What is Y^i?
Thank you for the review. Several of your points come from errors in the submitted PDF; we have fixed them and answer each directly.
"What are these propositions you keep referring to?"
You are right that they could not be found. Our LaTeX preamble never defined the proposition, definition and remark environments, so every result printed as a bare bracketed title with no number, and every cross-reference collapsed to "3.2". This is fixed; the revised PDF has Definition 3.1, Propositions 3.1–3.3 and Remarks 3.1–3.2, each numbered. We have also added a one-sentence reading after each proposition, removed repeated hedging, and corrected the inconsistent protocol descriptions (split, grid size) that made the submission harder to check.
(Fixed: §3.2–3.3.)
"What's the purpose of κδ,αi and piδ,α? What is Y^i?"
All valid, and the first two became more so after the correction to Proposition 3.3. κδ,αi is removed. piδ,α is now defined directly in Definition 3.1 and has one remaining role, in Proposition 3.3: it is the density the objective does not target. Y^i is a draw from the model's own law, i.e. the random variable with CDF F^i; Proposition 3.2 says p^iδ is the density of Y^i+δU. The submission used it without defining it, and the hat was ambiguous with the noised response y^ of Eq. (2). It is now defined in Proposition 3.2 and both symbols are in the notation table.
(Revised: Definition 3.1; Proposition 3.2; Appendix A, Table 5.)
"No comparison against classical methods."
This was a gap, and an awkward one for a method whose idea is borrowed from the explicit bandwidth of kernel estimators. We added conditional kernel density estimation (CKDE; Gaussian product kernels, bandwidths by cross-validated likelihood, Hall, Racine & Li 2004) on all synthetic targets and all seven UCI datasets. On UCI, capacity-matched CDF-First has lower held-out NLL on 6 of 7 (the exception is Parkinsons, 0.71 vs 0.61); CKDE degrades at dy=3 (Air: 1.55 vs 0.51) and dy=4 (Skillcraft: 3.27 vs 1.18), as expected for kernel methods. On the synthetic targets CKDE ranks 5th–9th of ten and CDF-First is lower on all four, though on Gaussian Stick only marginally (0.151 vs 0.152).
(New: CKDE rows in Tables 1 and 2; configuration in Appendix E.3.)
We also affirm to reviewer nrzU and the Area Chair that all research, mathematics, code, and original text were written by the authors; the compressed prose style resulted from manual editing to fit page limits, with LLMs used strictly for grammar/copy-editing as disclosed.
The paper proposes CDF-First, a conditional density estimator that parameterizes a monotone neural CDF and obtains its density through finite differences. The main claims are:
Exact normalization: Normalizing the monotone network using its boundary values produces a valid CDF, and therefore a normalized density.
Exact mollification: The returned density piδ is exactly the implicit density pi convolved with a uniform kernel of half-width δ.
Properness: The authors claim that the population objective is proper for the true conditional density convolved with Gaussian and uniform kernels.
The experiments investigate whether the stated smoothing bandwidth corresponds to the realized smoothing and compare density estimation accuracy against several baselines on synthetic and UCI datasets.
Critical strengths
S1. Simple construction. The idea of obtaining a normalized density from a monotone neural CDF is intuitive and mathematically clean.
S2. Exact mollification. The finite-difference density has a precise interpretation as a uniform convolution of the implicit model density. This is a useful property.
S3. Interesting empirical investigation. I appreciate that the paper explicitly investigates whether the stated bandwidth corresponds to the realized smoothing, rather than assuming that it does.
Critical weaknesses
W1. I have a serious concern about the properness claim.
The authors claim in Section 3.3 that the population objective is minimized at pδ=pδ,α, where pδ,α is the true density convolved with Gaussian and uniform kernels.
However, Equation (8) evaluates the model's uniformly smoothed density at observations perturbed only by Gaussian noise.
Consider the one-dimensional setting with α=0. The population objective becomes
LNLL=−EY∼plogpδ(Y)=H(p)+DKL(p∥pδ).
Thus, if representable, the optimum is pδ=p, rather than pδ=p∗U[−δ,δ].
I do not see how the proof in Appendix D.4 reconciles this distinction. Since properness is one of the main theoretical claims, this is a significant concern.
W2. The practical motivation for the explicit bandwidth is unclear.
The paper argues that δ makes the smoothing resolution controllable. However, the implicit density p is learned jointly with the smoothing operation. Since the likelihood encourages pδ to fit the observations, the network can sharpen p to compensate for the imposed smoothing.
This may actually make the implicit density harder to learn, potentially requiring larger gradients or a larger Lipschitz constant of the neural CDF. It is therefore unclear what practical control δ provides beyond acting as another regularization parameter.
W3. The experimental evidence is not convincing enough.
The baselines include several established normalizing flows, but I would have liked comparisons with more recent conditional continuous normalizing flows, neural ODEs, or diffusion-based density estimators.
The parameter counts are also substantially different. For example, CDF-First has roughly 388K parameters, compared with approximately 15K for the closely related NL-CDF baseline. This makes it difficult to attribute the reported improvements to the proposed method.
Finally, the qualitative density estimates in Appendix F.2 exhibit visible artifacts, despite the substantially larger model. I do not find these results particularly encouraging.
Questions for authors
Q1. Could the authors clarify the properness proof in Appendix D.4? In particular, why should Equation (8) be proper for pδ,α when uniform noise is applied to the model density but not to the training observations?
Q2. What practical advantage does the explicit bandwidth provide if the underlying neural CDF can compensate for it? Could the authors measure how the sharpness or Lipschitz constant of the implicit density changes as δ increases?
Q3. Could the authors provide capacity-matched comparisons, particularly against other neural CDF estimators, and include stronger contemporary conditional density estimation baselines?
Thank you for careful review. W1 is correct, and the correction changes how the paper should be read; we address it first.
W1 / Q1. Properness.
Your derivation is right. Eq. (8) scores the uniformly smoothed model density at observations that carry only Gaussian noise, so the objective is not proper for pδ,α. The submitted proof assumed the observations were distributed as pδ,α, which they are not. We have replaced Proposition 3.3 and its proof.
One detail of our implementation matters for α>0: Eq. (2) shifts the domain endpoints by the same draw as the observation, so the returned density depends on ϵ in general. Under a boundary-slack condition (no mass within a band of width a≤21 of either edge, and ∣αϵ∣≤a) that dependence vanishes and the objective reduces exactly to your form,
R=h(rα)+KL(rα∥p^δ),rα=p~∗N(0,α2),
with p~ the density of Ψ(Y). For α=0, your case, no condition is needed. The condition is an idealization, and Appendix D.4 says so; at the initial α=e−2 and a=21, P(∣αϵ∣>a)=2.2×10−4.
The corrected proposition shows the minimizer is the KL projection of rα onto the class the model can return, Qδ={μ∗U[−δ,δ]}, and is rα itself when rα∈Qδ; then pδ,α=rα∗U[−δ,δ] has variance larger by δ2/3 and is not a minimizer. Concretely, p~=U[c−δ,c+δ]=δc∗U[−δ,δ] is representable, so the objective is minimized at p~, not at the triangular density the submission predicted. The proposition also identifies when the original claim holds: if each observation additionally receives independent noise δU, the observations are distributed as pδ,α and Gibbs' inequality gives strict properness. That is a variant of the method, not the one we trained.
Two consequences. First, δ does not enter the target; it constrains the returned densities, and the fit is rewarded for undoing the uniform kernel wherever the class allows. This predicts, but does not bound, a realized smoothing below the stated one, which is what §4.3 measures (median 84%). Second, no experimental result depended on the incorrect statement, since §4.3 measures the realized smoothing directly against known ground truth.
(Revised: Proposition 3.3, §3.3; Appendix D.4; the old claim removed from the Abstract, §1, Contributions, §2, Definition 3.1, §5 and Appendix G.)
W2 / Q2. What does the explicit δ provide if the CDF can compensate?
We agree with the premise; it is the prediction of Remark 3.1 and the subject of §4.3. What δ provides is narrower than the submission implied, and we now state it as follows:
A hard ceiling on the returned density. Since F^i∈[0,1] at every parameter value, p^iδ≤1/(2δ) everywhere, and every returned density has the form μ∗U[−δ,δ]. A differentiated CDF gives no such bound.
A fixed, measurable offset from the model's own law.Var(p^iδ)=Var(p^i)+δ2/3 exactly, whatever the fit does. Whether this becomes smoothing relative to the truth is not guaranteed, since the fit may sharpen p^i below the truth; that is why §4.3 measures it: below the stated value in 253 of 258 identifiable fits, median 84%. We report this as a measured property, not a proven bound.
A regularizer of known form. Unlike weight decay or early stopping, the constraint has an explicit kernel and an upper bound on its effect that can be tested, which §4.4 shows cannot be done for the baselines' implicit resolution.
Empirical Measurement of Implicit Sharpness (Lipschitz Constant of F^i) vs. δ
We have not completed this measurement within the discussion period; the checkpoints from the δ sweep allow it without retraining and we will add it to the final version.
W3 / Q3. Capacity, baselines, artifacts.Capacity. We re-ran CDF-First at 4×4 SMM groups: 2,730 parameters on the synthetic targets against 15,074 for NL-CDF, and smaller than NL-CDF on four of seven UCI datasets (parameter table in our reply to Reviewer UqDp). Against NL-CDF, CDF-First is lower on 5 of 7 UCI datasets and 3 of 4 synthetic targets; against ATM, the triangular monotone transport map, on all 11. Against all methods it is lowest on 4 of 7 UCI datasets and 2 of 4 synthetic targets, fewer than at 388k, and we have revised the accuracy claims to say so.
Baselines. We added conditional kernel density estimation (CKDE). We were not able to add a conditional flow-matching or diffusion-based estimator within the discussion period; Appendix G now lists this as a limitation.
Artifacts in Appendix F.2. We do not think the texture comes from model size; the 2,730-parameter model shows it too. The corrected Proposition 3.3 predicts it: on flat regions the objective rewards a staircase-shaped CDF with steps 2δ apart, which partly cancels the uniform kernel, giving texture at the scale of δ. We now say this in §4.5 rather than only in the appendix.
(New: §4.1; §4.5, Table 1; §4.6, Table 2; Table 6; Appendix G.)
This work proposes estimating conditional densities by learning monotone conditional CDFs. The authors fix the domain endpoints, and then the density is computed from CDF difference, which corresponds to a uniform kernel smoothing of the underlying density. They do experiments on four synthetic distributions and seven UCI datasets, and show that it bounds the realized smoothing from above within 5% in most cases for whcih the smoothing is identifiable. They show high accuracy on the synthetic tests.
Critical strengths
The best strength of the paper is that it is simple and interpretable. The construction is easy to see as a valid CDF and finite difference density is easy to interpret as a uniform convolution. This helps with making the paper easy to follow and the work easy to verify. I also think the synthetic problems are good, and are nontrivial. The empirical results are very good (albeit a bit limited).
Critical weaknesses
The main weakness is that the novelty is not so clear to me. They discuss Chilinski and Silva as unconditional and univariate. However, the chilinski and silva paper definitely discusses conditional CDFs and masked autoregressive multivariate models. I think it needs to be made clear what the extension is from the existing work. One issue I have with the empirical section is that the experiments are not comparing models apples to apples. The CDF-First has around 400k parameters, whereas the comparison models have 15k parameters. It would be good to try make the comparisons closers.
Questions for authors
Could you explain the contributions in light of the existing literature?
Can you explain why the training objective is minimized by the setup in the propostion 3.3?
Do improvements over NL-CDF persist when the methods have similar parameters and tuning budgets?
Thank you for a clear and constructive review. Each of your three questions pointed at a weak claim, and we changed the paper in response to each.
Q1. Contributions in light of the existing literature.
Our description of Chilinski & Silva (2020) in §2 was wrong: NL-CDF already handles covariates and multivariate responses, which is exactly the autoregressive variant we ran as a baseline (Appendix E.3). We have corrected §2 and no longer present "a conditional CDF estimator" as the contribution. Relative to NL-CDF, three things differ, in decreasing order of importance:
How the density is read. NL-CDF differentiates the learned CDF. CDF-First reads an interval probability, [F^(y+δ)−F^(y−δ)]/2δ, which by Proposition 3.2 is exactlyp^∗U[−δ,δ], with no Taylor remainder. This makes δ a declared resolution whose effect can be measured against ground truth (§4.3), which cannot be asked of an estimator with no stated bandwidth.
Normalization. NL-CDF is bounded by a sigmoidal output with limits at ±∞; CDF-First is pinned to 0 and 1 at the endpoints of a declared compact domain, at every parameter value (Proposition 3.1), at the cost of compact support, which we state.
Architecture. We use masked smooth min–max monotone layers (Igel, 2023) rather than positive-weight MLPs. In the ablation (§4.7) a positive-weight monotone MLP raises error 4.4× on the annular target; this is an empirical difference, not a claimed contribution.
Accuracy is not the contribution; see Q3.
(Revised: §2 ¶2; Contributions bullets 1–2.)
Q2. Why is the objective minimized at the density in Proposition 3.3?
It is not, and thank you for asking; Reviewer 94WW identified the same error. Eq. (8) evaluates the returned density at observations carrying Gaussian noise but no uniform noise, so the objective targets the Gaussian-noised conditional, and δ only restricts the class of densities that can be returned. The corrected statement and proof are summarised in our reply to Reviewer 94WW (Q1).
(Revised: Proposition 3.3, §3.3; Appendix D.4.)
Q3. Do improvements over NL-CDF persist at similar parameters and tuning budgets?
We reduced CDF-First from 32×32 SMM groups (388k parameters) to 4×4 (feature widths 4,4,1; covariate widths 4,4,1×dx on synthetic and 4,2,1×dx on UCI), and added CKDE as a classical baseline. No method, ours included, was tuned per dataset; baselines keep their authors' default configurations and all use the same folds.
Dataset
dx
dy
CDF-First (4×4)
NL-CDF
Synthetic
1
2
2,730
15,074
Fish
6
1
4,306
7,681
Concrete
8
1
6,116
7,745
Energy
8
2
10,640
15,522
Air
8
3
15,932
23,331
Parkinsons
16
2
23,880
16,034
Temperature
21
2
34,430
16,354
Skillcraft
15
4
38,932
32,068
CDF-First is now 5.5× smaller than NL-CDF on the synthetic targets and smaller on four of seven UCI datasets (means 19.2k vs 17.0k).
Head-to-head against NL-CDF:
CDF-First lower
NL-CDF lower
UCI (NLL, 7)
Concrete, Energy, Temperature, Air, Skillcraft
Fish, Parkinsons
Synthetic (SSE, 4)
Squares, Half Gaussian (0.249 vs 0.260), Elastic Ring
Gaussian Stick
The improvement persists on 5 of 7 UCI datasets and 3 of 4 synthetic targets, but it is not uniform. Against all methods, the reduced model is lowest on 4 of 7 UCI datasets (Concrete, Energy, Air, Skillcraft) and 2 of 4 synthetic targets (Squares, Elastic Ring), fewer than the 388k configuration achieved: part of the original margin on Fish, Parkinsons, Temperature and the two synthetic targets without a void came from capacity. We have revised the accuracy statements accordingly.
(New: §4.1; §4.5 and Table 1; §4.6 and Table 2; parameter counts in Table 6; configurations in Table 7.)
Thanks for correcting prop 3.3 and also for comparing the smaller CDF-first model with NL-CDF, as suggested. This eases some of my concerns about how much improvement came from model capacity. I'll discuss significance and my smaller points about the density estimate with the other reviewers before finalizing my score.
Good that the Definitions and Propositions are now visible as such, and the main terms in the theory are defined. These are minimal standards before letting other people review the work.
The code has a README in a language I don't speak and, as far as I can tell, is incomplete (e.g., does not contain the CKDE baseline).
The writing is still terrible. Whether Claude generated it or just "copy-edited" the author's own writing is immaterial.
I will keep my score; this paper is far below the bar.
From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.
Response to Reviewer nrzU
Thank you for the review. Several of your points come from errors in the submitted PDF; we have fixed them and answer each directly.
"What are these propositions you keep referring to?" You are right that they could not be found. Our LaTeX preamble never defined the
proposition,definitionandremarkenvironments, so every result printed as a bare bracketed title with no number, and every cross-reference collapsed to "3.2". This is fixed; the revised PDF has Definition 3.1, Propositions 3.1–3.3 and Remarks 3.1–3.2, each numbered. We have also added a one-sentence reading after each proposition, removed repeated hedging, and corrected the inconsistent protocol descriptions (split, grid size) that made the submission harder to check. (Fixed: §3.2–3.3.)"What's the purpose of κδ,αi and piδ,α? What is Y^i?" All valid, and the first two became more so after the correction to Proposition 3.3. κδ,αi is removed. piδ,α is now defined directly in Definition 3.1 and has one remaining role, in Proposition 3.3: it is the density the objective does not target. Y^i is a draw from the model's own law, i.e. the random variable with CDF F^i; Proposition 3.2 says p^iδ is the density of Y^i+δU. The submission used it without defining it, and the hat was ambiguous with the noised response y^ of Eq. (2). It is now defined in Proposition 3.2 and both symbols are in the notation table. (Revised: Definition 3.1; Proposition 3.2; Appendix A, Table 5.)
"No comparison against classical methods." This was a gap, and an awkward one for a method whose idea is borrowed from the explicit bandwidth of kernel estimators. We added conditional kernel density estimation (CKDE; Gaussian product kernels, bandwidths by cross-validated likelihood, Hall, Racine & Li 2004) on all synthetic targets and all seven UCI datasets. On UCI, capacity-matched CDF-First has lower held-out NLL on 6 of 7 (the exception is Parkinsons, 0.71 vs 0.61); CKDE degrades at dy=3 (Air: 1.55 vs 0.51) and dy=4 (Skillcraft: 3.27 vs 1.18), as expected for kernel methods. On the synthetic targets CKDE ranks 5th–9th of ten and CDF-First is lower on all four, though on Gaussian Stick only marginally (0.151 vs 0.152). (New: CKDE rows in Tables 1 and 2; configuration in Appendix E.3.)
There’s no code. The code is now available at https://github.com/iclr-56391/CDF-First
We also affirm to reviewer nrzU and the Area Chair that all research, mathematics, code, and original text were written by the authors; the compressed prose style resulted from manual editing to fit page limits, with LLMs used strictly for grammar/copy-editing as disclosed.