acceptodds
Under review as a conference paper at ICLR 2027

Marginal Scores under Structured Missingness: Coverage and Joint Recovery

Abstract

Incomplete records constrain marginal scores, while joint predictions depend on relationships that may never be observed together. We characterize when marginal Fisher error controls joint-score error, and at what cost. For Gaussians with a fixed number of observed coordinates per record, we determine the worst recovery constant up to universal factors throughout the budget range. The conditioning cost is cubic at fixed missing fractions and decreases toward one as observations become sufficiently complete. For density-ratio ANOVA models, we prove a global comparison governed by interaction coverage and a matching quadratic gradient cost. A shared background can increase this cost without changing the observations informative about the contrast. Statistical rates weight each interaction's inverse sample count by its gradient energy, and retain the quadratic cost when the background amplitude is unknown. Mixture experiments examine recovery without a prescribed interaction basis. Full-waveform ECG experiments show that identical acquired marginals can support different joint scores and conditional predictions, with acquisition gains depending on the target and cohort.

Primary area:
learning theory
Keywords:
score matching, missing data, identifiability, statistical learning theory, Fisher divergence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_J3XaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This study discusses whether it is possible to obtain upper bound control for the joint Fisher divergence of the data when only some (rather than all) variables are used in each prediction task. The author understands the problem from the perspective of a Gaussian model and extends it to functional ANOVA. At the end of the article, the theoretical results were verified through experiments.

Critical strengths
  1. Established upper and lower bounds for rate matching in the Gaussian case, and characterized the influence of condition number on the control ratio (Theorem 2).

  2. Established the marginal control accuracy under density-ratio ANOVA (Theorem 3), and the matching upper and lower bounds of score learning (Theorem 4).

Critical weaknesses
  1. The results of Section 4 only discuss the properties of the population distribution, and do not answer what the optimal Fisher risk for estimating $ \mu, \Sigma $ (under finite samples) is, or whether the optimal statistical estimation rate has a similar formulation. Please distinguish between population comparison minimax optimality and statistical estimation minimax optimality, and explain how much support the former's conclusion can provide for the latter's rate analysis.

  2. The number of multivariate interaction terms is extremely large, which brings a significant amount of computational cost. In many cases, not all interactions are significant, so there should be many zeros in θj\theta_j in equation (21). However, existing theories have not yet taken into account the impact of sparse selection of interactions on the problem, or have directly assumed which interaction terms are known to be non-zero, which is quite unrealistic.

Questions for authors
  1. The model setting and processing results of the Gaussian model and ANOVA are separated: the two constitute the two main lines of the article, so the writing is not coherent, and it is difficult to understand that, to what extent Section 5 and Section 6 inherit, and expand the results of Section 4.

  2. Why is there no influence of $ b_j $ in the minimax rate of Theorem 4, even though those bjj\\{ b_j \\}_j in equation (21), to some extent, describe the smoothness of the function $ p_{\theta} $?

  3. There are problems with the formula expression and assumptions in some parts of this study, such as the definition in $ \mathbb P(B \subseteq A)$ in equation (14), and the assumption in equation (13): when $ d $ is even, for a density ratio of (1+f)/μ=∏ℓ=1d/2[1+ϵtanh⁡(x2ℓ−1)tanh⁡(x2ℓ)],(1+f)/\mu = \prod_{\ell=1}^{d/2} \left[ 1+ \epsilon \tanh(x_{2\ell-1}) \tanh(x_{2\ell}) \right], it will result in R=Mm=(1+ϵ1−ϵ)d/2R = \frac Mm = \left( \frac{1+ \epsilon}{1- \epsilon}\right)^{d/2}, which shows exponential growth in dimension dd. The existing theoretical analysis has not discussed the impact of dimension dd.

  4. Suggest that the paper should provide more specific steps for implementing the algorithm, such as pseudocode.

2 comments
AuthorsOpenReview ↗

Response to Reviewer J3Xa

Thank you for the detailed comments. We respond in the same order below.

Main weaknesses

  1. Section 4 now includes an initial explanation of the two notions of optimality. Under the theorem's assumptions, our comparison upper bound turns a bound on marginal estimation error into one on joint error. Using the comparison lower bound to obtain a statistical lower bound also requires studying how hard it is to distinguish the models from finite data. We will try to explain more clearly.

    Appendix A.5 adds a Gaussian example with only one unknown variance parameter. At a fixed observation fraction, the comparison factor grows cubically with the condition number, while the coefficient in the estimator's asymptotic risk grows quadratically. This shows that the worst-case error amplification in a comparison bound need not be an unavoidable cost of estimation.

    We have not yet derived the optimal finite-sample risk when both the mean and covariance are unknown. We will continue working on this question.

  2. We specify the candidate basis functions, but do not assume that we know which coefficients are nonzero. The model allows zero coefficients. However, our analysis does not yet address how to use sparsity to select interactions. We have noted this limitation in Section 6 and will explore ways to address it.

Questions

  1. Sections 4 and 5 study different models, but both ask how marginal error controls joint error. Observing every pair can suffice for Gaussian models, but may not suffice beyond them. Section 5 therefore studies which interactions need to be observed and how this affects error. Section 6 studies the sample requirements of a finite-basis version of these models.

    We have revised the introduction and section openings to explain these connections, and moved the general local operator formulation from Section 5 to Appendix B.2.

  2. bjb_j is the maximum amplitude of a basis function. It affects the constants in the error bounds and how many samples are needed to reach the stated rate (Appendix C, Eqs. (95) and (100) of the original submission). Theorem 4 gives the rate for a fixed basis, so it does not spell out each of these dependencies. We have added this explanation after the theorem. Appendix C.3, Eq. (110), now gives an explicit sufficient condition on the sample size that includes bjb_j.

  3. Your example is correct: RR grows exponentially with dimension dd, making the constants in our general bound deteriorate rapidly. We have added discussion in Section 5 and Appendix B.3 and further analyzed the example's independent blocks. For comparison with the reference distribution, error amplification can be controlled by the probability of observing both variables in each block together. It need not grow exponentially with dimension.

    We have also clarified the definition of Pr⁡(B⊆A)\Pr(B\subseteq A) in Eq. (14) of the original submission. Here BB is a fixed set of variables, AA is the random observed set, and Pr⁡(B⊆A)\Pr(B\subseteq A) is the probability that all variables in BB are observed in the same record.

  4. We will continue improving the writing and making the implementation steps more explicit. The revision already includes the steps for computing the moment estimator in Appendix C.2 and for fitting and selecting the mixture models in Appendix D.2.

AuthorsOpenReview ↗

Update on Revisions Addressing Your Main Concerns

Thank you again for your patience and insightful comments. We would like to briefly highlight two additions in the latest revision addressing your main concerns:

  1. Matching upper and lower bounds on finite-sample risk when both the mean and the full covariance matrix are unknown, under the stated conditions (Weakness 1; Appendix A.5, Proposition 8).
  2. A method for selecting interactions without knowing in advance which coefficients are nonzero, together with an error guarantee. We also provide a rule for reducing computation by ruling out some candidates early (Weakness 2; Section 6 and Appendix C.2, Proposition 12).

We have also substantially revised the writing. Further details are provided in our revision summary and our latest reply to WyPd (which also discusses research decisions related to your question about sparse interactions). We would appreciate any further feedback you may have.

Reviewer_WyPdOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies score matching under partial observations. The main results concern when average score matching over the marginals is enough to recover the actual joint score.

Critical strengths

The paper studies a question left open by prior work (Givens et al), which is to actually understand whether marginal score accuracy is useful to control joint score accuracy. They provide some approximation results in the Gaussian setting, and establish qualitative tightness of their bound, as well as consider some non-Gaussian models and what I think are finite-sample extensions.

Critical weaknesses

There has been no effort made to make the paper readable to a potentially interested reader, which is a huge downside for representation at a conference. I cannot point to a single passage in the paper that does not appear to be blatantly AI generated. I can only conclude that the authors have no interest in claiming intellectual ownership of their submitted work, and a natural follow-up question is whether they have actual interest in the subject matter at all, and if so, why it's completely unreflected in the submission.

In slightly more detail, the paper is presented as just a chain of results, with little to no actual interpretation of how the bounds compare to the literature and what their downstream implications are supposed to be. Almost every sentence is dense and full of jargon. It is difficult to even understand what the intended takeaway of many sections is. (I have no idea what Section 6 is supposed to imply. I would encourage the authors to try to answer this question without having AI write their answer for them.)

Regarding the substance itself, the main setting studied is extremely toy. The authors did no interpretation of their result, so I will do it for them. It seems like the intended takeaway for Theorem 1 is that if you have some measure of "average marginal score matching" then it also controls the "joint score matching" of Gaussian distributions P, Q. This is supposed to say something about downstream generative modeling (again, downstream implications are not mentioned at all!) The issue is that taking the score approximator to be Gaussian is very strong. Even if you grant this assumption (as it is a toy model), what the theorem is actually saying is that every subset score is also Gaussian, and they are all consistent with a single high-dimensional Gaussian. What training procedure does this correspond to? It seems like the suggestion is to fit a single Gaussian with a score matching training loss that's averaged over observed marginals. I'm pretty unconvinced this is a meaningful result for Gaussians, where one would rather just use MLE on the observations and potentially even save some condition number factors. This is corroborated by the experiments in Section 7.1. Even if the paper was actually readable, the hodgepodge collection of theoretical results and experiments are not focused enough to clear the bar for acceptance at ICLR.

Questions for authors

The problem you're studying is actually interesting. But you appear to have no interest in actually learning about it yourself. I don't have any questions other than why you expect a reviewer to take your submission seriously given the level of effort you put into the submission.

5 comments
AuthorsOpenReview ↗

Response to Reviewer WyPd

Thank you for pointing out these issues, and we apologize for making the paper difficult to read. We acknowledge the extensive use of AI and the paper's many shortcomings. Still, this research represents months of genuine effort on our part, and we take your concerns seriously.

The work began with some Gaussian results that we found interesting. We tried to build on them and extend the analysis beyond Gaussians. As we added material and tried to fit it into the paper, the writing became harder to follow and the connections less clear. We have revised the introduction and section openings and added explanations of the results. We will continue improving the writing.

Your understanding that the marginal scores come from a single joint Gaussian model is correct. Although some settings are simplified, we believe the main findings still have value:

  • For Gaussian models, we quantify how missingness and covariance structure together amplify error, with matching upper and lower bounds. The comparison also applies to Gaussian models fitted by MLE under the theorem's assumptions.
  • The non-Gaussian results show that even when the observations can distinguish the models, learning an accurate joint score may require more data. Section 6 explains why: in our example, the data provide the same information about one unknown parameter, yet the joint score can become more sensitive to errors in that parameter. We prove that the resulting learning cost is unavoidable in the worst case, even with MLE (Appendix C.4).

Both parts ask which relationships must be observed and how accurately they must be learned.

The paper included some discussion of applications, but we did not explain the connections clearly. ECG lead reconstruction is one example (Section 7.3, Figure 3). When some leads are never observed together, two Gaussian models can have exactly the same observed marginals yet make different predictions for the missing leads. Checking marginal fit alone is therefore not enough to judge reconstruction accuracy. We have added an explanation of this connection after Eq. (3) and in the conclusion.

We will continue improving the manuscript.

Reviewer_WyPdOpenReview ↗

Confused about "months of genuine effort"

I am quite skeptical of this claim that the authors spent "months of genuine effort" on this project. You're telling me that in the process of trying to write this paper yourself, you found some difficulties in doing so, and so you decided to have AI write the whole thing? Or are you saying at no point did you ever make any effort to do any writing yourself?

Hopefully you can understand why if this is the case, I find it dubious that having the authors present a work that they don't even understand or take any ownership of will be of value to the ICLR community / attendees.

Maybe to clarify, did you actually write any of the manuscript yourselves, or have any substantial opinions about your "own work"? If so, could you point me to a passage that you wrote, and explain why you made the presentation choices you did?

The paper was mainly drafted by AI, and we cannot point to a passage written entirely without AI assistance. We made research decisions, chose what to include, and repeatedly revised and checked the manuscript. We did not simply accept AI output unchanged.

We take responsibility for the paper. In a separate comment, we explain how we have strengthened the theoretical results and made the paper easier to follow.

Reviewer_v2y6OpenReview ↗
Rating
3
Confidence
1
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies when learning marginal distributions from partially observed data is sufficient to recover the full joint distribution. It establishes theoretical bounds for Gaussian and structured non-Gaussian distributions, examining the roles of observation and coverage and conditioning. The paper also presents synthetic and real-data experiments, including ECG reconstruction.

Critical strengths

The theoretical contribution seems substantial and goes beyond the Gaussian setting. The question of when partial observations are sufficient and recovering a joint distribution seems interesting and important. The ECG experimental validation seems quite extensive.

Critical weaknesses

I found the main paper quite technical, with relatively limited intuitive explanations of the theoretical contributions and their practical implications. I believe the presentation would benefit from a more accessible discussion of the main results and contributions, including their intuition, significance, and applicability, while leaving some of the more technical details to the appendix.

Questions for authors

Could the authors provide a more intuitive explanation of how the main theoretical results should guide the choice of observation patterns in practice?

2 comments
AuthorsOpenReview ↗

Response to Reviewer v2y6

Thank you for your suggestion. Here is a simpler explanation of our main findings:

When choosing what to observe, we need to consider both how often each variable is measured and which variables are measured together. In some non-Gaussian models, observing every pair still leaves a relationship among three variables undetermined. All three then need to be observed together.

We also quantify the error cost once these relationships have been observed. The Gaussian results show how missingness and covariance structure amplify error, and how this cost decreases as observations become sufficiently complete. In our Gaussian model class, when each record reveals the same number of variables, choosing uniformly among subsets of that size minimizes the worst-case error amplification. The non-Gaussian results show that even with sufficient coverage, more data may be needed when the joint score is more sensitive to parameter error.

Thus, when choosing what to observe, we should first check whether important relationships can be observed at all. We should then consider how often the relevant variables are observed together and how errors in learning those relationships affect the target.

We have added these explanations to the introduction and section openings, and moved the general local operator formulation from the start of Section 5 to Appendix B.2. We will continue improving the presentation.

We thank the authors for their response. My overall rating and confidence remain unchanged.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Thank you to all three reviewers for your comments. We have further revised the paper. The main changes are:

  1. Statistical risk for Gaussian models. We extended the single-parameter example to a model where both the mean and the entire covariance matrix are unknown. Under the stated scale bounds, fixed dimension and observation budget, and balanced pair observation counts, we obtained matching upper and lower bounds on finite-sample risk with quadratic dependence on the condition number (Reviewer J3Xa, Weakness 1; Appendix A.5, Proposition 8).

  2. Sparse interactions. We added a method and error guarantee for selecting interactions without knowing which coefficients are nonzero. We also added a rule for excluding some candidates early to reduce computation (Reviewer J3Xa, Weakness 2; Section 6 and Appendix C.2, Proposition 12).

  3. Comparison with prior work. We expanded our comparisons with work by Givens, Sell, Ding and others, clarifying how the questions and results differ (Section 2 and Appendix F). We also added comparisons near the relevant results.

  4. Writing and organization. We rewrote dense or unclear passages, simplified terminology, and clarified the connections between sections. We shortened secondary discussions in the main text to keep the focus on the main results and their intuitive explanations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.