acceptodds
Under review as a conference paper at ICLR 2027

Observation-Aligned Mask Priors for Learning Physical Fields from Authentic Occlusions

Abstract

Learning physical fields directly from incomplete observations is challenging because authentic occlusions are structured, sample-dependent, and often missing not at random, whereas existing methods typically rely on heuristic masking rules or predefined mask distributions. We propose Observation-Aligned Mask Priors, a framework that learns the distribution of authentic observation masks and uses it to construct context-query partitions for training from incomplete data. Specifically, we pretrain a Bayesian Flow Network (BFN) on binary observation masks to capture real occlusion topologies, then guide BFN sampling with a globally normalized cross-entropy objective to generate sample-specific masks aligned with each sparse observation. The intersection between the guided mask and the observed mask defines the context, and the remaining observed entries become query targets for a diffusion-based reconstruction model. We show that this intersection-based partitioning gives every valid observed dimension a strictly positive probability of being queried, preventing zero-query dead zones and local generative collapse. Experiments on three real-world oceanographic datasets with authentic satellite occlusions, across resolutions up to 256256, show consistent improvements over strong diffusion baselines in MSE and PSNR. These results demonstrate that learning mask priors from authentic occlusions is an effective alternative to heuristic masking for learning from incomplete physical observations without access to fully observed fields.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI4PDE

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qB9zOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies physical field reconstruction under authentic missing observations. To address the limitation that conventional random or block-wise masking strategies cannot adequately capture realistic occlusion patterns, the authors propose Observation-Aligned Mask Priors. Specifically, a BFN is first trained to learn the distribution of authentic observation masks, and guidance is applied during sampling to generate masks aligned with the current observation pattern. The intersection of the generated mask and the original observation mask is then used as the context, while the remaining observed entries serve as queries for training the physical field reconstruction model. Theoretically, the method guarantees that each valid observed location has a non-zero probability of being selected as a query. Experiments on three oceanographic datasets show that the proposed method outperforms several baselines in terms of MSE and PSNR.

Critical strengths
  1. The paper identifies an important limitation of existing context-query partitioning strategies: handcrafted random or block-wise masks may fail to reflect authentic, structured occlusion patterns in real-world observations.
  2. The paper provides substantial theoretical analysis. In particular, the authors theoretically establish the positive query probability property of the proposed mask partitioning strategy, providing justification for avoiding spatial locations that never receive reconstruction supervision.
  3. The authors evaluate the method on three oceanographic datasets, where the proposed method achieves consistently competitive or improved reconstruction performance.
Critical weaknesses

The experimental validation is insufficient.

Questions for authors
  1. The number of comparison methods is limited. The authors only compare with DINDiff, Ambient Diffusion, and MissDiff.
  2. There is no analysis of inference latency. Since the proposed method relies on an additional BFN to learn the mask distribution, it is unclear whether the additional complexity is necessary.
  3. The experimental scenarios are relatively limited, as all three datasets are from oceanographic settings.
  4. If the missingness depends on the underlying physical state, is modeling the mask topology alone sufficient to capture the MNAR missingness emphasized in the paper?
4 comments

We sincerely thank the reviewer for your constructive feedback.

Q1. Additional Baseline Comparisons

To further strengthen our empirical evaluation, we have additionally included two more baselines: MSM (Park et al., 2026) and DINEOF (Beckers and Rixen, 2003), covering both recent diffusion-based approaches and classical oceanographic reconstruction methods. Our method consistently outperforms both newly added baselines, further demonstrating the effectiveness and robustness of our method compared with both modern diffusion-based approaches and classical oceanographic reconstruction methods.

Black Sea CHL│Baltic Sea NANO│Global Ocean SSS
MSE ↓PSNR ↑│MSE ↓PSNR ↑│MSE ↓PSNR ↑
DINEOF0.62014.781│0.59913.608│0.46016.395
MSM0.29815.412│0.35915.375│0.38616.808
Ours0.19317.825│0.29416.101│0.27618.192

[1] Park, C. Y., Shoushtari, S., An, H., and Kamilov, U. S. (2026). Measurement Score-Based Diffusion Model. International Conference on Learning Representations (ICLR).

[2] Beckers, J.-M., and Rixen, M. (2003). EOF Calculations and Data Filling from Incomplete Oceanographic Datasets. Journal of Atmospheric and Oceanic Technology, 20(12), 1839–1856.

Q2. Additional Cost

An important clarification is that BFN is only used to generate context-query partitions during training and is not required at inference time. Once the imputation model is trained, reconstruction is performed directly using the learned imputation model, without any BFN sampling or guidance computation. Therefore, our framework introduces no additional inference latency compared with the same imputation backbone.

We acknowledge that online BFN sampling introduces additional training-time cost. However, this is largely an engineering choice and can be eliminated by pre-generating a diverse pool of observation-aligned masks for each sample and randomly selecting them during training, bringing per-iteration costs close to conventional masking with negligible expected impact on performance. Moreover, learning binary mask priors requires only a lightweight BFN network, whose pretraining cost is quite small relative to the training of main imputation model. Thus, computational cost can be substantially reduced through implementation choices without compromising our core methodology.

Q3. Additional Experimental Domain

To further evaluate the generalizability of our framework beyond oceanographic applications, we have conducted additional experiments on NASA's MOD11A1 dataset, which provides daily satellite observations of land surface temperature and emissivity at 1-km spatial resolution. Our method also achieves strongest reconstruction performance on this non-oceanographic dataset, demonstrating that the benefits of observation-aligned mask priors are not limited to oceanographic fields. These additional results provide further evidence of the generalizability of our framework across different physical domains and observation patterns.

MOD11A1
MSE ↓PSNR ↑
DINEOF0.36511.074
MSM0.40410.481
AmbientDiff0.31511.443
DINDiff0.6198.256
MissDiff0.44610.476
Ours0.28911.595

Q4. Clarification on MNAR

We believe this concern stems from a misunderstanding of our problem setting. Following Incomplete Data, Complete Dynamics: A Diffusion Approach (Zhou et al., 2025), our theoretical framework adopts the data–mask independence assumption, rather than explicitly modeling state-dependent missingness. Zhou et al. (2025) have already established that strictly positive query probability enables unbiased conditional-expectation learning from incomplete observations under mask-data independence assumptions. Building upon this foundation, our work focuses on how to achieve this essential property in real-world applications characterized by complex and structured observation patterns.

Importantly, if a coordinate's missingness is determined by its own underlying physical state, that state may consistently prevent the coordinate from being observed (Mi=0M_i=0). In such cases, the physical state and its missingness are inherently coupled: the true value remains permanently unobserved and cannot be learned from the available data, making unbiased reconstruction impossible without additional assumptions, which falls outside our problem setting. Instead, we address authentic, structured observation gaps arising from factors such as cloud cover, satellite trajectories, and geographical constraints, which are poorly captured by heuristic masking.

Dear Reviewer qB9z,

We hope our responses have addressed your concerns. We would greatly appreciate it if you could take a moment to share your thoughts when you have time.

Thank you for your consideration and efforts.

I thank the authors for their detailed response and additional experiments. The newly included baselines and the evaluation on MOD11A1 substantially address my concerns regarding experimental comparisons and generalizability. The clarification that BFN is used only during training also resolves my concern about additional inference latency.

I appreciate the clarification regarding the data–mask independence assumption. I encourage the authors to explicitly state this assumption and distinguish structured observation missingness from state-dependent MNAR throughout the revised manuscript. Additionally, reporting the training-time overhead of BFN would help provide a more complete picture of the computational cost.

Overall, the rebuttal has adequately addressed my main concerns, and I will maintain my positive rating of 3 (Weak Accept).

Thank you for your time and effort.

We appreciate the reviewer's positive recognition of our work.

Reviewer_ABwMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the challenge of reconstructing physical fields from sparse observations—characterized by structured, non-random (MNAR) occlusions—in the absence of complete ground truth. We propose the "Observation-Aligned Mask Priors" framework, which consists of the following key components:

  1. Observation-Aligned Generative Mask Partitioning: We employ a Bayesian Flow Network (BFN), pre-trained on binary observation masks, to capture the topology of real-world occlusions. Through stochastic anchors and sampling guided by globally normalized cross-entropy, the framework generates masks aligned with the specific sparse input. These masks are then used to dynamically partition the domain into "context" and "query" regions via intersection operations.
  2. Proof of Strictly Positive Query Probability: We provide a theoretical proof—covering both unconditional sampling (Theorem 1) and constrained, guided sampling (Theorem 2)—demonstrating that this intersection-based partitioning strategy ensures a strictly positive probability (P((Mqry)i=1)>0P((M_{qry})_i = 1) > 0) for querying any valid observational dimension. This effectively eliminates "zero-query" dead zones and prevents localized generative collapse.
  3. Multi-Resolution Validation on Real-World Data: Experiments conducted on three real-world satellite oceanographic datasets (Black Sea CHL, Baltic Sea NANO, and Global Ocean SSS) at resolutions up to 256×256256\times 256 demonstrate that our method consistently outperforms existing diffusion-based baselines—such as AmbientDiff, DINDiff, and MissDiff—in terms of both MSE and PSNR.
Critical strengths
  1. A generative mask-partitioning mechanism that addresses key pain points in scientific computing: In real-world remote sensing and physical observations, data absence is often highly dependent on factors such as cloud cover, satellite orbits, and coastline topology—a phenomenon known as "Missing Not At Random" (MNAR). This paper replaces traditional heuristic random masking with a learned BFN (Bayesian Flow Network) mask prior. Furthermore, it employs a random-anchor-guided approach to mitigate the issue of overly sparse context—a common drawback when unconditional mask intersections are too small. The design is both sound and effective.

  2. Rigorous controlled ablation and multi-scale experimental validation: The method achieves state-of-the-art performance on oceanographic datasets at resolutions up to 256×256256\times 256, across three distinct patterns of data absence. Through strictly controlled experiments that hold the context-to-query ratio constant (see Appendix E.5 and E.6), the study provides compelling evidence that the observed performance gains stem from the alignment of the mask topology, rather than merely from adjustments to the masking ratio.

Critical weaknesses
  1. There is a disconnect between the theoretical assumptions (Theorem 2) and the actual algorithm (Algorithm 1), and the proofs are somewhat tautological: the positive-probability conclusions in Theorems 1 and 2 follow almost directly from the definitions regarding the existence of support sets in Assumptions 1 and 2, rendering the mathematical depth limited. Furthermore, Theorem 2 relies on a hard cardinality intersection constraint CkC_k (∣M^⊙M∣=k\vert{}\hat{M}\odot M\vert{}=k), whereas Algorithm 1 actually employs Bernoulli anchor sampling guided by soft cross-entropy gradients. Given the finite number of ODE steps and the large guidance weight (wg∈[80,200]w_g \in [80, 200]), Theorem 2 fails to provide a non-degeneracy guarantee for the soft-guided sampling used in practice.

  2. There is no evaluation of the computational overhead associated with online guided sampling. Algorithm 2 requires running Algorithm 1—which involves 10–20 BFN sampling steps with gradient computation—online during each training iteration (batch) to generate masks. The paper does not report the additional wall-clock training time or the GPU memory overhead incurred by the pre-trained BFN and the online sampling process, making it difficult to assess the method's scalability to larger-scale scientific datasets.

Questions for authors
  1. Computational Overhead and Feasibility of Offline Generation (Clarification Request): Compared to random masking or empirical masking, how much does the inclusion of 10–20 steps of BFN-guided sampling during online training increase the total training time? Furthermore, would it be possible to eliminate this online computational overhead—while maintaining performance—by pre-generating a set of guidance masks offline for each training sample and sampling from them randomly during the training process?

  2. Details on Main Model Training and Inference (Clarification Request): Equation (11) in the main text and Algorithm 2 formulate the primary imputation model as a deterministic regression, uϕ(Mctx⊙uobs,Mctx)u_\phi(M_{ctx}\odot u_{obs}, M_{ctx}). However, the abstract and Appendix G (Algorithms 10–12) describe it as a diffusion model that incorporates a timestep ss, denoted as uϕ(s,Mctx⊙xs,Mctx)u_\phi(s, M_{ctx}\odot x_s, M_{ctx}). Could you please clarify the noise-addition mechanism used during the training of the main model? Additionally, which of the inference sampling strategies outlined in Appendix E.7 is employed by the "Ours" method in Tables 1, 5, and 6?

  3. Modeling Choices in Theorem 2 (Clarification Request): Given that Algorithm 1 utilizes a Bernoulli retention rate ρ∈(0,1)\rho \in (0, 1), a finite guidance step size wgw_g, and Gaussian noise initialization—where the continuous latent variables naturally possess full support—why is the proof not based directly on the actual soft-guidance dynamics to show that P(M^i=0)>0P(\hat{M}_i = 0) > 0? Instead, why introduce the hard constraint CkC_k, which is not actually implemented in the algorithm?

5 comments

We sincerely thank the reviewer for your constructive feedback.

W1 & Q3. Theoretical Contributions and Consistency with Algorithm

We acknowledge that there is a tiny gap between the algorithm and the theory. However, our empirical experiments (Sec. 4.4) have already demonstrated their consistency in practice. And we have also provided rationality justification Appendix C.4 (Remark 1)

(1) Empirical results support this consistency.

Our empirical experiments have already provided direct evidence that Algorithm 1 exhibits the non-degenerate sampling behavior consistent with Theorem 2. Specifically, Appendix E.8 evaluates query exposure across all three real-world datasets using numerous ensemble samples. Unlike random partitioning, which exhibits extensive near-zero query-exposure regions, our observation-aligned sampler maintains consistently positive estimated query exposure across valid spatial locations. Furthermore, Appendix E.9 examines different anchor retention rates and shows that stochastic anchors preserve mask diversity, whereas fully retaining observations substantially reduces diversity.

Together, these experiments support that Algorithm 1 preserves the sampling diversity and query-coverage properties underlying Assumption 2 in practice.

(2) Theoretical significance beyond the proofs.

While the positivity results follow naturally from the support assumptions, the contribution mainly lies in identifying the conditions that ensure effective query supervision, rather than in the complexity of the proofs. Specifically, Theorem 1 establishes context-conditioned query positivity under independent mask intersection, while Theorem 2 further characterizes sufficient support conditions under observation-guided partitioning. Together with the learning-dynamics analysis (Appendix D, Eq. 41), these results explain why mask-distribution support matters for avoiding zero-query supervision, providing a principled motivation for our learned mask priors.

(3) Consistency between Theorem 2 and Algorithm 1.

As explicitly clarified in Appendix C.4 (Remark 1), the exact-overlap constraint in Theorem 2 is an analytical abstraction used to characterize the sufficient support condition for positive query exposure, rather than a hard constraint required by Algorithm 1.

In practice, Algorithm 1 preserves the underlying stochastic non-degeneracy through Bernoulli-based stochastic anchors and soft guidance. For high-dimensional observation masks, stochastic anchors remain non-empty with overwhelming probability, while no individual observed coordinate is deterministically retained across sampling runs. Meanwhile, soft guidance encourages observation alignment without imposing fixed-cardinality or coordinate-wise constraints. Together, these mechanisms are designed to maintain the sampling diversity underlying Assumption 2. Therefore, Theorem 2 and Algorithm 1 are consistent in their underlying support-based design principle.

(4) Why use the analytical constraint CkC_k instead of directly proving positivity from the soft-guidance dynamics?

The key distinction is that full support of the initial Gaussian latent does not automatically imply positive probabilities for the final discrete masks. Establishing this directly would require additional assumptions on the support-preserving properties of the complete sampling transformation, including finite-step ODE updates, gradient guidance, and discrete decoding.

More importantly, Theorem 2 aims to characterize sufficient conditions for positive query exposure under observation-guided partitioning, rather than analyze the numerical dynamics of a specific sampler. The constraint CkC_k provides a tractable setting for identifying the essential support condition: every observed coordinate must remain excludable with positive probability under the guided distribution.

This support-level formulation is independent of the particular implementation method, making the theoretical insight applicable to a broader class of observation-guided mask generation methods. A direct proof based on Algorithm 1's specific dynamics would instead be tied to its implementation choices and offer a less general characterization of the underlying partitioning principle.

Q2. Clarification of Imputation Model

We thank the reviewer for pointing out this minor wording error. The term "diffusion-based reconstruction model" in the abstract is overly specific and should be replaced with the more general "imputation model", since our framework is compatible with various different reconstruction approaches. We will revise this typo.

To clarify, all our method's results in Tables 1, 5, and 6 use direct regression (Direct Projection), consistent with Eq. (11) and Algorithm 2. Specifically, the model directly predicts the reconstructed field from the observed context, with MSE supervision applied only to the withheld query region.

Below, we provide the training mechanisms for both two ways:

  • For direct imputation, the training mechanism is: \mathcal L_{\mathrm{direct}}=\mathbb{E}\left[\left\\|M_{\mathrm{qry}}\odot\left(u_\phi(M_{\mathrm{ctx}}\odot u_{\mathrm{obs}},M_{\mathrm{ctx}})-u_{\mathrm{obs}}\right)\right\\|^2\right].

  • For diffusion-based imputation, the noise-addition and training mechanism is: xobs,s=M⊙(αsuobs+σsϵ),ϵ∼N(0,I).x_{obs,s}=M\odot(\alpha_s u_{obs}+\sigma_s\epsilon),\qquad \epsilon\sim\mathcal N(0,I). \mathcal L_{\mathrm{diff}}=\mathbb E_{s,\epsilon}\left[\left\\| M_{\mathrm{qry}}\odot\left(u_\phi(s,M_{\mathrm{ctx}}\odot x_{obs,s},M_{\mathrm{ctx}})-u_{obs}\right)\right\\|_2^2\right].

Thus, the main experimental formulation is consistent throughout. We will correct this ambiguity in the final version.

W2 & Q1. Training Cost

We thank the reviewer for the meaningful suggestion regarding offline mask pre-generation. We acknowledge that online BFN-guided sampling increases per-iteration training time by 2.1×, while introducing only a 0.3% increase in peak GPU memory usage.

We agree that offline mask pre-generation is an effective solution. This eliminates repeated BFN sampling and gradient computation from the online training loop, bringing per-iteration costs close to conventional masking methods. In fact, a related strategy has already been evaluated in Sec. 4.5 (Context Mask Selection Strategy Comparison), where authentic observation masks are randomly selected from the dataset and intersected with the current observation mask to construct context-query partitions. Although this ablation uses empirical rather than BFN-generated masks, it demonstrates the feasibility of constructing partitions from pre-existing masks.

We believe that this is merely an engineering implementation choice that does not alter our core methodological contribution or materially affect reconstruction performance.

I appreciate the authors' response to the review comments. The additional information provided helps clarify some ambiguities present in the original manuscript, particularly regarding the actual training procedure and computational overhead.

However, I still have concerns regarding several key issues.

First, the relationship between Theorem 2 and Algorithm 1 remains insufficiently explained. The authors note that the "fixed intersection constraint" in Theorem 2 is primarily a theoretical abstraction rather than a strict requirement to be enforced by the actual algorithm. While this explanation is plausible, the issue is that the algorithm employs Bernoulli stochastic anchors and soft guidance, and the conditions required by the theorem are not explicitly guaranteed by this sampling process. Although the additional experiments (E.8 and E.9) demonstrate that the sampling approach provides a degree of diversity and query coverage, these are empirical findings; they do not constitute a formal proof that the algorithm satisfies the strict "positive probability" condition stipulated in the theorem. In my view, this remains an area that requires further clarification within the theoretical framework of the paper.

Second, regarding computational overhead, the authors report a 2.1× increase in training time and a 0.3% increase in GPU memory usage; these details are helpful. However, the authors have not provided direct experimental evidence as to whether the training cost could be reduced by pre-generating BFN masks offline while maintaining performance. The existing experiments on "empirical mask selection" are not equivalent to the "BFN-guided mask pre-generation" suggested in the review, and thus do not adequately address this question.

As for the inconsistency in terminology between "deterministic regression" and "diffusion-based imputation," the authors have provided a clear explanation, which I find acceptable.

Overall, while the authors' response addresses some of my concerns, the primary theoretical issue remains unresolved. Given that this concerns the applicability of the paper's core theoretical conclusions. Rather than merely a matter of unclear exposition, I do not find sufficient grounds to alter my initial assessment. Consequently, I maintain my original rating.

Reviewer_faHbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Observation-Aligned Mask Priors for learning physical-field reconstruction from incomplete observations with structured missingness. The method learns a generative prior over authentic observation masks using a Bayesian Flow Network-inspired formulation, and applies cross-entropy guidance based on stochastic observation anchors to generate sample-specific masks. The intersection between the generated mask and the original observation mask defines the context, while the remaining observed entries provide query targets for reconstruction training. The authors additionally establish query-positivity results under support-coverage assumptions. Experiments on three oceanographic datasets at resolutions from 64×64 to 256×256 show consistent improvements over the evaluated baselines. Overall, learning observation-aligned training partitions is a promising idea, but the theoretical connection to reconstruction under MNAR missingness and the interpretation of the query-coverage experiments require substantial clarification.

Critical strengths
  1. The paper addresses an important practical setting in which complete training fields are unavailable and observation masks exhibit structured spatial dependencies. Learning a mask prior from authentic observations is a reasonable approach to improving context–query partitioning beyond independent pixel dropout or predefined block masking.

  2. The experiments demonstrate consistent improvements across three oceanographic datasets and three spatial resolutions. The gains at higher resolutions are particularly encouraging. The comparisons against empirical mask sampling and unconditional mask generation also provide useful evidence that observation-aligned guidance contributes to reconstruction performance.

  3. The supplementary experiments include approximately matched context–query ratios and complementary partitions, helping address a potential confounding factor in the main comparison. The appendices also provide detailed derivations concerning the softmax representation and the relationship between data prediction and noise prediction.

Critical weaknesses
  1. The theoretical guarantees do not fully support the paper’s claims. Positive query probability alone does not establish unbiased reconstruction under MNAR missingness, and the coverage assumptions are not shown to hold for the implemented soft-guidance sampler.

  2. The query-probability experiments require clarification. Independent Bernoulli partitioning already gives every observed coordinate a positive marginal query probability, making the reported “zero-query” regions difficult to interpret. The authors should distinguish directly measured sampling frequencies from classifier estimates.

  3. The empirical evidence remains incomplete. Evaluation on additionally masked observations does not directly validate recovery of originally missing regions under MNAR. A direct comparison with the closely related context–query method, clearer downstream training details, and results across multiple training seeds would strengthen the conclusions.

Questions for authors

See Weaknesses

4 comments

We sincerely thank the reviewer for your thoughtful feedback.

However, we must point out that some of your concerns may stem from a misunderstanding of the theoretical motivation of our work and its connection to the foundational study Incomplete Data, Complete Dynamics: A Diffusion Approach (Zhou et al., 2025). We will briefly clarify this connection before addressing the specific concerns:

The central finding of Zhou et al. (2025) is that learning unbiased conditional expectations from incomplete observations, under the assumed mask-data independence, requires every spatial coordinate outside the context to have a strictly positive probability of receiving query supervision. To achieve this requirement, this prior work proposes that context mask sampling should preserve the structural patterns of the authentic observation-mask distribution.

Our work is directly motivated by this fundamental requirement which has been established. While the prior approach relies on manually designing context-selection strategies based on known observation patterns, authentic occlusions exhibit complex, sample-dependent structures that are almost impossible to characterize heuristically. We therefore learn the authentic observation-mask distribution using a BFN and construct observation-aligned context-query partitions through guided sampling, aiming to preserve realistic mask structures while maintaining positive query exposure. Our theoretical analysis characterizes sufficient conditions for this property, and our experiments further evaluate its effectiveness under authentic occlusions.

W1. Clarification of Theoretical Guarantees and Practical Guidance

(1) Contribution Clarification

As shown above, Zhou et al. (2025) have already established that strictly positive query probability enables unbiased conditional-expectation learning from incomplete observations under mask-data independence assumptions. Building upon this foundation, our work focuses on how to achieve this essential property in real-world applications characterized by complex and structured observation patterns.

(2) Consistency between Theorem 2 and Algorithm 1

We would like to emphasize that Algorithm 1 not only satisfies the intended positivity requirement of Theorem 2 in actual training practice, but also goes beyond this theoretical minimum to improve training efficiency. Our empirical experiments (Sec. 4.4) have already demonstrated their consistency in practice. And we have also provided rationality justification Appendix C.4 (Remark 1)

  • (i) Reliable Query Coverage Throughout Actual Training

Appendix E.8 demonstrates that, even with finite sampling runs, all valid observable locations achieve strictly positive empirical query frequencies across three real-world datasets. More importantly, these frequencies remain appreciably above zero, indicating stable coverage rather than occasional random occurrences. With substantially more steps during actual training, this non-negligible exposure certainly ensures repeated supervision across all observable locations, providing strong empirical support that Algorithm 1 reliably realizes the intended condition of Theorem 2. This will certainly be realized in actual training and it is statistically significant.

  • (ii) Beyond Theoretical Positivity: Sufficient Supervision for Effective Learning.

Theorem 2 establishes only a minimum requirement for unbiased conditional-expectation learning, not a guarantee of reaching the theoretical optimum under finite training. Extremely low query probabilities may satisfy the theorem while yielding insufficient gradient updates in practice (see Appendix D, Eq.41). Algorithm 1 further addresses this limitation by maintaining appreciable query frequencies across observable locations, thereby improving supervision efficiency and practical convergence. Appendix E.9 demonstrates that stochastic anchor guidance preserves mask diversity, which supports this effective query coverage.

  • (iii) Rationality Justification

As explicitly clarified in Appendix C.4 (Remark 1), the exact-overlap constraint in Theorem 2 is an analytical abstraction used to characterize the sufficient support condition for positive query exposure, rather than a hard constraint required by Algorithm 1. In practice, Algorithm 1 preserves the underlying stochastic non-degeneracy through Bernoulli-based stochastic anchors and soft guidance. For high-dimensional observation masks, stochastic anchors remain non-empty with overwhelming probability, while no individual observed coordinate is deterministically retained across sampling runs. Meanwhile, soft guidance encourages observation alignment without imposing fixed-cardinality or coordinate-wise constraints. Together, these mechanisms maintain the sampling diversity underlying Assumption 2.

W2. Clarification of Query Probabilities and Classifier-Based Estimation

The reviewer's concern arises from conflating two fundamentally different probabilities: the query probability conditioned on the original observation mask and the query probability conditioned on the model's input context. These are two different quantities, as emphasized in the context-query learning framework of Zhou et al. (2025).

(1) Why can Bernoulli partitioning still produce zero-query regions?

We agree that Bernoulli partitioning assigns every observed coordinate a positive probability of being queried:

P(Qi=1∣Mi=1)=r>0,P(Q_i=1\mid M_i=1)=r>0,

where MM and QQ denote the original observation mask and query mask, respectively. However, this only describes how an already observed coordinate is randomly selected. What matters for learning is whether a coordinate can receive supervision given the context available to the model (C=MctxC=M_{\mathrm{ctx}}).

Since a query must originate from an observed coordinate,

P(Q_i=1\mid C=c) &=P(M_i=1\mid C=c)\ \cdot P(Q_i=1\mid M_i=1,C=c). \end{aligned}$$ Consider a coordinate $i$ that is unobserved in every original observation mask compatible with a particular context $c$. In this case, $P(M_i=1\mid C)=0$, so $P(Q_i=1\mid C)=0$, regardless of the Bernoulli selection probability. Thus, **positive query probability for observed coordinates does not guarantee positive query probability given a particular context**. **(2) Why do we train a classifier rather than directly count sampled queries?** As described in Sec. 4.4, we train a binary classifier on context-query pairs generated from authentic observation masks in the dataset. For each original observation mask, a partitioning strategy constructs the corresponding context and query masks, which serve as the classifier input and prediction target, respectively. Under binary cross-entropy training, the population-optimal predictor satisfies $f_{\psi}^{*}(C)_i = P(Q_i = 1 \mid C).$ Crucially, our experiment combines classifier-based estimation with ensemble sampling, rather than directly counting sampled queries. Specifically, for a fixed observation mask $M$, we generate N contexts through repeated partition sampling and average their classifier predictions: $$\frac{1}{N}\sum_{n=1}^{N} f_\psi(C^{(n)})_i \approx \mathbb{E}_{C\sim P(C\mid M)}\left[P(Q_i=1\mid C)\right].$$ This differs from directly counting query occurrences, which estimates $P(Q_i=1\mid M)$. For example, a coordinate missing in the fixed observation mask $M$ can never be counted as a query, **but another observation mask may contain that coordinate and produce the same context**, allowing the classifier to assign it a positive conditional query probability. **Direct Bernoulli sampling from the fixed mask cannot capture this cross-sample possibility, because it only partitions coordinates already observed in that mask.** Therefore, **the classifier estimates whether each coordinate can receive query supervision given a context, rather than simply measuring how often it is selected from one fixed observation mask**. This distinction explains both the motivation for our classifier-based analysis and why Bernoulli marginal positivity alone does not address the zero-query issue. And near-zero values indicate limited estimated supervision opportunities across sampled contexts, not necessarily mathematically zero query probabilities. [Reference: Zhou et al. (2025). Incomplete Data, Complete Dynamics: A Diffusion Approach.(ICLR2026)]

W3. Clarification of Empirical Evaluation, Related Comparisons, and Reproducibility

(1) Evaluation

In real-world incomplete datasets, ground-truth values at originally missing locations are inherently unavailable, making direct quantitative evaluation impossible. To address this, our evaluation protocol (Sec. 4.3) overlays an additional authentic observation mask randomly sampled from the dataset, thereby simulating realistic structured occlusions while retaining known observations for evaluation. This enables reliable quantitative comparisons under authentic missingness patterns without requiring fully observed ground truth.

(2) Comparison with related context-query methods

On one hand, our main experiments have already included representative context-query-based methods such as AmbientDiff in Table 1. On the other hand, we have already conducted extensive comparisons of multiple context-query partitioning strategies in ablation study, including pixel-level, block-wise, saliency-driven, empirical-mask, and unconditional-prior partitioning (Table 2 and Appendix E.6). These controlled comparisons follow the same reconstruction framework, isolating the partitioning strategy as the only difference to ensure fair comparison. In addition, we provide detailed algorithm implementations of these partitioning strategies in Appendix E.6.

(3) Performance Significance

Our method outperforms baselines on all three real-world datasets across all three resolutions, with particularly substantial gains at higher resolutions (Tables 5 & 6). At 256 × 256, compared with the strongest baseline for each dataset, it reduces MSE by 37.5%, 28.7%, and 15.8% on three datasets respectively. These consistent and significant improvements across diverse authentic occlusion patterns and high-resolution settings provide stronger evidence of the method's effectiveness than results limited to simplified synthetic settings.

We also add new experiments with three independent seeds, reporting the results as mean ± std on Black Sea CHL (due to limited computational resource constraints). Our extensive evaluations across three real-world datasets, three spatial resolutions, and multiple controlled ablation studies consistently demonstrate substantial improvements, providing strong evidence for the robustness and reliability of our method.

MethodBlack Sea CHL
MSE ↓PSNR ↑
AmbientDiff0.229 ± 0.01017.665 ± 0.194
DINDiff0.310 ± 0.01715.384 ± 0.224
MissDiff0.602 ± 0.01913.542 ± 0.222
Ours0.191 ± 0.00617.852 ± 0.181

Dear Reviewer faHb,

We hope our responses have addressed your concerns. We would greatly appreciate it if you could take a moment to share your thoughts when you have time.

Thank you for your consideration and efforts.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.