acceptodds
Under review as a conference paper at ICLR 2027

CoSGF: 4D Scientific Gaussian Fields for High-Resolution Reconstruction from Complementary Observations

Abstract

High-fidelity numerical simulations resolve the spatiotemporal structures needed to study complex physical systems, but generating and retaining long high-resolution trajectories can be computationally and storage intensive. Existing approaches reduce data requirements through augmentation, few-shot learning, flexible representations, or physical priors, but reconstruction is often organized around a dominant observation pathway; increasing model capacity or adding auxiliary constraints alone cannot recover information that is not identifiable from coarse measurements. We instead formulate scientific-field reconstruction as the joint inference of a shared physical field from heterogeneous but complementary scientific evidence. We instantiate this formulation with a 4D Scientific Gaussian Field whose primitives encode physical variables rather than appearance. Numerical measurements and retained scientific visualizations constrain the same field through their corresponding forward processes, while dataset-specific physical and geometric knowledge further restricts residual ambiguity. Rather than treating physics as an unrestricted third loss, physics-driven optimizer displacements are coordinated using the local sensitivity of active observations and finite forward verification, so refinement preferentially acts on weakly observed degrees of freedom while preserving evidence-supported content. The HR field remains inaccessible during reconstruction and is used only for evaluation, while the framework directly produces an analysis-ready high-resolution scientific field. In the reported comparison over four single-phase datasets at and reconstruction ratios, CoSGF achieves the highest mean PSNR among the compared methods.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Scientific Field Reconstruction, Scientific Machine Learning, 4D Gaussian Representation, Weakly Supervised Learning, Scientific Visualization

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_c4VSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoSGF, a 4D Gaussian representation for reconstructing high-resolution scientific fields from coarse numerical measurements and scientific visualizations. Both observation types constrain a shared physical field, while physics-guided refinement is restricted to weakly observed directions to preserve the available evidence. The method is evaluated on four scientific datasets at 4x and 8x reconstruction, showing improvements over existing methods, particularly at 8x, with supporting ablations and qualitative comparisons.

Critical strengths

Combining coarse numerical observations and scientific visualizations to reconstruct a shared physical field is an interesting idea. The use of Gaussian primitives to represent physical variables rather than appearance is also meaningful, particularly for scientific reconstruction where fine-scale information is often unavailable. The sensitivity-based approach to prevent physics-driven updates from overriding observed information is a useful contribution. The quantitative results are encouraging, especially at 8x reconstruction. For example, isotropic turbulence improves from 20.99 dB to 25.99 dB over PINGS-X. The ablations also show that physical refinement and observation protection contribute to the method.

Critical weaknesses

CoSGF uses both numerical measurements and scientific visualizations, while it is unclear whether the compared baselines receive equivalent information. Since the visualizations are generated from the high-resolution reference field, they provide additional information beyond the coarse numerical observations. This is a reasonable reconstruction setting, but the comparison should make this distinction clearer. Moreover, removing visualization supervision decreases 8x PSNR only from 30.66 to 30.54 dB in Table 4. This raises a question about how much of the reported improvement comes specifically from complementary observations rather than the representation or optimization.

Questions for authors
  1. Do the compared baselines have access to the same visualization observations? If not, could the authors clarify how much of the improvement is attributable to the additional information versus the proposed reconstruction method?

  2. Table 2 reports results over 20 runs. Are these independent runs, and can the authors provide the corresponding variability? Also, could the authors clarify the missing Tangaroa configuration in Table 1?

  3. How is the observation sensitivity operator in Eq. (13) computed in practice? Is the constrained update solved directly or approximately, and how are the observation leakage thresholds selected?

2 comments

Response to Reviewer c4VS

Paired values are reported as 4× / 8× in that order. The slash separates reconstruction ratios and does not denote division.

W1 / Q1. Do baselines receive the same visualizations, and how much of the gain comes from additional information?

In the original submission, baselines used LR numerical observations only and did not receive the retained visualization images. Their native network formulations do not support the differentiable-rendering input used by CoSGF. The current comparison is limited to the completed LR-only baseline results and the CoSGF ablations.

Table 4 measures the incremental effect of visualization within the same representation and optimization. The 8× PSNR changes from 30.66 to 30.54 dB, and the 4× result changes from 33.07 to 32.99 dB. LPIPS changes from 0.142 / 0.168 to 0.176 / 0.203. The relative contribution of each modality depends on the dataset and metric. LR measurements and physical constraints can dominate field-level PSNR, while visualization supplies complementary information about fine-scale structures. The shared-field and protection ablations provide the completed method evidence.

Q2a. Are the 20 runs independent, and what is the variability?

The 20 runs use independent random seeds, with the data, Gaussian budgets, optimizers, schedules, stopping rules, and evaluation frames held fixed. HR data are used only for evaluation.

The revised Table 2 reports mean ± standard deviation and 95% confidence intervals. The ranges are 0.05–0.14 dB for individual datasets and 0.04–0.08 dB for the four-dataset aggregate. We use Welch t-tests. The p values are:

Settingp value
Sphere 4×0.18
Sphere 8×<0.001
Isotropic 4×0.91
Isotropic 8×<0.001
Tangaroa 4×<0.001
Tangaroa 8×0.004
Half-cylinder 4×<0.001
Half-cylinder 8×0.008

The Sphere 4× and Isotropic 4× differences are within the observed run-to-run variation.

Q2b. Missing Tangaroa configuration

The revised Table 1 lists Tangaroa as an incompressible velocity sequence with variables u, v, and w, an HR grid of 300×180×120 over 201 time steps, 4× and 8× grids of 75×45×30 and 38×23×15, and evaluation frames 196–200. We remove the provisional wording and use this verified configuration in Table 2.

Q3. How is Eq. (13) computed?

The implementation answers the reviewer’s questions in two parts.

  1. Observation sensitivity and constrained update. We first fit the numerical downsampling operator D_d, add the differentiable rendering constraint, and then apply physics refinement while protecting active observation blocks. O_k collects the active numerical and visual observation blocks. We do not form the full Jacobian. JVPs compute A_kv and VJPs compute A_k^T w. 2. Forward verification and thresholds. Each candidate update is re-evaluated with the nonlinear numerical and differentiable rendering operators. Updates exceeding the step or cumulative leakage budget are reduced or rejected. The thresholds are defined relative to the observation residual after fitting and selected without HR data.

The code will be released after acceptance, consistent with the paper’s current availability statement.

Thanks for your detailed response. It addresses some of my concerns. I will maintain my positive score.

Reviewer_Ls1kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CoSGF reconstructs high-resolution 4D scientific fields from two kinds of evidence: downsampled fields and projection-based visualizations. The field is represented as a set of 4D (xyz-t) Gaussian primitives that carry physical variables rather than RGB appearance. Two differentiable forward operators fit this representation to the evidence. A block-averaging operator D_d reproduces the coarse measurements, and Gaussian-splatting volume rendering reproduces the visualizations. The central argument is that many high-resolution fields are consistent with the same coarse data, and the renderings supply the extra constraints needed to disambiguate fine-scale structure.

Critical strengths

The problem setting is interesting and, to my knowledge, novel. The evidence-preserving physics step, which combines a constrained projection with forward verification, is a thoughtful and original design. The evaluation also goes beyond PSNR to include volume renderings and streamline topology, which are more meaningful to domain scientists.

Critical weaknesses
  1. Practical motivation of the problem setup. My main concern is that it is unclear when this setting arises in practice. The method assumes simultaneous access to a low-resolution field and high-resolution visualizations of the same evolving scene. If the low-resolution field comes from a cheaper, low-fidelity simulation run to save compute, its trajectory will generally drift from the high-fidelity dynamics or real observations, so the two sources would not be well aligned. Handling that misalignment would require an explicit mechanism, which the paper does not consider. I would ask the authors to describe concrete use cases and, ideally, to evaluate robustness when the coarse field and the visualizations are misaligned.

  2. Fairness of the comparison. The comparison with baselines is not apples-to-apples. CoSGF receives renderings of the high-resolution ground truth, which is information the baselines never see. The paper neither evaluates baselines given equivalent visual input (e.g., a super-resolution model conditioned on the same renderings, or a baseline optimized with the same rendering loss) nor quantifies how much high-resolution information the renderings carry, for instance through an ablation over the number of views or rendering resolution. As a result, it is hard to tell how much of the improvement comes from the method itself versus the additional input. I consider this a significant issue for the paper’s empirical claims, and addressing it would substantially strengthen the submission.

Questions for authors
  1. When does this setup arise in practice?
  2. Are you claiming methodological improvements or just the new reconstruction setup?
1 comment

Response to Reviewer Ls1k

Paired values are reported as 4× / 8× in that order. The slash separates reconstruction ratios and does not denote division.

W1 / Q1. Practical use and possible misalignment

A concrete use case is in situ scientific visualization. CoSGF does not combine two separately simulated trajectories. The LR field is obtained by downsampling the same physical field that produces the retained visualization, so the observations share the same state and time steps. The numerical downsampling operator D_d and the differentiable rendering operator describe these two observation processes.

Combining a low-fidelity simulation with high-fidelity observations from a different trajectory is a separate data-assimilation problem. It requires an explicit model-error or alignment term and is outside the scope of this paper. We state this limitation explicitly.

We also add a calibration study on Sphere Re300 at 8×:

PerturbationCoSGFCoSGF w/o VisBest LR-only baseline
None39.1439.0237.07
LR noise 1%38.9538.9036.95
Offset ±1 / ±2 frames38.80 / 38.2039.02 / 39.0237.07 / 37.07
Pose 2° / TF shift 5%39.00 / 38.8539.02 / 39.0237.07 / 37.07

CoSGF remains above the LR-only baseline under the tested perturbations. Temporal, pose, and transfer-function mismatch can make the visual term less reliable than the LR-only variant, and we report this limitation.

W2. Fairness and information carried by renderings

The baseline formulations accept LR numerical observations but do not support the differentiable visualization input used by CoSGF. The current empirical comparison remains in the native LR-only setting. We limit the claim accordingly and use the completed -Shared ablation and refinement comparisons to assess the reconstruction method itself.

Q2. Method improvement or only a new setup?

CoSGF improves reconstruction by making the LR measurement, differentiable visualization observation, and physical constraint act on one shared 4D field. The compared baseline formulations accept LR-only inputs and cannot directly use this visual observation. The completed ablations support the method claim.

In Table 4, Full, -Vis, -Phy, -Shared, and -Protect give 33.07 / 30.66, 32.99 / 30.54, 33.01 / 30.57, 32.92 / 30.42, and 33.00 / 30.53 dB at 4× / 8×. LPIPS, physical residual, and leakage change in the corresponding ablations. Table R4 replaces the refinement with weighted physics loss, PCGrad, ConFIG, and projection without forward verification.

Comments on the paper

1 comment

Response to Reviewer 5JQZ

W1. Repackaging graphics tools and lack of a physics-specific representation

Gaussian kernels and differentiable Gaussian splatting are components of CoSGF. Our contribution is a joint inference framework in which LR numerical measurements, differentiable scientific-visualization observations, and dataset-specific physical constraints act on one shared 4D physical field.

Each primitive carries physical variables and mass on a joint (x,t) support, so one set represents a trajectory. The numerical measurements act on the field through the numerical downsampling operator D_d. Because the rendering process is differentiable, image-space errors are propagated through the renderer to the same physical variables and jointly constrain the reconstruction. Rendering attributes come from the physical state through the prescribed transfer function rather than from an independent RGB representation. Dataset-specific P_d operators encode incompressibility, boundaries, geometry, and the bubble-case VOF formulation.

Table 4 gives Full versus -Shared as 33.07 / 30.66 versus 32.92 / 30.42 dB, with LPIPS changing from 0.142 / 0.168 to 0.181 / 0.214. We do not claim a new symplectic or differential-form discretization. The contribution is the joint modeling and inference of these scientific observations.

W2. Evidence-preserving refinement is repackaged constrained optimization

We do not claim a new constrained-optimization theory. The method preserves measured scientific observations while optimizing numerical, visualization, and physical constraints on one shared field. A single constraint leaves unobserved degrees of freedom. Their intersection narrows the admissible solution.

The tentative physics update is projected under the optimizer metric while limiting changes in active numerical and visual observations. Nonlinear forward checks and a cumulative leakage budget prevent drift. PCGrad and ConFIG address gradient conflicts. Our procedure protects the scientific observations themselves.

The ablation supports this design. Full, -Vis, -Phy, -Shared, and -Protect give 33.07 / 30.66, 32.99 / 30.54, 33.01 / 30.57, 32.92 / 30.42, and 33.00 / 30.53 dB at 4× / 8×. LPIPS and leakage change with the removed component. Leakage is 0.7% for Full and 3.2% for -Protect. Table R4 compares weighted physics loss, PCGrad, ConFIG, and projection without forward verification under the same PSNR, leakage, and physical-residual protocol.

W3. The problem formulation is artificial

Reconstructing a high-resolution field from reduced simulation outputs is a common task in large-scale scientific computing. In situ systems such as Cinema, ParaView Catalyst, and Ascent generate visualization under I/O and storage constraints, while spatially reduced numerical data are retained for analysis. In our formulation, the numerical downsampling operator D_d performs block averaging or spatial downsampling to map the high-resolution field to the stored LR data. The differentiable renderer maps the same field to the retained visualization.

W4. Ill-posedness of inverse volume rendering

Recovering a physical field from renderings alone is non-unique. CoSGF avoids this image-only inverse problem by combining LR numerical measurements, differentiable visualization observations, and physical compatibility on one shared field. LR data constrain coarse content, visual observations constrain complementary fine-scale structures, and the physical operator restricts the remaining solutions. Together, these modalities resolve ambiguities that any one of them would leave unresolved.

W5. Baselines do not receive the renderings

In the original comparison, the baselines used LR numerical observations and did not receive the retained visualization images. Their native formulations accept LR inputs and do not provide the differentiable-rendering interface used by CoSGF. The revised description lists each method’s inputs and regularizers.

The current empirical claim is limited to the completed LR-only comparison and the CoSGF ablations. Table 4 measures the roles of visualization, physical compatibility, the shared field, and evidence protection under fixed initialization and optimization budgets. HR fields are never used for initialization, optimization, tuning, or early stopping.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.