acceptodds
Under review as a conference paper at ICLR 2027

Learning Physics-Constrained Collocation Samplers for Dual-Cone Optimization in Physics Informed Neural Networks

Abstract

Physics-informed neural networks (PINNs) are highly sensitive to the choice of collocation points used to enforce PDE residuals and boundary conditions, particularly when trained with modern optimization methods such as dual cone gradient descent that rely on stable gradient geometry. Existing sampling strategies are largely heuristic and fail to adapt to the optimization dynamics induced by these optimizers, leading to poor sample efficiency and unstable training in high-dimensional and complex PDEs. We propose CoDyS (Collocation for Dynamic Sampling), a framework that learns physics-constrained collocation samplers to optimize PINN training under a fixed dual cone gradient descent optimizer. CoDyS formulates sampler design as a bi-level learning problem in which a generator agent proposes structured, physics-valid sampling programs, a PINN agent executes training with a fixed architecture and optimizer, and a critic agent evaluates candidate samplers using held-out PDE residual and boundary losses together with dual-cone stability diagnostics. Final performance is then reported with an independent relative L2 error metric on a disjoint test set. Without modifying the PINN or optimizer, CoDyS consistently improves sample efficiency and training stability compared to space-filling and adaptive sampling baselines across benchmark and high-dimensional PDEs, while remaining compatible with consumer-grade local workstations for open-weight models and lightweight API access for proprietary generators, without requiring model fine-tuning.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Physics-Informed Neural Networks, PINNs, Scientific Machine Learning, Partial Differential Equations

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_VSEwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work propose Collocation for Dynamic Sampling (CoDyS), which is LLM-based collocation sampling method for PINNs training, especially with DCGD optimizer.

Critical strengths

None

Critical weaknesses
  1. Table 1 is described as reproducing the optimizer results of the original DCGD paper [1]. I checked the two tables: all baselines match the original exactly, but the DCGD row does not match. The most notable case is Burgers, where the original reports 0.0124 (0.0046) while this paper reports 0.0304 (0.0040). Other examples are Helmholtz (0.0029 vs. 0.0040) and Helmholtz 3D (0.0774 vs. 0.0739). On Heat 5D only the mean agrees (std 0.0003 vs. 0.0005). This difference is important because If the original DCGD numbers are used, the gain of CoDyS becomes minimal on most benchmarks. The authors should explain why the DCGD results differ from the original paper, and justify the claimed advantage of CoDyS when compared against the original numbers.

  2. The described Heat 5D equation is differ from [1], but the paper use the same number. In the original paper, they use unit-ball domain with Neumann condition whereas this work use [0,1]×[−1,1]5[0,1]\times [-1,1]^5.

Overall, I believe this work is not ready for publication in its current form.


[1] Hwang, Youngsik, and Dong-Young Lim. "Dual cone gradient descent for training physics-informed neural networks." Advances in Neural Information Processing Systems 37 (2024)

Questions for authors

See Weaknesses

Reviewer_rDCGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary
  • CoDyS searches for collocation sampling programs for PINNs.
  • An LLM generator proposes sampler edits.
  • A fixed DCGD optimizer trains the PINN.
  • A critic summarizes loss and gradient diagnostics for subsequent proposals.
  • The outer search selects programs using reference-based validation error.
Critical strengths

None.

Critical weaknesses

1. The DCGD-specific motivation rests on an incorrect geometric condition.

Section 4.1 claims that DCGD requires non-conflicting gradients or a restricted gradient-norm ratio. This misstates Theorem 4.2 of DCGD [1]. That theorem characterizes when the unmodified sum of two gradients belongs to their dual cone. It is not a prerequisite for DCGD to construct a common-descent direction.

Consider

gr=(10,0),gb=(−3,4).g_r=(10,0),\qquad g_b=(-3,4).

Then R=∥gr∥/∥gb∥=2R=\|g_r\|/\|g_b\|=2 and cos⁡ϕ=−0.6\cos\phi=-0.6. The paper's stated condition fails: 2>−1/cos⁡ϕ=5/32>-1/\cos\phi=5/3. Indeed, the raw sum gr+gb=(7,4)g_r+g_b=(7,4) conflicts with gbg_b, since gb⊤(gr+gb)=−5g_b^\top(g_r+g_b)=-5. Yet the DCGD (Center) rule [1] gives

v&=\frac{g_r}{\|g_r\|}+\frac{g_b}{\|g_b\|}=(0.4,0.8),\\ d&=\frac{(g_r+g_b)^\top v}{\|v\|^2}v=(3,6),\\ g_r^\top d&=30>0,\qquad g_b^\top d=15>0. \end{aligned}$$ Thus, the stated condition can fail while DCGD still constructs a direction that decreases both losses to first order. This reveals a fundamental misinterpretation of DCGD's geometric conditions. The authors should revisit the original DCGD analysis and correct their theoretical motivation accordingly. More generally, collocation-dependent gradients arise under any sample-based optimizer. **2. The bi-level objective is ambiguous and admits degenerate interpretations.** Equation (11) minimizes $\mathbb E_{S_t\sim\pi}[L(\theta_T)]$, but does not specify the measure defining $L(\theta_T)$. Eqs. (3)–(6) instead define losses using policy-dependent collocation points. If $L$ is the resulting empirical loss, the formulation can admit degenerate policies that leave physical constraints unobserved. Consider $$-u''(x)=\pi^2\sin(\pi x),\quad x\in(0,1),\qquad u(0)=u(1)=0,$$ with $u_\theta(x)=\sin(\pi x)+\theta x^2(x-\frac32)$. Its PDE residual is $r_\theta(x)=\theta(3-6x)$. A policy concentrating interior samples at $x_r=1/2$ and boundary samples at $x_b=0$ gives $$r_\theta(1/2)=0,\qquad u_\theta(0)=0 \quad\text{for every }\theta.$$ Thus $L(\theta)=0$ for every $\theta$. Yet $r_\theta$ is nonzero away from $x_r=1/2$, and $u_\theta(1)=-\theta/2$ violates the unsampled boundary condition when $\theta\ne0$. Both loss components are sampled, but the empirical objective becomes trivial. This example exposes a degeneracy in the empirical-loss interpretation. Equation (11) does not specify restrictions on $\Pi_{\mathrm{prog}}$ sufficient to exclude concentrated sampling policies. If $L$ instead denotes a fixed continuum loss, its measure and relation to the sampled objectives must be defined. The implemented outer search instead ranks policies by relative $L^2$ error against a reference solution, rather than by the objective in Eq. (11). In most forward PDE problems, the solution is precisely the unknown quantity that the solver is intended to recover. Computing this ranking score therefore requires an oracle (reference solution). Without such information, the proposed sampler-selection procedure cannot be applied as described. **3. The two-objective reduction hides conflicts among physical constraints.** Equation (7) aggregates all non-residual constraints into $g_c=\sum_j\lambda_j g_{c,j}$. DCGD then operates only on $(g_r,g_c)$. A common-descent direction for these two gradients need not decrease each individual constraint. Conflicts among boundary and initial-condition losses can cancel in the aggregate. ConFIG [2] and HARMONIC [3] address individual loss gradients, but lines 186–187 mention their use only as a possible extension. The proposed method does not perform this extension. Moreover, Eq. (7) introduces individual constraint losses without defining their collocation sets, sampling distributions, or how their weights are determined. The sampling formulation is therefore incomplete for PINNs with multiple independently enforced physical constraints. **4. The methodological distinction is unclear, and the exposition obscures it.** The manuscript describe CoDyS as an *optimizer-conditioned program search* that *shapes* the empirical loss landscape. The concrete operation is to change sampled points and thereby change the finite-sample loss. That dependence is elementary and is not unique to geometry-aware optimization. The remaining construction is a generator–critic search over executable sampling programs, using an existing agent-design pattern. The manuscript does not identify a DCGD-specific property of the search rule or a distinct mathematical principle behind the program representation. The text repeatedly defines the contribution by exclusion. Similar qualifications appear in lines 124–132, 168–170, 213–215, 248–252, and 285–287. Section 4.1 also repeats the PINN objective and sampling dependence established earlier. In particular, saying that a property is *not a theorem* or that a method does *not* smooth the global landscape does not explain what mechanism the proposed search establishes. The relationship between the cited generator–critic pattern, the proposed search space, and the DCGD-specific contribution needs a direct and precise account. **References** [1] Hwang et al., "Dual Cone Gradient Descent for Training Physics-Informed Neural Networks". NeurIPS 2024. [2] Liu et al., "ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks". ICLR 2025. [3] Bu et al., "Harmonized Cone for Feasible and Non-conflict Directions in Training Physics-Informed Neural Networks". ICLR 2026.
Questions for authors

1. DCGD geometry. How do lines 279–283 follow from DCGD [1]? The cited condition applies to the raw gradient sum, as the explicit counterexample shows. Which property of CoDyS depends specifically on the DCGD update rather than on sampling-dependent gradients in general?

2. Objective and oracle requirement. Does L(θT)L(\theta_T) in Eq. (11) denote a policy-dependent empirical loss or a fixed population loss? What prevents the one-point degeneracy above? Why does Eq. (11) differ from the reference-solution error used to select programs? How is the latter available for an unknown PDE solution?

3. Individual constraints. What are the exact definitions, sampling domains, and sampling measures of Lc,jL_{c,j} in Eq. (7)? How can two-vector diagnostics detect conflicts hidden by aggregation?

4. Contribution and clarity. What new mathematical or algorithmic principle distinguishes this search from applying a generic generator–critic program search to PINN sampling? Please state the positive contribution directly and remove redundant qualifications and repeated definitions.

Reviewer_7UYVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper builds on the idea that choosing collocation points also changes the gradients that a PINN has to optimize. The authors therefore try to find sampling rules that work well with DCGD, including how points are split between the interior and boundary. An LLM edits sampler programs using feedback from training runs, and reference-solution error is used to choose the best program. The experiments cover five PDEs.

Critical strengths
  1. Choosing the sampling rule with the optimizer in mind makes sense. Interior and boundary points affect the two loss gradients, so there is a reasonable motivation for choosing them together.
  2. Table 2 reports encouraging improvements over residual-adaptive sampling on three equations. Keeping the network and optimizer fixed also helps make the effect of changing the sampler easier to study.
Critical weaknesses

1)I would like to understand the idea behind the gradient feedback better. DCGD already adjusts conflicting gradients, so what do the authors expect better sampling to improve here? Does it reduce how much DCGD has to change the update, or help in some other way? Table 2 suggests that the feedback matters, but I could not find the gradient summaries or separate critic in the supplied implementation. The feedback I found contains reference error and loss summaries. This makes it hard to follow how the proposed mechanism leads to the reported gains. 2) I am unsure how much of the improvement carries over after sampler selection. Appendix B.4 says the reference grid is queried during search and then used to report the selected sampler’s error. The abstract says the test is independent, so I’m not sure which setup was used. The results could still reflect better sampling, but the current evaluation does not tell me how much the repeated selection affects the reported gain. It would also help to explain the intended use when a reference solution is unavailable. 3) I could not match some of the baseline results to those in the cited DCGD paper. For example, Table 1 gives a Burgers error of 0.0304, while the published paper reports 0.0124. Its 5D heat problem also uses Neumann conditions on a ball, whereas the supplied code uses Dirichlet conditions on a cube. Were these results taken from another configuration? It would be helpful if the authors can detail which comparisons use which problem and training setup.

Questions for authors
  1. Could you explain what you expect the gradient feedback to change during sampling search? A worked example, or a result connecting the feedback to the learned rule would help. It would also help if you could point me to the code used for the experiments in Table 2.
  2. Do you have results for the selected sampler on points and training seeds that were not used during search? 3)Could you clarify the source and setup of the DCGD baseline results? and Do you have results for CoDyS using the same problem setup and training settings as the DCGD baselines? If so, does CoDyS still show similar improvements?
  3. Does repeating the search tend to produce similar sampling rules, or can quite different rules give the same improvement? Feel free to include other justifications that you feel would be relevant. I currently lean towards reject because I am not completely sure how the feedback helps and how the gains are measured. The sampling idea seems worth pursuing, and further explanation/evidence from the authors may change my assessment. In any case I believe the writing needs a lot of improvement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.