- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes DEPTH, a weight-shared recursive transformer for predicting spatially indexed PDE solution fields. The model encodes the input conditions once, then propagates a hidden state sequentially along a physical coordinate using a shared transformer block, with predictions and supervision at each spatial slice. Experiments on thermo-mechanical stress, warpage, and electrostatic field datasets show competitive accuracy and computational efficiency compared with several recursive transformer and recurrent baselines.
Critical strengths
- The proposed architecture is simple and computationally efficient. By sharing the transformer block and carrying hidden states across spatial slices, it avoids some redundant computations compared with predicting each slice independently.
- The paper provides reasonable ablation studies on coordinate injection, per-slice supervision, recursion depth, and gating mechanisms, which help explain the effects of different design choices.
Critical weaknesses
- I find the methodological novelty somewhat limited. The proposed approach looks quite similar to a coordinate-conditioned recurrent latent-state model, with a transformer block used as the recurrent update. Weight sharing, hidden-state propagation, and per-step supervision are all well-established ideas. It is not entirely clear to me what the main technical novelty is compared with existing recurrent and latent-state prediction methods.
My major concern is the motivation and justification for sequentially predicting spatial slices. The considered thermo-mechanical and electrostatic problems are governed by globally coupled, largely elliptic equations. These problems do not naturally have a one-directional information propagation process along a spatial coordinate. Therefore, predicting the solution through a single sequential sweep in a fixed direction seems to impose an artificial ordering that may not properly capture global interactions and dependencies.
I am not convinced that the paper sufficiently addresses this issue. For such globally coupled problems, the solution at one location can depend on conditions throughout the entire domain, rather than just information propagated from previous slices. The paper provides little analysis of how the proposed approach handles these nonlocal dependencies or maintains global physical consistency. It would be important to examine the effects of prediction direction, reverse/bidirectional propagation, or some form of global/iterative refinement.
- I also find the experimental validation rather limited. The tested problems involve relatively simple, structured, low-resolution fields (13 × 13 grids with only 15–16 slices) and very few input parameters. It is unclear whether the method would remain effective for much higher resolutions, more complex geometries, heterogeneous problems, or stronger global coupling.
The baseline comparisons are also not fully convincing. Most comparisons are against recursive transformer variants, while the basic RNN baseline has much lower representational capacity. I would expect comparisons with stronger and more relevant methods, such as coordinate-conditioned models, modern local-global neural operators, and latent/compressed transformer-based models.
In addition, the evaluation mainly focuses on latent-space retrieval metrics, which do not directly reflect the accuracy or physical consistency of the predicted PDE solutions. More detailed field-wise errors, physics-related evaluations, and actual computational timings would be helpful to better support the claimed accuracy and efficiency.
Questions for authors
-
What is the physical motivation for predicting spatial slices sequentially in one direction, especially for globally coupled PDE problems? How does the model capture nonlocal dependencies and global effects of boundary conditions? Have the authors considered reverse/bidirectional propagation or iterative refinement?
-
How does DEPTH perform on higher-resolution fields, longer sequences, nonuniform spatial discretizations, and more complex geometries? In particular, can a model trained on 16 slices generalize to substantially more slices without retraining?
-
Could the authors compare against stronger, capacity-matched baselines, including modern coordinate-conditioned models, neural operators, and latent/compressed transformer approaches? How would the results change with a more faithful TRM implementation?
-
Could the authors report more comprehensive field-wise prediction errors, physical-consistency measures, and actual training/inference times, rather than mainly retrieval scores and estimated FLOPs? Also, could the authors clarify the test-based run selection in Appendix B.3 and report results consistently across all independent runs?
14 comments
Weakness 1 (major concern) and Q1
(What is the physical motivation for predicting spatial slices sequentially in one direction, especially for globally coupled PDE problems? How does the model capture nonlocal dependencies and global effects of boundary conditions? Have the authors considered reverse/bidirectional propagation or iterative refinement?)
| Concern | Response | Location |
|---|---|---|
| "What is the physical motivation for predicting spatial slices sequentially?" | All slices of a design come from a single global FEA solve (Table 6), so no slice is physically computed before another. DEPTH reflects this: it encodes the full design once as the initial state H(0), so every step starts with global information about the whole domain. Predicting slices in sequence is therefore an inductive bias, not a model of information travelling through the material. It encodes two physical properties. First, neighbouring slices are strongly correlated, so carrying a state from one slice to the next lets the model predict each slice as a change from the previous one rather than from scratch. Second, in the package, the stress in each layer is set by its CTE mismatch with the layers above and below it, so visiting slices in physical order lets the state follow this layer-to-layer structure. The No-carry ablation, which predicts each slice independently, supports this: removing the carry lowers MAP from 0.617 to 0.11 (Table 1). | Sec. 3.1, Eq. 1; Sec. 6 |
| "governed by globally coupled, largely elliptic equations" | The two benchmarks pose different problems. Package stress and warpage are governed by the static equilibrium equations of linear thermoelasticity over a heterogeneous 3D domain: a stack of materials (die, moulding compound, underfill, redistribution layer) whose stiffness and CTE jump at each interface. The CTE mismatch drives stress concentrations at these interfaces. The equations are elliptic, but the field changes sharply from layer to layer, so the slices are physically distinct. The PINN benchmark instead solves Laplace’s equation on a homogeneous domain. Each slice is an independent 2D solve at a different plate width d, so the slices are not spatially coupled; moving from one slice to the next follows a smooth change in a geometric parameter. DEPTH gives the best or near-best MAP on both. | Sec. 6; Appendix C.2–C.3; Tables 2–3 |
| "maintains global physical consistency" | For PINN, we test the governing equation directly on every predicted slice with the 5-point residual at each interior node, averaged as . The autoencoder-reconstructed ground truth gives the reference floor (). DEPTH-1b tracks it at every slice (ratio 0.985). UT3x-1b matches it only on average, through errors that cancel between shallow and deep slices (1.004). RNN-emb exceeds it at every slice, with the gap growing with depth from about 10% to 17% (1.114). | Sec. 6, Eq. (X); Fig. 4 |
Table 6: State update of DEPTH and its three ablated variants
| Variant | Update |
|---|---|
| DEPTH | |
| No-attn | |
| No-coord | |
| No-carry |
Figure 4: Laplace residual on PINN
| Model | |r| (×10⁻³) | Ratio |
|---|---|---|
| GT | 5.90 | 1.000 |
| RNN-emb | 6.58 | 1.114 |
| UT3x-1b | 5.93 | 1.004 |
| DEPTH-1b | 5.82 | 0.985 |
Table 1: Component-removal ablations of DEPTH on Stress10K
| R@1 ↑ | R@5 ↑ | MAP ↑ | RMSE ↓ | SSIM ↑ | Peak ↓ | |
|---|---|---|---|---|---|---|
| RNN-emb | 0.40 ± 0.031 | 0.73 ± 0.021 | 0.55 ± 0.027 | 0.0157 ± 0.0005 | 0.95 ± 0.003 | 0.065 ± 0.003 |
| DEPTH (full) | 0.48 ± 0.077 | 0.80 ± 0.063 | 0.62 ± 0.068 | 0.0132 ± 0.0018 | 0.96 ± 0.0074 | 0.057 ± 0.0086 |
| No-attn | 0.29 ± 0.082 | 0.61 ± 0.088 | 0.44 ± 0.083 | 0.0204 ± 0.0072 | 0.92 ± 0.053 | 0.082 ± 0.017 |
| No-coord | 0.35 ± 0.029 | 0.69 ± 0.0032 | 0.50 ± 0.020 | 0.0189 ± 0.0025 | 0.94 ± 0.0019 | 0.074 ± 0.0045 |
| No-carry | 0.05 ± 0.020 | 0.15 ± 0.041 | 0.11 ± 0.031 | 0.0597 ± 0.021 | 0.82 ± 0.032 | 0.157 ± 0.024 |
Weakness 1 (major concern) and Q1
(What is the physical motivation for predicting spatial slices sequentially in one direction, especially for globally coupled PDE problems? How does the model capture nonlocal dependencies and global effects of boundary conditions? Have the authors considered reverse/bidirectional propagation or iterative refinement?)
| Concern | Response | Location |
|---|---|---|
| "seems to impose an artificial ordering" | We tested the ordering directly. With no propagation at all (No-carry DEPTH, each slice predicted independently), MAP falls from 0.62 to 0.11; with a random slice order it is 0.51. Propagation along neighbouring slices is therefore essential, and removing the ordering hurts rather than helps. | Sec. 5; Sec. 6; Table 1 |
| "Have the authors considered reverse ... propagation?" | Yes. Bottom-up and top-down sweeps perform the same: MAP 0.62 vs. 0.61, R@1 0.48 vs. 0.47, with RMSE and SSIM equal within one std. The result does not depend on direction, consistent with load transfer having no preferred direction. | Sec. 6; Table 1 |
| "How does the model capture nonlocal dependencies and global effects of boundary conditions?" | Global conditions enter through H(0) and are available at every step. In PINN, the boundary-defining parameters (a, b) are encoded in H(0), while the plate width d, which moves the boundary junction, is the slice coordinate. Because each target slice is a full-domain solution, the per-slice loss trains every state on the global field. If one-directional propagation lost global information, errors would build up towards the end of the sweep; instead, DEPTH matches the ground-truth physics at every slice, including the last (next row). | Sec. 3.1; Sec. 6; Fig. 4 |
| "bidirectional propagation or iterative refinement?" | We added an iterative refinement pass after the sweep. The standard sweep gives for . A second pass then recomputes each slice from both of its neighbours, using the same block: , with weighting the slice below and the slice above (the top slice uses its lower neighbour only). This is a learned Jacobi relaxation step: every slice is updated from the first-pass states of its neighbours, and gives the prediction. In a preliminary run it reaches MAP 0.55, within the range of a single sweep (0.617 ± 0.068); we will evaluate it over multiple seeds for the final version. Together with the direction test, this suggests that a single sweep from the global initial state already captures most of the coupling at this scale. | Sec. 6; new experiment (rebuttal) |
Table 1: Sweep order along for DEPTH on Stress10K
| R@1 ↑ | R@5 ↑ | MAP ↑ | RMSE ↓ | SSIM ↑ | Peak ↓ | |
|---|---|---|---|---|---|---|
| Bottom-up | 0.48 ± 0.077 | 0.80 ± 0.063 | 0.62 ± 0.068 | 0.0132 ± 0.0018 | 0.96 ± 0.0074 | 0.057 ± 0.0086 |
| Top-down | 0.47 ± 0.026 | 0.79 ± 0.013 | 0.61 ± 0.020 | 0.0124 ± 0.0006 | 0.97 ± 0.0030 | 0.050 ± 0.0027 |
| Random | 0.36 ± 0.10 | 0.68 ± 0.12 | 0.51 ± 0.10 | 0.0164 ± 0.0032 | 0.95 ± 0.014 | 0.072 ± 0.010 |
Q3: Stronger, capacity-matched baselines. (Could the authors compare against stronger, capacity-matched baselines, including modern coordinate-conditioned models, neural operators, and latent/compressed transformer approaches? How would the results change with a more faithful TRM implementation?)
We thank the reviewer for this suggestion. All new models are within 2% of DEPTH-1b's parameter count, use the same per-slice loss and optimiser, and predict the same frozen-autoencoder latent.
For transparency, Appendix A.3 provides complete forward and backward flowcharts for DEPTH and each ablation (No-carry, No-coord, No-attn), showing exactly which component each variant removes. Likewise Appendix A.4 for UT.
Table 10: Capacity-matched comparison on Stress10K
Table 11: How each model in the capacity-matched comparison