- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes POLYGATE, a neural network architecture that combines affine transformations with learnable continuous power-product gates. The authors introduce positive-domain normalization, dimension-normalized gating, and a staged optimization strategy to improve stability. The method is evaluated on physical-law regression, synthetic monomial extrapolation, tabular datasets, and vision benchmarks. The results show improvements on some physical-law tasks and ConvNeXt models, while the benefits are less consistent on other tasks.
Critical strengths
-
The paper addresses a relevant problem in scientific machine learning. Explicitly modeling multiplicative interactions and power laws can provide a useful inductive bias for extrapolation.
-
The paper combines theoretical analysis with experiments across several domains. The integration of power-product gating into convolutional architectures is interesting, and the results on some physical-law and ConvNeXt tasks are promising.
Critical weaknesses
-
The novelty is limited compared to existing product units, Neural Power Units, and polynomial neural networks. The paper does not directly compare against the closest related methods, making it difficult to assess the advantage of the proposed combination.
-
The theoretical claims are not fully consistent with the implementation. In particular, the exact monomial representation results do not account for the numerical clamps used in the implemented operator.
-
The experimental reporting contains several inconsistencies. Table 2 specifies 10 epochs for the AI-Feynman experiments, whereas Table 6 specifies 4 warm-up and 4 specialization epochs for Feynman, and Figure 3 shows an 8-epoch run. In addition, the variance calculation in Appendix I gives approximately 675.3 for the stated settings, while the text claims a value above 1,000. Appendix H.3 makes a similar claim at rank 128 without specifying the parameters needed to verify it.
-
The empirical results are mixed. POLYGATE often obtains negative OOD (R^2) on the controlled monomial tasks and underperforms standard MLPs and tree-based methods on tabular datasets. The vision gains are modest in some settings, while the larger ConvNeXt improvements come with substantial parameter and latency overhead.
Questions for authors
1- How do the authors reconcile the exact monomial representation claims with the numerical clamps used in the implementation? Can they clarify which formulation was used in the experiments?
2- Why is POLYGATE not directly compared against existing product units, Neural Power Units, and polynomial neural networks under matched experimental settings?
3- Can the authors clarify the inconsistencies in the reported training budgets and variance calculations, and provide the exact configurations used for the main results?
4- Can the authors provide ablations to isolate the contributions of positive-domain normalization, geometric-mean gating, and DPBO?