- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TRIO, a representation-learning framework whose predictors retain expressive nonlinear forward modeling while admitting exact finite-union-of-ellipsoids preimages.
Critical strengths
This paper is generally clear-written. It is easy to follow the motivation and the technical proposal, which are pretty matching, in the introduction. And the math formulation is natural and clear.
The experiments are also reasonably complete, covering forward accuracy, preimage quality, computational benefit, sensitivity to backbone complexity, and an AC power-flow application, and giving good support to the claims.
Critical weaknesses
The practical scope is still somewhat limited. Most is low-dimensional or structured, while the universal approximation rate scales as , leaving scalability to genuinely high-dimensional inverse problems unclear.
The richer anisotropic/neural radial parameterization is not sufficiently isolated from the much simpler isotropic quadratic subclass that already enjoys the paper’s main universality and preimage guarantees.
Questions for authors
-
How much empirical benefit comes specifically from learned anisotropic metrics and expressive radial functions compared with the isotropic quadratic TRIO subclass at matched or parameter count?
-
For the bounded-domain optimization experiments, can the authors clarify whether the reported solutions retain exact global-optimality guarantees once the ellipsoidal preimages are intersected with additional domain constraints?
4 comments
W2: Benefits of anisotropic geometry and expressive radial transformations
Thank you for this excellent suggestion! We conducted a new ablation experiment, which provides valuable insights into the different contributions of learned (i) anisotropic geometries and (ii) expressive neural radial transformations.
Specifically, we compare three TRIO variants on the Six-Hump Camel benchmark at , with identical training configurations as in our main studies: (i) the restricted isotropic quadratic subclass from Theorem 4.3, (ii) learned anisotropic quadratic components, and (iii) the full anisotropic neural radial TRIO. We evaluate forward test RMSE and mean ground-truth preimage IoU across target levels.
| Method | Test RMSE ↓ | Mean IoU ↑ |
|---|---|---|
| Isotropic quadratic TRIO | 0.36954 ± 0.14745 | 0.71650 ± 0.05574 |
| Anisotropic quadratic TRIO | 0.08893 ± 0.00173 | 0.89756 ± 0.00916 |
| Neural radial anisotropic TRIO | 0.06102 ± 0.02351 | 0.93850 ± 0.02736 |
| Improvement vs. isotropic quadratic | 83.49% reduction | 30.98% increase |
| Improvement vs. anisotropic quadratic | 31.38% reduction | 4.56% increase |
The results demonstrate two important findings:
- (i) Learned anisotropic geometry substantially improves finite-component expressiveness. Compared with the isotropic quadratic subclass, learning anisotropic metrics strongly improves both forward accuracy and preimage reconstruction. This confirms that adapting the shape and orientation of individual components is highly beneficial at finite , even though the simpler subclass already satisfies our universality theorem.
- (ii) Neural radial transformations provide additional, substantial improvements. Moving from anisotropic quadratic to neural radial TRIO further reduces forward RMSE by 31.38% and improves preimage IoU by 4.56%.
Importantly, the latter corresponds to an approximately 40% reduction in the remaining IoU error (1-IoU). Thus, this improvement is meaningful even when preimage reconstruction is already highly accurate. The results highlight that expressive radial transformations further improve forward modeling and the target-dependent refinement of preimages, but do not sacrifice their exact geometric representation.
These findings demonstrate that the richer parameterization offers substantial practical benefits beyond the theoretical universality guarantees, particularly at finite model capacity.
Action: We add the new ablation study in our new Table 3 in our revised Section 5, to highlight the benefits of anisotropic geometries and expressive radial transformations.
Responses to Questions:
Q1: Benefits of anisotropic geometry and expressive radial transformations
Thank you. We kindly refer to our answer W2.
Q2: Global optimality under bounded-domain constraints
Thank you. We would like to clarify that global optimality is preserved under the additional domain constraints considered in our experiments. Importantly, the exact optimization decomposition in Equation (9) holds for arbitrary input domains and objectives , since
where denotes the full ellipsoid in . Thus, global optimization over the complete learned preimage remains equivalent to independent regionwise optimization, which includes all domain constraints.
In our experiments, the additional domain constraints are convex: the unit disk for the complex-power benchmark and the box for AC power-flow. Since ellipsoids are convex, their intersections with these domains remain convex. Furthermore, the corresponding objectives are convex (Euclidean projection and linear optimization). Hence, each regionwise problem is a convex optimization problem and can be solved globally, which yields the global optimum over the complete, potentially nonconvex learned preimage.
The closed-form expression in Theorem 4.2 assumes , but the exact geometric decomposition and global-optimality remain valid for the bounded domains considered here.
Action: We clarify the above in our revised Section 4.2.
Response to Authors
The authors have addressed my concerns. I remain slightly positive. Thank you.
Dear Reviewer wfPG,
Thank you very much for your detailed review. We appreciate your thoughtful feedback, which has helped us improve our work.
We are glad that we could address your questions and concerns, and we would appreciate it if you would consider increasing your score.
Thank you again for your time!
Thank you very much for your positive review of our paper! We improved our paper as a result of your comment, and highlighted all changes in blue color in our revised rebuttal PDF.
Responses to Weaknesses:
Thank you for raising this important point. We would like to address your concern from three complementary perspectives, together with two new experimental studies.
(i) The approximation rate is a worst-case guarantee (and it does not show the scaling behavior in practice). The constructive rate O(Q−1/d) is derived using a highly restricted subclass of isotropic quadratic components. It does not exploit the learned anisotropic geometries or expressive nonlinear radial transformations available in the full TRIO architecture. => Consequently, while the bound correctly reflects the difficulty of approximating arbitrary high-dimensional functions, it does not imply that individual high-dimensional problems necessarily require exponentially many components. The relevant practical complexity depends strongly on the geometry of the target function and its preimages.
(ii) New architectural ablations demonstrate this directly. To quantify this distinction, we conducted a new ablation at fixed Q=64. Here, we compare isotropic quadratic, anisotropic quadratic, and the full neural radial TRIO.
The results show that learning anisotropic geometries reduces test RMSE by 75.93% and increases mean ground-truth preimage IoU from 0.717 to 0.898. Adding expressive neural radial transformations further reduces RMSE by 31.38% and improves IoU to 0.939. Overall, the full architecture achieves an 83.49% reduction in RMSE relative to the isotropic quadratic subclass on which the theoretical approximation bound relies. These results demonstrate that the architectural flexibility omitted from the constructive bound yields substantial practical benefits at finite Q.
(iii) New dimensional scaling experiments demonstrate competitive performance through d=64. We additionally introduce a new multimodal high-dimensional benchmark with highly complex, nonlinear sinusoidal interactions, and increase the input dimension from d=2 up to d=64.
With only Q=16 components fixed across all dimensions, TRIO is highly competitive in forward accuracy compared to an unrestricted MLP, and consistently outperforms the ICNN baseline. Moreover, mean ground-truth preimage IoU remains above 0.89 even for d=32.
However, we also explicitly acknowledge the practical limitation found here: with fixed Q=16, preimage IoU decreases to 0.76 at d=64. This shows that increasingly complex high-dimensional preimages require additional representational capacity. Nevertheless, the experiments demonstrate that the worst-case dimensionality dependence is not necessarily representative of practical performance, and that TRIO can remain competitive with unrestricted neural predictors even at much higher dimensions.
Action: We add the new architectural ablation (new Table 3) and the new dimensional scaling experiment (new Figure 7) to our revised Section 5. We further provide full experimental specifications in the Supplement, and explicitly discuss dimensional scaling in our new Limitations section.