- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes OASENT, a neural tensor factorization framework for tensor completion. It applies a Householde orthogonal transformation and a Gumbel-Softmax selector to form a compact task representation. The orthogonal basis and selection are jointly optimized. The experiments verify the performance gains across diverse datasets and metrics.
Critical strengths
The experiments are thoroughly designed with rigorous logic, extensive coverage, and comprehensive evaluation metrics.
Critical weaknesses
- As mentioned in section 3.5, the full-dimensional orthogonal transformation alone can be absorbed into an unconstrained first decoder layer and therefore does not change the represented decoder function class. This makes the Householder layer look like an unnecessary one.
- Are the Gumbel-Softmax logits generated from fixed parameters? If it does not depend on the input sample, the model picks the same coordinates for every observation, which is not truly adaptive in the usual sense.
- Does the theoretical analysis prove that the learned basis makes the selected coordinates more useful than arbitrary coordinates?
- Computational cost is not compared in practice. How about efficiency and computational cost? Suggest conducting experiments to demonstrate the increase in computational complexity as rank increases, such as the real runtime and memory usage of OASENT compare with the baselines.
- How stable is OASENT without dataset-specific tuning? Is Hyperparameter sensitivity high?
Questions for authors
- If the orthogonal layer only works because selection follows it, what is the real architectural benefit of rotating the space instead of just selecting from the original space?
- Are the Gumbel-Softmax logits generated from fixed parameters? Does the auto-selective design learn an input-dependent feature?
- Does the theoretical analysis prove that the learned basis makes the selected coordinates more useful than arbitrary coordinates?
- How about efficiency and computational cost in practice? Suggest conducting experiments to demonstrate the increase in computational complexity as rank increases, such as the real runtime and memory usage of OASENT compare with the baselines.
- Is Hyperparameter sensitivity high?
5 comments
Response 2
Weakness 2 Are the Gumbel-Softmax logits generated from fixed parameters? If it does not depend on the input sample, the model picks the same coordinates for every observation, which is not truly adaptive in the usual sense.
Question 2 Are the Gumbel-Softmax logits generated from fixed parameters? Does the auto-selective design learn an input-dependent feature?
Thanks for the question. The Gumbel–Softmax logits are generated from trainable parameters that are updated during training. It allows the free exploration of different subset of direction selections.
When training complete, it is held fixed at inference. Every observation uses the same deterministic selection matrix at inference. The selected indices are therefore shared, while the selected feature values depend on each observation through .
In OASENT, auto-selective refers to task-supervised global coordinate selection. The prediction loss jointly learns the orthogonal basis and the selector, determining which interaction directions are retained within the -channel budget.
Weakness 3, Question 3 Does the theoretical analysis prove that the learned basis makes the selected coordinates more useful than arbitrary coordinates?
We thank the reviewer to raise this insightful question. Our theoretical analysis only establishes the isometry of the orthogonal transform and characterizes the effective width of selection. Establishing when a learned basis help retaining more task-relevant information than an arbitrary basis remains an open question. A predictive guarantee would require additional assumptions relating the target to the retained subspace. Recent work on sufficient dimension reduction and orthogonal multi-index models provides starting points for such an analysis [1, 2]. We view extending our work along this as a promising future research direction.
Empirically, Section 4.3 shows that learning an orthogonal basis before selection yields lower mean RMSE on all three tested datasets compared with selecting directly from the original tensor-product coordinates. Both variants retain the selector and nonlinear decoder, so this comparison provides evidence for the usefulness of the learned basis.
References
[1] T. Pautrel and F. Portier. “Riemannian Stochastic Optimization for Sufficient Dimension Reduction.” ICML, 2026.
[2] Y. Ren and J. D. Lee. “Learning Orthogonal Multi-Index Models: A Fine-Grained Information Exponent Analysis.” NeurIPS, 2025.
Response 3
Weakness 4 Computational cost is not compared in practice. ...
Question 4 How about efficiency and computational cost in practice? ...
Thank you for the question and suggestion.
Comparison with baselines at optimal ranks. On PCTemp, the default Householder OASENT at achieved 18.56% lower mean RMSE than CoATF at its validation-selected rank, with approximately 4.0× the total training time and 2.0× the test-set evaluation time.
| Model | RMSE | Total Training Time (min) | Per Epoch Training Time (s) | Inference Time (s) | Peak training GPU allocation (MiB) | |
|---|---|---|---|---|---|---|
| CPD | 8 | 2.7761 ± 0.1109 | 1.09 ± 0.01 | 0.589 | 10.463 | 0.46 |
| CoSTCo | 32 | 0.8923 ± 0.0085 | 2.91 ± 0.29 | 0.620 | 10.761 | 21.48 |
| HOCTC | 20 | 0.8823 ± 0.2060 | 3.65 ± 0.06 | 0.942 | 12.778 | 176.91 |
| CoATF | 30 | 0.6508 ± 0.0180 | 2.92 ± 0.63 | 0.686 | 11.070 | 40.54 |
| NEAT | 8 | 1.1246 ± 0.0317 | 3.19 ± 0.06 | 0.637 | 10.954 | 18.12 |
| ConvNTC | 32 | 0.7544 ± 0.0179 | 3.71 ± 0.01 | 0.741 | 11.316 | 53.99 |
| ANTucF | 5 | 0.6861 ± 0.0568 | 2.86 ± 0.38 | 0.631 | 10.821 | 19.33 |
| OASENT (Householder) | 5 | 0.5300 ± 0.0097 | 11.63 ± 0.02 | 2.325 | 21.809 | 42.95 |
Rank scaling within OASENT. We vary the orthogonal parameterization while retaining the selection and decoding design. Householder achieved the lowest mean RMSE at every tested rank. Cayley and Gram–Schmidt offered lower measured runtime and peak GPU allocation at those same ranks. In the Householder implementation, the transform applies reflections to -dimensional interaction vectors before selection. For a batch of size , this stage costs .
| Model | RMSE | Total Training Time (min) | Per Epoch Training Time (s) | Inference Time (s) | Peak training GPU allocation (MiB) | |
|---|---|---|---|---|---|---|
| OASENT (Householder) | 5 | 0.5300 ± 0.0097 | 11.63 ± 0.02 | 2.325 | 21.809 | 42.95 |
| OASENT (Cayley) | 5 | 0.6059 ± 0.0248 | 3.43 ± 0.01 | 0.684 | 11.623 | 20.55 |
| OASENT (Gram-Schmidt) | 5 | 0.6623 ± 0.0501 | 3.49 ± 0.05 | 0.698 | 11.893 | 22.34 |
| OASENT (Householder) | 6 | 0.4941 ± 0.0072 | 17.97 ± 0.14 | 3.594 | 29.599 | 139.38 |
| OASENT (Cayley) | 6 | 0.5460 ± 0.0162 | 3.49 ± 0.06 | 0.698 | 11.511 | 24.32 |
| OASENT (Gram-Schmidt) | 6 | 0.5577 ± 0.0267 | 4.16 ± 0.02 | 0.830 | 14.151 | 24.14 |
| OASENT (Householder) | 8 | 0.4450 ± 0.0011 | 41.73 ± 0.01 | 8.344 | 64.049 | 1571.25 |
| OASENT (Cayley) | 8 | 0.5180 ± 0.0056 | 4.17 ± 0.36 | 0.834 | 15.282 | 38.00 |
| OASENT (Gram-Schmidt) | 8 | 0.4755 ± 0.0191 | 4.72 ± 0.03 | 0.943 | 16.332 | 37.00 |
Above results also suggest that choose a different orthogonal parameterization in OASENT can help reducing the computational cost while maintain its accuracy in a certain level. How to improve the computation efficiency with parallel computing is also an open problem worth of investigation in future.
We will add these results to Appendix E.2 and explain the accuracy–cost trade-off in the discussion.
Response 1
We thank reviewer for the thoughtful review, constructive feedback, and recognition of our work’s strengths.
Weakness 1
As mentioned in section 3.5, the full-dimensional orthogonal transformation alone can be absorbed into an unconstrained first decoder layer and therefore does not change the represented decoder function class. This makes the Householder layer look like an unnecessary one.Question 1
If the orthogonal layer only works because selection follows it, what is the real architectural benefit of rotating the space instead of just selecting from the original space?Thank you for the question. The orthogonal transformation lets each selected channel retain a learned linear combination of the original interaction coordinates. With direct selection, z=xP⊤, each channel retains a single original coordinate. With the learned basis, z=xQP⊤, each channel retains the projection of x onto a learned basis direction, which can combine multiple original coordinates. From the view of information flow, jointly learning Q and the selector thus lets the objective guide the transformation to allow more essential task-relevant information passed through the selection bottleneck, yielding performance gain.
The comparison between the two model variants ν2 and ν1 in Section 4.3 verifies the benefit directly. Both variants retain the selector and nonlinear decoder under the same channel budget. The former selects from the original coordinates, while the latter selects with the learned orthogonal basis. Learning the basis help reducing mean RMSE by 6.8% on NYCTaxi, 31.3% on PCTemp, and 2.7% on QoS-RT. These results show that selection after space rotation improved prediction relative to selecting directly from the original coordinates in the tested settings.
We will improve the presentation in Sections 3.5 and 4.3 to clarify the architectural benefit.