Orthogonal Auto-Selective Encoding Tucker Factorization

Neural Tucker factorization provides a promising framework for learning representations of high-order tensor data by modeling multiplicative interactions among mode-specific embeddings. The outer-product construction from mode-specific embeddings defines a latent interaction space whose coordinates are not explicitly organized according to their relevance to downstream tasks. A central challenge is to adapt this space so that task-relevant interaction directions can be identified and retained. We propose Orthogonal Auto-Selective Encoding Tucker Factorization (OASENT), which combines an orthogonal coordinate transformation with a task-driven selection module. A Householder-parameterized layer learns a norm-preserving coordinate transformation. A Gumbel-Softmax selector then forms a compact $K$-channel task representation using continuous masks during training and deterministic coordinate selections at inference. The nonlinear decoder then maps this compact representation to the original interaction dimension for prediction. Our analysis shows that a full-dimensional orthogonal transformation alone leaves the represented decoder function class unchanged. Subsequent selection forms a bottleneck with effective width $J_{\mathrm{eff}}\le K$ at inference. Across seven real-world tensor datasets, OASENT significantly outperforms all evaluated baselines. All findings indicate that adapting the interaction representation space before selective encoding and decoding can improve higher-order tensor representation learning.