acceptodds
Under review as a conference paper at ICLR 2027

A Multilayer Latent Factorization of Nonstandard Tensors Model for Accurately Representing Higher-Order Graphs

Abstract

Higher-order graphs characterize complex interaction systems involving multiple types of entities and relations, and can be naturally modeled as nonstandard tensors with extreme sparsity and high dimensionality. Tensor decomposition models have been widely used to learn low-dimensional representations of such tensors for higher-order graph inference. Existing tensor decomposition methods typically employ single-layer architectures, limiting their ability to learn expressive representations. To address this limitation, we propose a Multilayer Latent Factorization of Nonstandard tensors (MLFN) model, which enables progressive nonlinear representation learning through multiple latent factorization layers. Specifically, MLFN incorporates three key innovations: 1) a multilayer learning framework that progressively learns low-dimensional representations through layer-by-layer feature learning, 2) a nonlinear activation-based sampling strategy that generates informative training samples to alleviate the extreme sparsity of nonstandard tensor data, and 3) an efficient parallel training scheme using an alternating stochastic gradient descent for scalable model optimization. Experimental results on four types of higher-order graphs demonstrate that MLFN improves the average prediction accuracy by 22.81% over state-of-the-art methods for unobserved-link prediction.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Higher-order graphs, nonstandard tensors, multilayer learning, nonlinear representation learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Y8SrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MLFN, a multilayer nonstandard tensor latent factor model for representation learning and missing relation-value prediction in extremely sparse multirelational graphs. The method first constructs a third-order node–node–relation tensor using meta-paths and trains a CP latent factor model independently at each layer. It then predicts a subset of unobserved entries, transforms the predictions into pseudo-observations through a nonlinear mapping, and filters samples based on consistency between current predictions and existing pseudo-labels. The retained pseudo-observations augment the training constraints for the next layer. Experiments on several public datasets report competitive predictive performance, and the paper also presents a parallel optimization strategy with empirical speedups.

Critical strengths
  1. A coherent training procedure tailored to extreme sparsity. The method retains the original observations while controlling the influence of pseudo-observations through weighting and consistency-based selection. Combining observed-entry reconstruction, pseudo-observation generation, and successive training stages provides a practical approach to strengthening the training signal in sparse tensor completion.

  2. Broad empirical evaluation with attention to computational efficiency. The experiments cover four datasets, three data-split settings, and seven baselines, with competitive results and evidence supporting the sample-selection mechanism. The paper also exploits the independence of updates across data blocks when two factor groups are fixed to enable parallel optimization, and reports empirical speedups. Together, these results support the method's practical promise.

Critical weaknesses
  1. The source of the gains and the necessity of the multilayer procedure remain insufficiently established. The current results more directly support improved parameter estimation through progressive self-training than an expansion of the final prediction function class. Moreover, the layer-count and sample-selection experiments do not fully separate the effects of repeated refinement, the number of pseudo-observations, and cumulative optimization effort. A single-stage CP baseline with pseudo-observation augmentation, evaluated at the same rank and a comparable total optimization budget, would provide a more informative comparison. Reporting the retained and newly added sample counts at each layer would further clarify what drives the improvements. This distinction matters for assessing the method's specific contribution beyond additional data augmentation and optimization.
Questions for authors
  1. What evidence shows that repeated pseudo-observation generation and filtering provide benefits beyond a single-stage augmented CP model at the same rank and a comparable total optimization budget?

  2. How are the tensor-entry values constructed from meta-path-induced relations?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Y8Sr

We thank the reviewer for the constructive comments and for recognizing the practical promise of MLFN. We have added controlled experiments and clarified the tensor-construction procedure in the revised manuscript.

Q1: Does repeated pseudo-observation generation and filtering provide benefits beyond a single-stage augmented CP model?

To isolate the contribution of progressive multilayer refinement, we introduced Single-EFC (Equal Final Count), a single-stage augmented CP baseline. It uses the same CP rank, nonlinear mapping, confidence screening rule, final retained pseudo-observation count, and total entry-update budget as MLFN. The only substantive difference is that Single-EFC generates and screens pseudo-observations once, whereas MLFN progressively regenerates, screens, and refines them across layers.

SettingMLFN winsRMSEMAEMAPE
Relative error increase of Single-EFC over MLFN
All cases30/36+8.65%+11.55%+23.64%
3:1:6 split12/12+12.36%+21.82%+30.34%

Here, each positive value denotes the relative error increase of Single-EFC over MLFN. Across all dataset–split–metric cases, MLFN wins 30 of 36 comparisons, with an average error reduction of 14.61%14.61\%. Under the most sparse 3:1:6 setting, MLFN wins all 12 comparisons. Single-EFC remains competitive in a few higher-training-ratio cases, which is consistent with our motivation: when sufficient observations are available, one-shot parameter estimation becomes more reliable, whereas progressive generation and repeated screening are particularly beneficial under extreme sparsity. Since the pseudo-observation count, rank, and optimization budget are controlled, these gains cannot be attributed merely to more generated data or additional updates. The complete protocol and results are provided in Section 4.6 and Appendix C.4 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Y8Sr

Q2: Meta-path-based tensor values.

For each relation, we multiply the adjacency/incidence matrices along its meta-path to obtain the path-count matrix C(r)C^{(r)}. After removing self-connections, positive counts are standardized relation-wise and mapped to (0,1)(0,1):

yijr=Φ((cij(r)−μr)/σr).y_{ijr}=\Phi\left((c_{ij}^{(r)}-\mu_r)/\sigma_r\right).

The resulting matrices are stacked as a node–node–relation tensor; zero path counts remain unobserved coordinates rather than observed zeros. The complete construction and examples are now provided in Appendix C.1.

We thank the reviewer again for the helpful feedback. We believe these additions clarify both the source of MLFN's gains and the construction of its input tensors.

Reviewer_dyvzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MLFN, a multilayer latent factorization framework for link prediction on extremely sparse, high-dimensional "Node-Node-Relation" nonstandard tensors. Each layer performs CP-based latent factorization, generates synthetic entries for unobserved positions via a sampling strategy, filters them with a confidence score and dynamic threshold, and passes the augmented tensor to the next layer. An ASGD-based training scheme with a convergence proof and experiments on four datasets support the claims. The idea of progressively densifying a sparse tensor across layers is reasonable and the empirical results are promising, but several methodological and reporting issues need to be resolved.

Critical strengths
  1. The multilayer pipeline is technically coherent and clearly presented.

  2. The sampling strategy is non-trivial and reasonably designed.

Critical weaknesses
  1. Risk of error accumulation across layers. Synthetic entries are fed back as observations, and errors introduced at layer nn propagate to all later layers.

  2. Incomplete ablation. MLFN-1 removes screening and MLFN-2 removes both sampling and screening, so the individual contribution of the activation function and of the confidence threshold δ\delta is not isolated.

Questions for authors

See Weaknesses

2 comments
AuthorsOpenReview ↗

Author Response to Reviewer dyvz

We thank the reviewer for recognizing the technical coherence of the multilayer pipeline and the non-trivial design of the sampling strategy. We address the two concerns through additional controlled experiments, whose complete results are reported in Sections 4.6–4.8 and Appendix C.4 of the revised manuscript.

Q1: Risk of error accumulation across layers.

We agree that feeding unreliable pseudo-observations into subsequent layers could propagate errors. MLFN mitigates this risk through several complementary safeguards: the original observations are retained at every layer; pseudo-observations are separately weighted by α\alpha; the nonlinear mapping restricts generated values to the valid range; confidence-based screening removes inconsistent predictions; and the number of newly generated entries is bounded at each layer.

We added controls that isolate these safeguards. RandEq retains exactly the same number of pseudo-observations as MLFN but selects them randomly from the same candidate pool. MLFN achieves lower errors in 32 of 36 comparisons, while RandEq increases the average relative error by 7.93%. Replacing the adaptive threshold with Fix.99 increases the average error by 19.51%, with MLFN winning 34 of 36 comparisons. Increasing the generation target by 50% also worsens 30 of 36 comparisons, indicating that indiscriminately retaining more pseudo-observations does not improve accuracy.

These results do not imply that every generated entry is correct; rather, they show that confidence-guided selection, adaptive thresholding, and controlled generation generally mitigate the risk of propagating unreliable entries.

AuthorsOpenReview ↗

Author Response to Reviewer dyvz

Q2: Individually controlled ablations.

We agree that the original MLFN-1 and MLFN-2 variants jointly removed multiple components. We therefore added controls in which only one design choice is changed while the remaining pipeline and hyperparameters are preserved.

Summary of the added controlled experiments. W/L denotes MLFN/control wins; Avg. increase is the control's average relative error increase over MLFN.

ControlIsolated componentW/LAvg. increase
Single-EFCProgressive vs. one-shot augmentation30/614.61%
IdentityNonlinear activation36/0118.29%
Hard-ClipSmooth mapping vs. range clipping36/060.03%
Generation 0.5×0.5\timesGenerated-sample quantity35/120.16%
Generation 1.5×1.5\timesGenerated-sample quantity30/68.55%
RandEqConfidence-based selection at equal count32/47.93%
Fix.99Adaptive confidence threshold34/219.51%

Identity removes the nonlinear mapping, whereas Hard-Clip replaces it with direct clipping; both preserve multilayer generation and screening. RandEq isolates the confidence score by matching the retained count, and Fix.99 isolates the adaptive threshold by replacing it with a fixed cutoff. Generation 0.5×0.5\times and 1.5×1.5\times vary only the generation target. The consistent overall degradation of these controls supports the individual contribution of the corresponding components. The specific experimental result is provided in the Appendix C.4 of the revised paper.

We thank the reviewer again for the constructive comments. The additional controls allow the effects of nonlinear activation, confidence-guided selection, adaptive thresholding, generation quantity, and progressive refinement to be evaluated separately.

Reviewer_PJFzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the challenge of learning useful representations from extremely sparse higher-order graph data modeled as tensors. It proposes a multilayer tensor factorization framework that progressively uses predictions from earlier layers to enrich the data available to later layers, while filtering unreliable generated entries. The method is designed to improve prediction accuracy while retaining computational efficiency through parallel optimization.

Critical strengths
  1. The progressive self-augmentation mechanism is intuitive for sparse tensors, using earlier predictions to enrich later layers while screening helps limit error propagation.

  2. The evaluation is systematic within the selected experimental setup, using multiple data splits, three metrics, repeated runs, and statistical significance tests to assess performance consistency.

  3. The paper explicitly addresses efficiency through complexity analysis, parallel ASGD, and runtime scaling with the number of threads.

Critical weaknesses
  1. Limited methodological novelty. Many ingredients of MLFN have close precedents in recent tensor-factorization work, including multilayer nonlinear LFT architectures (e.g., TCCN [1]), nonlinear activation in NLFT, tensor completion/refinement, alternating optimization, and convergence analysis. The main novelty appears to be generating pseudo-observations and iteratively refining/reusing them across LFT layers, making the contribution substantially narrower than the paper's framing.

  2. Missing comparisons with the closest recent methods. The experiments mainly compare against methods from 2019–2024, while several substantially closer recent NLFT approaches are omitted, including TCCN [1], FRNL [5], BNLNR [2], CL-NLFT [4], and CB-NLFT [3]. These methods already explore tensor-factorization mechanisms making them particularly important baselines for establishing MLFN's incremental contribution. Their absence makes it difficult to determine whether MLFN improves upon the current state of nonlinear/multilayer LFT methods rather than primarily upon older single-layer approaches.

  3. The ablation study is too limited. The two variants primarily isolate sampling and screening, but do not separately evaluate several central design choices, including the multilayer construction, confidence formulation, generated-sample rate, or optimization scheme. Consequently, it remains unclear which components actually drive the reported improvement.

  4. Limited experimental diversity in relation structure. All four datasets contain only 2–5 relation types, despite the paper being motivated as a general method for higher-order graphs. The experiments therefore mainly demonstrate scalability under node sparsity and do not establish robustness to graphs with substantially richer or more heterogeneous relation spaces.

  5. Serious reproducibility limitations. No source-code repository is provided, and the final values of important parameters used for the reported results are not given. This is particularly concerning because the sensitivity analysis shows large performance variation across these settings.

  6. The theoretical guarantee does not fully cover the novel part of the method. The convergence analysis assumes smooth/strongly convex layer-wise subproblems and sets α=1\alpha=1, despite the overall model being nonconvex with tuned α\alpha. It also does not establish convergence of the cross-layer pseudo-observation and screening process.

[1] Liao, Xin, Hao Wu, and Xin Luo. “A Novel Tensor Causal Convolution Network Model for Highly-Accurate Representation to Spatio-Temporal Data.” IEEE Transactions on Automation Science and Engineering, vol. 22, 2025.

[2] Wang, Lan, et al. “Nonnegative Latent Factorization of Tensors with Swish Regularization for Water Quality Data Imputation.” Proceedings of the 12th International Conference on Communications and Broadband Networking (ICCBN), 2024.

[3] Wang, Qu, Hao Wu, and Xin Luo. “A Convolution Bias-Incorporated Nonnegative Latent Factorization of Tensors Model for Accurate Representation Learning to Dynamic Directed Graphs.” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 55, no. 12, 2025.

[4] Wu, Hao, and Jiajia Mi. “A Cauchy Loss-Incorporated Nonnegative Latent Factorization of Tensors Model for Spatiotemporal Traffic Data Recovery.” Neurocomputing, vol. 626, 2025, article 129575.

[5] Wu, Hao, Yan Qiao, and Xin Luo. “A Fine-Grained Regularization Scheme for Non-negative Latent Factorization of High-Dimensional and Incomplete Tensors.” IEEE Transactions on Services Computing, vol. 17, no. 6, 2024.

Questions for authors
  1. How is MLFN fundamentally different from prior multilayer nonlinear LFT architectures such as TCCN? Is the intended novelty specifically the iterative use of screened predicted entries as observations for subsequent factorization layers?

  2. Can the authors provide controlled ablations for the multilayer architecture, nonlinear activation, confidence score, thresholding strategy, and generated-sample quantity, rather than only removing sampling and screening jointly/sequentially?

  3. Can the authors compare MLFN against the recent closely related methods (TCCN, FRNL, BNLNR, CL-NLFT, and CB-NLFT) to establish that the proposed multilayer pseudo-observation mechanism provides gains beyond existing nonlinear LFT approaches?

  4. Will the authors release the implementation and provide the complete final hyperparameter configuration for any dataset-specific settings and the seeds to required to reproduce the main results?

5 comments
AuthorsOpenReview ↗

Author Response to Reviewer PJFz

We thank the reviewer for the detailed and constructive assessment. We have revised the paper to clarify the methodological contribution, add a recent nonlinear LFT baseline, provide controlled ablations, and correct the scope of the convergence analysis. All changes are highlighted in blue in the revised PDF.

{Q1: Difference from prior multilayer nonlinear LFT models.} MLFN is specifically designed for nonstandard tensors with extreme sparsity and high dimensionality, where the limited number of observed entries makes it difficult for conventional one-stage factorization models to learn reliable latent representations. Its key distinction is a progressive multilayer refinement framework: each stage enriches the effective observations available to the next stage, while nonlinear mapping and confidence-based screening control the reliability of the incorporated information. Therefore, the successive stages progressively stabilize latent representation learning under extreme sparsity, rather than merely stacking nonlinear factorization layers.This differs from TCCN, which employs causal convolution to learn representations from spatio-temporal tensors. In contrast, MLFN focuses on progressively alleviating the scarcity of effective observations in extremely sparse nonstandard tensors through cross-stage refinement and reliability control.

To verify that the improvement is not simply caused by using additional observations or a larger optimization budget, we introduced Single-EFC. It uses the same rank, nonlinear mapping, confidence screening, final retained-observation count, and total entry-update budget as MLFN, but performs augmentation only once. MLFN achieves lower errors in 30 of the 36 comparisons and in all 12 comparisons under the sparsest 3:1:6 setting. Under this setting, Single-EFC increases RMSE, MAE, and MAPE by 12.36%, 21.82%, and 30.34%, respectively. These results support the value of progressive multilayer refinement for reliable factor estimation under extreme sparsity. Complete results are provided in Section 4.6 and Appendix C.4.1.

AuthorsOpenReview ↗

Author Response to Reviewer PJFz

Q2: Controlled ablations.

We added controls for the progressive schedule, nonlinear activation, confidence-based selection, adaptive thresholding, and generated-sample quantity. The table reports MLFN/control wins and the control's average relative error increase over MLFN across the 36 metric-case comparisons.

ControlIsolated design choiceW/LAvg. increase
Single-EFCProgressive vs. one-shot augmentation30/614.61%
IdentityNonlinear value mapping36/0118.29%
Hard-ClipSmooth activation vs. clipping36/060.03%
Generation 0.5×Generated-sample quantity35/120.16%
Generation 1.5×Generated-sample quantity30/68.55%
RandEqConfidence selection at equal retained count32/47.93%
Fix.99Adaptive vs. fixed confidence threshold34/219.51%

These results separately support the progressive schedule, nonlinear activation, confidence score, adaptive thresholding, and default generation quantity. The complete results are provided in Sections 4.6–4.8 and Appendix C.4. ASGD parallelization optimizes the same objective and is evaluated as an efficiency component through the thread-scaling and runtime experiments in Section 4.5 and Appendix C.6. Comparisons with alternative optimizers will be considered in future work.

AuthorsOpenReview ↗

Author Response to Reviewer PJFz

Q3: Recent nonlinear LFT baselines.

We agree that the original comparison set did not sufficiently cover recent nonlinear LFT methods. We therefore added BNLNR (M9,2024) and CL-NLFT (M10,2025), two recent approaches identified by the reviewer, and evaluated them on four datasets under three training-validation-testing ratios with ten repeated runs. MLFN achieves lower errors in 32 of the 36 metric-case comparisons against each model. In the expanded ten-model comparison, MLFN, BNLNR, and CL-NLFT obtain Friedman ranks of 1.22, 5.92, and 5.22, respectively. The one-sided Wilcoxon tests yield p-values of 0.00073, 0.01050, and 0.00024 against BNLNR, and 0.00171, 0.02612, and 0.00171 against CL-NLFT for RMSE, MAE, and MAPE, respectively. All comparisons remain significant after Holm correction.

MLFN versus BNLNR and CL-NLFT over the 12 dataset-split cases.

MetricMLFN/BNLNR W/LBNLNR Wilcoxon ppMLFN/CL-NLFT W/LCL-NLFT Wilcoxon pp
RMSE11/10.0007311/10.00171
MAE9/30.0105010/20.02612
MAPE12/00.0002411/10.00171
Overall32/4--32/4--

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.