Gradient-Based Learning of Minimal Predictive Realizations from Finite Hankel Blocks

We study gradient-based learning of linear recurrent models in an overparameterized hidden space from finite Hankel matrices, whose entries are scores of concatenated prefixes and suffixes. We assume that the target has finite Hankel rank and observe a base matrix attaining this rank together with its one-symbol shifts. Exact fitting of the base and one-symbol Hankel blocks through a decoder does not guarantee correct multistep predictions with unconstrained transitions. We construct a training procedure that updates state, decoder, and transition parameters concurrently, with each loss restricted to its corresponding parameter group. We prove global convergence from any full-rank state initialization and arbitrary finite initializations of the remaining parameters. The limiting state matrix has the target Hankel rank, and removing redundant dimensions yields a model with the minimum possible state dimension that predicts every finite sequence exactly. For every fixed positive feedback strength, we exhibit a normally attracting manifold of non-global local minima with an open basin containing full-rank initializations. A computable switching certificate then combines global triangular training with convergent fully coupled updates. We also prove deterministic fixed-step convergence under sufficient step-size conditions and derive fixed-length prediction-error bounds for least-squares reconstructions from perturbed Hankel matrices. Experiments on finite-rank weighted sequence systems recover the target rank in all 90 runs and illustrate failures of one-step fitting, local convergence under coupled gradients, and prediction stability under Hankel perturbations.