Nethobench: Forecasting design and conditioning shape the structural realism of generated neural dynamics

Forecasting models are increasingly used to model neural population activity, yet existing benchmarks typically evaluate them by pointwise predictive fidelity to held-out observations. In stochastic, partially observed systems, agreement with one sampled realization is distinct from preserving the statistical and dynamical structure of the process. We introduce Nethobench, a benchmark that evaluates structural agreement across five metric families: distributional, temporal spectral, relational, geometric, and state-dynamical. We validate Nethobench using targeted perturbations and stochastic systems with known dynamics, and apply it to mouse calcium imaging, mouse Neuropixels recordings, and human intracranial recordings, using empirical references to calibrate interpretation. Nethobench detects targeted structural failures and assigns high scores to independent samples from the correct synthetic process despite trajectory mismatch. It tracks training progress and structural changes under domain shift. Model comparisons reveal how forecasting formulation, architecture, training regime, and loss shape structural profiles, while input ablations demonstrate structural improvements from stimulus and behavioral conditioning. These results motivate reporting calibrated family-level structural metrics alongside pointwise accuracy, proper probabilistic scores, and task-specific endpoints when evaluating neural forecasts across species and recording modalities.