acceptodds
Under review as a conference paper at ICLR 2027

Jointly Corrected Parallel Responses in Forecasting Models for Nonlinear Dynamical Systems

Abstract

How should added nonlinear computation be organized for forecasting nonlinear dynamical systems? We introduce the Jointly Corrected Parallel Forecaster (JCPF), which combines explicit aggregation of learned parallel increments with response-conditioned vector correction. The Compact Shared Forecaster (CSF) provides a common encoder/head reference and remains competitive on several generic forecasting tasks. With known future driving inputs, JCPF substantially improves on CSF across three measured-system benchmarks. On Silverbox and Wiener–Hammerstein, it reduces mean MSE relative to near-budget additive branches by 18.99% and 32.54%, with every seed agreeing, and improves on a wider residual MLP. Retrained ablations support retaining both direct aggregation and response-conditioned correction. Branch-conditioned and aggregate-conditioned corrections attain comparable final accuracy but different add-only-to-full error reductions: a larger correction gain can reflect greater reliance on that path rather than a better predictor. Complete increments can also be composed as parallel macro modules, with denoting their number. From to , WH improves under both fixed total budgets and fixed module widths, while F16 retains comparable accuracy. These findings support joint correction of parallel responses for nonlinear-system forecasting and distinguish final predictive value from the division of work learned by internal paths.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Nonlinear Dynamical Systems, Modular Neural Networks, MLP

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dZXXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

JCPF studies the organization of forecasting capacity through parallel predictors and a response-conditioned joint correction. Its computation has four branches, a normalized direct aggregate, a correction conditioned on branch responses, and a z bypass. The paper compares alternative capacity allocations and correction routes. These include separately retrained ablations and fixed-parameter/fixed-width macro comparisons. The strongest reported gains occur on particular nonlinear physical systems. The manuscript also discusses near-ties on generic tasks and the additional computational cost.

Critical strengths
  1. The reported comparisons support response-conditioned correction. The supplied reassessment reports mean-MSE reductions over PNEAR of 18.9938% on Silverbox and 32.5389% on WH (Table 2). The prior arithmetic check supports the consistency of those numerical records, but does not independently replicate training.

  2. Alternative capacity allocations, independently retrained route controls, aggregate-conditioned correction, and fixed-parameter/fixed-width macro comparisons address the structural question more directly than a larger-model comparison alone. The paper distinguishes fitted-path removal from retraining. It acknowledges the small generic gains, comparable F16 accuracy, and increased latency. These limitations define the contribution's scope while leaving the targeted structural comparisons valuable.

Critical weaknesses

The main concern is the independence of the strongest physical-system evidence. Appendix A.3 acknowledges historical development exposure to the physical test records. Current validation-only checkpoint selection leaves open whether earlier test observations influenced the architecture, hyperparameters, comparator selection, or analysis. The supplied reassessment did not identify a specific chronology or freeze record that answers this question. Historical exposure does not prove test-informed selection or future-output leakage. The concern remains until the relevant chronology and evaluation boundary can be checked. Known future driving inputs are explicitly part of the stated response-prediction task.

Questions for authors
  1. For each physical system, what test information was viewed, and when? Which architecture, hyperparameter, comparator, or analysis choices changed afterward? Please identify the freeze boundary for the central structural comparisons and the existing records that support it.

  2. Which evaluation relevant to the central comparisons remained untouched during development? Existing frozen records and a clear chronology may resolve the concern. If test records informed the central choices and no untouched evaluation remains, can the frozen comparison be evaluated on an unused record or system? Additional seeds on the same previously viewed test segment would not restore independence.

  3. As a secondary clarification, please provide the main PatchTST adapter/version and test-origin manifest. How do the 117/477 counts in Appendix E.4 relate to the 243/981 populations in Appendix A.1/Table 5? These records would help determine whether the external comparison uses the stated protocol.

Additional presentation feedback The supplementary documentation identifies Figure 2 as a reviewed GPT Image asset. The CSF-only route T(q)=z and the full modular transition both lead to the same output head. Their alternative status is not marked. The z input arrow into LN(z) is also disconnected from the base-activation z node. These drawing ambiguities do not show that the implementation adds z twice. Please distinguish the alternative routes and connect or explicitly label the repeated z node. Provide an editable SVG produced with AI assistance and then manually checked, or redraw the figure correctly with another suitable tool. Verify every label, formula, arrow, and connection before export.

Reviewer_94utOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

JCPF runs a few small MLPs in parallel, adds them up, and then corrects the sum with a network that sees every branch output. On Silverbox and WH it outperforms the baseline at the same size.

Critical strengths

Well designed experiments with ablation study. The parameter sizes are comparable and (re)training are from scratch.

Critical weaknesses

Only two datasets show improvements. PatchTST baseline is lower than usual.

Questions for authors
  1. Did authors see the test sets during development?
  2. How is the proposed method compared to a deeper ML at same size?
  3. Is the design choices made blind to the test results?
Reviewer_1zbYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores an interesting concept in the context of sequence to sequence modeling (forecasting) for dynamical systems. The idea centers around the hypothesis that modeling linear systems is "cheap", and modeling more complex, linear systems amounts to adding (not necessarily literally "adding") nonlinear correction mechanisms. One of the core contributions is a nonlinear corrector architecture that includes a set of parallel, independently learned branches that can be aggregated. The paper provides a justification for this approach to nonlinear dynamical systems via Taylor series expansion, and then provides several alternatives in order to (a) ablate the original idea and (b) enable resource=equivalent comparisons.

Critical strengths
  1. Surrogate models for dynamical systems, reduced order models, and sequence forecasting are all interesting and relevant problems for the ML community.
  2. Despite some confusion (see questions below), the paper is well organized and largely well written.
Critical weaknesses
  1. It's difficult to be convinced that JCPF is a worthwhile choice. First of all, nearly every column of Tables 1, 2, etc have almost identifical performance within uncertainty bounds. Second, R2048 is very simple and seems to do the job just fine. Assuming that resource consumption is similar, at least R2048 is simpler in terms of implementation, hyperparameter / design choices, etc.
  2. The setting is interesting, but from a philosophical perspective why not go in a completely different direction? Why not choose architectures and training methods that model these phenomena "better"? There are of course many sequence:sequence modelign approaches that achieve better performance than the variants presented in this paper. This could be fine, but these setting seems somewhat contrived.
  3. Several design choices and experimental studies seem relatively arbitrary and lack sufficient justification.
Questions for authors
  1. What is an example of a (real) system where future inputs are known a priori? I would assume that for many systems, such inputs also have to be forecasted (and are therefore estimates, with errors), or are a function of the system state itself (e.g. feedback control).

  2. Equations (1) and (2) are confusing at first for someone uninitiated to this field (or for a general ML reader). This seems like a very particular (arbitarry?) architectural choice among many, perhaps infinite options. Why these choices? I believe it is to provide a setting that allows for "apples to apples" comparisons in the rest of the study, to the extent possible. (a) is that the case? (b) why this particular choice? For example, qcq_c reads like an affine mapping of scaled inputs (plus a bias). Or is this an operation on the latent space produced by the encoder? It is unclear what xcx_c is here, is it a vector of input histories? Assume an encoder is, say, an MLP or any variant from deep learning, a notion of an affine mapping of inputs is strange and probably not sufficiently general. Ah, I see in Figure 1 that this is a linear encoder. I am leaving the above question to hopefully help with paper editing. I should also note that earlier parts of the paper discuss modeling linear systems and then correcting for the nonlinear residual; this now (sort of) makes sense to me but I am still a bit confused about how this architecture achieve that objective. Linear layers, stacked with nonlinearity later, with a linear prediction head is...just a nonlinear model. It is still (again at this point in the paper) unclear if this actually achieves something notionally akin to "predict the linear part and then correct for nonlinearity later"; it is just one big model that is trained to predict system outputs, no?

  3. Why these particular alternatives in Section 3.4? I believe it is because they require compute that is similar to the proposed architecture. Please address this, but the second part of this quesiton is why these two? Again, there is (at least theoretically) an infinite number of choices here.

  4. The paper largely does well to describe the various mechanims and the overall architecture. However, both the Linear residual head and Output restoration blocks are unclear. While this may seem like an odd point and these are not really part of the "contributions", a core characteristic of deep learning is that all kinds of interesting things arise from the composition (many, many compositions) of otherwise relatively simple things. As noted elsewhere, I have an issue with the paper's assertion that things are indeed de=coupled. Further, if one of the central aspects of the paper is an evaluation of CSF vs/with/and JCPF, it is unclear how these two things are "merged" in the residual head. Perhaps I have missed this. Either way, this seems critical as these are two of the sub-architectures that are being compared or contrasted.

  5. Why B=4B=4? These parallel branches do not seem to be too heavy, and scale linearly with B anyway. Why stop with 4?

Other minor issues:

  • It is unclear what Figure 2 refers to. I believe it is Section 3.5 (it is also not explicit what Figure 1 is associated with, but that is more obvious).
  • Several places use the word "structure" or its variants, e.g. "structural evidence" or "structural study". I have no idea what this means. I could see this as an architectural study, or perhaps these words mean something else entirely.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.