acceptodds
Under review as a conference paper at ICLR 2027

Complementing Models via Residual Learning

Abstract

Physics-based models of dynamical systems deviate from observed behavior due to simplifying assumptions, abstraction, epistemic uncertainty, unmodeled variability, couplings with other systems, and environmental influences. We introduce a dynamical state reconstruction architecture that deploys an existing physics-based model as a predictive backbone, enhanced with data-driven residual corrections learned by a self-supervised transformer. The learning protocol relies on epistemic tokenization, representing each observation as a tuple of value, observation coordinate, and epistemic state. The epistemic state distinguishes available evidence from reconstruction queries, providing an explicit learning and inference signal. The learned correction is explicitly bounded, preventing arbitrarily large deviations from the physics-based model predictions. We instantiate the approach using transformer-based state reconstruction of a permanent magnet synchronous machine from multimodal signals, including quantities observed during training but unavailable at inference time. The resulting architecture integrates transparent, interpretable physics-based models with evidence-driven deep learning to close the gap between the reference model's predictions and observed system behavior.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
hybrid modeling, residual learning, self-supervised learning, state reconstruction, epistemic tokenization, physics-informed machine learning, transformers, dynamical systems

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_pVnsOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes WRAPPER, which retains an existing physical model as a predictive backbone while employing a Transformer to learn corrections based on historical residuals and input signals. Its core technical design features explicit epistemic tokenization—distinguishing between known evidence and the targets to be reconstructed—as well as bounded residual correction.

Critical strengths

None

Critical weaknesses
  1. Limited novelty. The proposed epistemic tokenization largely resembles existing masked time-series reconstruction methods, while the combination of Transformer-based residual prediction and a mechanistic predictor appears incremental. The paper does not sufficiently establish the methodological novelty over existing approaches.

  2. Insufficient baseline comparisons. The evaluation lacks comprehensive comparisons with standard MLP-, GRU-, and Transformer-based reconstruction methods. It is therefore unclear whether the proposed architecture offers meaningful advantages over simpler alternatives.

  3. Missing ablation studies. The individual contributions of epistemic tokenization, self-supervised query construction, and bounded residual correction are not adequately isolated. The effectiveness of the key design choices remains unverified.

  4. Mismatch between the claimed task and reported results. Although reconstructing unavailable torque measurements is a central motivation, the main results primarily report electrical-current residual metrics. Quantitative evidence demonstrating accurate torque reconstruction is insufficient.

  5. Limited experimental scope. The method is evaluated on a single permanent-magnet synchronous motor using 21 healthy recordings. Its generalizability to different machines, operating conditions, and system dynamics remains unclear.

  6. Insufficient validation of fault-diagnosis capabilities. The experiments focus on improving residual behavior under healthy operation, without evaluating fault detection, fault isolation, or false-alarm rates under actual faulty conditions. Consequently, the practical diagnostic benefits remain unsubstantiated.

  7. Unclear justification for bounded residual correction. Although the correction is explicitly bounded, the paper does not sufficiently establish how the bound is selected or demonstrate its benefits over unbounded correction or alternative constraint mechanisms.

  8. Limited reproducibility. The experimental dataset is not publicly redistributed, and access depends on the original data providers. This limitation makes independent verification of the reported results more difficult.

  9. Overly elaborate formulation relative to the technical contribution. The introduction of numerous formal definitions and named components complicates the presentation without clearly demonstrating corresponding algorithmic benefits. A more direct formulation and stronger empirical justification would improve the paper's clarity and credibility.

Questions for authors

None

Reviewer_wW7uOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes WRAPPER, a framework for dynamical state reconstruction that combines a prior predictor with data-driven residual corrections. It uses explicit epistemic tokenization to distinguish available observations from reconstruction queries and trains a self-supervised Transformer to learn bounded corrections while retaining the original physical model. Experiments on a permanent magnet synchronous machine show reduced current reconstruction errors and residual bias under healthy operating conditions, while fault detection and isolation remain untested.

Critical strengths
  1. The paper combines physics-driven and data-driven modeling, using learned residual corrections to improve predictions while retaining the original mechanistic model.
  2. The self-supervised reconstruction scheme can help address the practical challenge of reconstructing signals that are available during training but unavailable at deployment.
  3. Experiments on real motor measurements demonstrate that the method can reduce reconstruction error.
Critical weaknesses
  1. Limited experimental evaluation. The experiments are restricted to one motor system and compare WRAPPER only with the original physics-based predictor. Comparisons with existing hybrid or data-driven methods, as well as runtime and memory measurements, are missing.

  2. The presentation could be improved. The layout and organization make the paper difficult to follow, and terminology is not always used consistently. For example, it is unclear whether “evidence-driven deep learning” and “data-driven components” refer to the same learned component or represent a meaningful distinction.

  3. Some methodological details and claims need clarification. The Governance module lacks a sufficiently explicit description of the bounding operator and threshold selection. The statement that a physics-based prior is unnecessary also needs to be distinguished more clearly from the experimentally evaluated configuration.

Questions for authors
  1. The experiments compare WRAPPER only with the prior predictor. Could the authors clarify the baseline setup and include comparisons with two or three additional data-driven or hybrid methods under the same data splits and input conditions? Could the authors also report runtime and memory consumption to assess computational efficiency?

  2. What does epistemic tokenization contribute beyond conventional masking? Could the authors provide an ablation to demonstrate its benefit?

  3. Could the authors explicitly define the bounding operator BB and explain how its bound is selected? Does it require manual adjustment when applying WRAPPER to physical systems other than the IPMSM?

  4. Could the authors clarify the statement that the architecture does not require a physics-based prior? What requirements apply to the backbone, and how does performance change when the physical backbone is replaced by a null or data-driven predictor?

Reviewer_tWVTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper uses a masked transformer architecture to predict physical time series using a hybrid architecture with a known physics model. The masked transformer predicts the residual instead of directly the time series signal, which is probably the main novelty. How the masked is constructed (its basically a next time step prediction with holding out some additional variables) and how the mask is given to the transformer. The approach is evaluated on data from a magnetic motor.

Critical strengths
  • A new dataset for physical time series prediction for magnetic motors
Critical weaknesses
  • Very limited novelty which is "masked" by introducing many novel terms for existing things such as "epistemic state" or "mechansistic prediction". In the end the approach is a standard masked auto-encoder with normalization of the input
  • Only one experiment. No comparisons to SOTA or simple base (hybrid and purely data driven) models (recurrent models, transformers with traditional mask encoding) and no ablations. I have no idea whether the presented results are good or useful as I am no expert in the application, but there are a multitude of approaches that could be used for such datasets and compared to.
  • The writing is very hard to follow. Most details are hidden in the appendix, main text describes standard things in a confusing way. In addition, the abundance of tables do not help the readability of the paper
Questions for authors
  • Please explain the difference to standard architectures such as mask auto-encoders
  • If there is a significant difference, please ablate it
  • The overview diagram is very confusing
  • What makes the state definition "epistemic"? Why is this different from a standard state with a mask flag?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.