acceptodds
Under review as a conference paper at ICLR 2027

Interpretable Deep State Space Models via Semantic Alignment

Abstract

Deep state-space models (DSSMs) are promising tools to control complex dynamical systems, but their latent states are typically unintelligible to humans and therefore difficult to inspect, steer, and debug. In response, we propose a general methodology for constructing expressive and intrinsically interpretable DSSMs that permit the alignment of both their state representations and dynamics with human semantics. Specifically, we decompose a DSSM's internal representation into an expert-defined dynamical concept state and a residual latent state, which is learned to compensate for the predictive information that is missing from the concepts. Notably, the resulting formulation supports interventions that propagate through the learned dynamics and steer the subsequent concept trajectory. We instantiate the methodology in both deterministic and stochastic model variants and evaluate them across four dynamical systems, demonstrating free-running concept forecasting accuracy on par with opaque DSSMs while also enabling interventions.

Primary area:
interpretability and explainable AI
Keywords:
Interpretability, Intrinsic Interpretability, Concept-based Interpretability, Deep State Space Models, Steerability, Semantic Alignment

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_6nkzOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an interpretable DSSM (IntDSSM) whose state includes expert-defined concepts. The concept dynamics combine a user-specified mechanism derived from domain knowledge with a neural network that compensates for the residuals. The paper claims that the design of IntDSSM improves forecasting accuracy and demonstrates through experiments that it enables interventions.

Critical strengths

The design in Section 3 is rigorous and well-founded, both in accounting for the concept closure gap and in compensating for the discrepancy between the user-specified mechanism and the mechanism of the real system.

Critical weaknesses
  1. The paper claims that IntDSSM can handle interventions, but in both the definition in Section 3.3 and the experiments in Section 4.3, the learned model is corrected with the ground-truth concept values or mechanism, which is closer to data assimilation than to intervention. In general, an intervention means that, if we change the concepts or the mechanism of both the true system and the learned model, the learned model should be able to capture the resulting change in the true system. IntDSSM may fail to handle this, since its latent dynamics involve both c_t and h_t, whereas an intervention can only change c_t or the mechanism f_ϑ, which is insufficient without also updating h_t.
  2. The core idea of IntDSSM is very similar to hybrid (physics + neural) models, which combine a predefined physics function with a neural network to fit the mechanism of the true system. The paper should discuss these related works and add an ablation that simply removes the physics function, to show how much the physics component contributes in this hybrid design.
Questions for authors
  1. How should "interpretability" be understood in this paper? Does it mean that the learned representations (concepts) carry specific semantic information, that the learned dynamics are operable (e.g., via interventions), or that they incorporate domain knowledge?
  2. Access to concepts is in fact a strong assumption. Some hybrid models, such as the physics-integrated VAE (PI-VAE; Takeishi & Kalousis, 2021), do not make this assumption and only embed the physics function into the framework. How do you view this difference?
  • Takeishi & Kalousis (2021). Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling. NeurIPS 2021.
4 comments
AuthorsOpenReview ↗

Rebuttal 1/2

We thank the reviewer for their review, and address the main points below.

W1 – Interventions

The paper claims that IntDSSM can handle interventions, but [...] the learned model is corrected with the ground-truth concept values or mechanism [...]. In general, an intervention means that, if we change the concepts or the mechanism of both the true system and the learned model, the learned model should be able to capture the resulting change in the true system. IntDSSM may fail to handle this [...]

There might be a misunderstanding here, as the term intervention has different interpretations depending on the field. Please allow us to clarify.

The reviewer’s comment follows the meaning in causal inference. There, an intervention acts on a causal model of the data-generating process and its effect is meant to describe the response of the true system. In the field of interpretability, an intervention is a form of user--model interaction in which a user changes the value of one or more of the model's internals (in our case, concepts and/or mechanisms) and observes how the model responds [1, 2, 3, 4]. Interventions allow users to correct a model at test time, steer it toward a desired behavior, or analyse the causal concept effect (in the sense of [5, 6, 7]) on model predictions.

In our experiments, ground-truth interventions are a particular type of intervention where the values simulate the action of an ideal expert. Whether the model also reproduces the response of the true system is a question of causal reliability (see also Q1). Answering it requires a structural causal model, of which the intersection with concept-based models is an active research area even in static settings [6, 7], and beyond our scope.

On this matter, we recognize that Sec. 3.3 used a general heading but then discusses only ground-truth interventions. We have corrected this with a preamble that clarifies the above, and renamed the following paragraph to Ground-truth concept interventions.

We thank the reviewer for highlighting this point, as it improved the clarity of the paper.

W2 – Hybrid models

The core idea of IntDSSM is very similar to hybrid (physics + neural) models

We agree with the reviewer that hybrid physics–neural models, such as PI-VAE, share some methodological aspects with our approach. Yet, the two approaches fundamentally differ in scope. Hybrid models target world–model alignment: they inject physical knowledge about the real-world phenomenon into the model, so that the model generalizes better. Interpretability targets user–model alignment, and accuracy is pursued subject to it. This has consequences for both variables and mechanisms:

  • Variables. In interpretability, the model concepts need not correspond to physical variables. A user may still choose to reason in terms of physical concepts, but this is not required. Since concepts are typically inferred from high-dimensional inputs, their alignment must be explicitly enforced (D1). Instead, hybrid physics-neural models generally assume the system is measured directly in the variables of interest (alignment is presumed), or these are latent variables learned without supervision, leaving the model's internal states with no agreed semantics with the user, a condition that is instead the starting point of intrinsic interpretability.

  • Mechanisms. While hybrid models typically assume that the functional form of the physics is known, intrinsic interpretability only requires the concept compositions to belong to a (possibly broad) class of functions (D2). For example, a user who knows nothing about the phenomenon may restrict the mechanism to linear maps, in favor of a simple explanation (as for our experiment with Lorenz–96); concept bottleneck models commonly make this choice for their task predictor [1].

This difference is also reflected in the two distinct, albeit adjacent, research communities: physics-informed machine learning, and intrinsic (concept-based) interpretability.

We have added a dedicated discussion of hybrid physics–neural models to the Related Work section (ll. 459-477). We thank the reviewer for raising this point.

The paper should [...] add an ablation that simply removes the physics function, to show how much the physics component contributes in this hybrid design.

As discussed above, the physical equation in a hybrid model helps the model better match the real world, and thus generalize better. In contrast, the interpretable mechanism helps the user understand the model's behavior (semantic alignment), regardless of whether it mirrors the real world. Incorrect mechanisms are therefore completely acceptable in interpretability, as long as they belong to the class of functions the user understands. For this reason, we believe that an ablation removing the mechanism would not be informative in our setting.

AuthorsOpenReview ↗

Rebuttal 2/2

Q1 – Meaning of interpretability

How should "interpretability" be understood in this paper?

In our paper, interpretability must be understood as semantic alignment between the model and the human. It requires that (1) the model and the human associate the same meaning with the same symbols (D1), and (2) the mechanisms are tractable for human understanding (D2).

On the three readings proposed by the reviewer:

learned representations (concepts) carry specific semantic information

Yes, at least partially. This corresponds to D1 in our paper.

the learned dynamics are operable (e.g., via interventions)

We see the possibility of performing interventions as a consequence of the alignment above.

incorporate domain knowledge

More generally, the concept dynamics must belong to a hypothesis class that the user deems admissible (D2). This class includes domain knowledge but is not limited to it, as it can also reflect human preferences (see W2).

Q2 – Concept availability

Access to concepts is in fact a strong assumption. Some hybrid models, such as the physics-integrated VAE (PI-VAE; Takeishi & Kalousis, 2021), do not make this assumption and only embed the physics function into the framework. How do you view this difference?

To discuss this point, please allow us to go back to the difference in scope between hybrid models and interpretability (see W2). If the goal is better generalization, it is perfectly fine to supervise only the output and to fix a mechanism from domain knowledge, which adds robustness. If the goal is instead the interpretability of the model's inference process, we remark that concept alignment is a necessary requirement in intrinsic interpretability. Otherwise, the model is free to assign arbitrary semantics to the concepts, and any explanation loses its meaning and reliability, i.e., the language between model and human is different. Crucially, fixing the mechanism is not sufficient to fix the concepts, since in general there exist multiple inputs that produce a given output (as in reasoning shortcuts). This is exactly the reason why we demand two separate desiderata, one for concepts and one for mechanisms.

As further support, concept availability is a standard assumption in concept-based interpretability [1, 4]. Nevertheless, the community is aware that annotating concepts can sometimes be costly, which is motivating active research in automatic concept annotation pipelines [8, 9] (generally referred to as label-free approaches).

We hope to have clarified all of the reviewer's concerns and are happy to provide further details if needed.

References

  1. Koh et al., Concept bottleneck models, ICML, 2020.

  2. Steinmann et al., Learning to intervene on concept bottlenecks. ICML, 2024.

  3. Ismail et al., Concept bottleneck generative models, ICLR, 2024.

  4. Poeta et al., Concept-based explainable artificial intelligence: A survey, ACM Computing Surveys, 2025.

  5. Goyal et al., Explaining classifiers with causal concept effect (CaCE), arXiv, 2019.

  6. Dominici et al., Causal concept graph models: Beyond causal opacity in deep learning, ICLR, 2025.

  7. De Felice et al., Causally reliable concept bottleneck models, NeurIPS, 2025.

  8. Oikarinen et al., Label-free concept bottleneck models, ICLR, 2023.

  9. De Santis et al., Learning concept bottleneck models from mechanistic explanations, ICLR, 2026.

Thank you for the explanations and clarifications. I am not very familiar with concept-based interpretability or semantic alignment. From the perspective of time-series modeling and causal inference, I maintain my earlier assessment: in terms of its framework, the paper is still hybrid modeling that requires ground-truth concepts, and it cannot perform interventions in the causal sense. Nevertheless, this work may well be valuable to the concept-based interpretability community, so I am raising my score from 2 to 3 and lowering my confidence to 2.

We deeply thank the reviewer for the discussion, its honest considerations, and the positive assessment.

Reviewer_uhN2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper considers modeling interpretable deep state-space models by decomposing the concept state into admissible concepts plus an hidden state, where the concept in the next state is the sum of (i) a function from a user-specific class and (ii) a learned residual correction. The authors show on various simulation studies that the model performs well while furnishing more interpretable and intervenable concepts.

Critical strengths
  1. I think the main idea (at least in the main body) is rather cleanly communicated.
  2. I think Figure 3 cleanly communicates and echoes with the core methodological contributions, making it easier to follow overall.
Critical weaknesses
  1. I am not sure how much the mathematical exposition is warranted, i.e., it appears that this decomposition has appeared in similar prior works? For instance, see pg. 5 of APHYNITY (Yin et al., 2021), which also "decomposes the dynamics into physical and augmented terms ... leveraging [prior] knowledge."
  2. There does not appear to be limited investigation of the trade-off between the expressiveness of the experts-concept class and the residual prediction: i.e., I wonder whether if it is entirely possible for the residual correction to "override" the expert mechanism. What will happen in this case, i.e., how will you be able to preserve the desirable properties of your existing framework?
Questions for authors

See above in weaknesses Q2.

6 comments
AuthorsOpenReview ↗

Rebuttal 1/2

We thank the reviewer for their positive feedback. We address the two points below.


W1 – Positioning

I am not sure how much the mathematical exposition is warranted, i.e., it appears that this decomposition has appeared in similar prior works? For instance, see pg. 5 of APHYNITY (Yin et al., 2021) [...]

We agree with the reviewer that the decomposition is a methodological aspect our work shares with hybrid physics-neural approaches such as APHYNITY and PI-VAE. We certainly do not mean it as a contribution per se. Rather, our mathematical exposition establishes when and why the decomposition is needed under semantic alignment, namely when the concept process is not dynamically closed (Eq. 5), and shows that it comes with no loss of expressivity (Proposition B.1). Yet, hybrid physics-neural approaches fundamentally differ in scope from intrinsic (concept-based) interpretability. We kindly refer the reviewer to our response W2 to Reviewer 6nkz on this same matter.

AuthorsOpenReview ↗

Rebuttal 2/2

W2 – Role of the residual

[...] I wonder whether if it is entirely possible for the residual correction to "override" the expert mechanism. [...] how will you be able to preserve the desirable properties of your existing framework?

We find this an interesting point.

First, note that the properties we discuss require only exposing the aligned concepts and transitions. These create the human–model interface through which users inspect and intervene on the model.

Once we also require expressivity, an open question arises about which design principles should guide the extension of the ideal interpretable model, depending on the properties one demands of the complete model. In our paper, we propose a methodology to keep D1 valid and recover D2 for r=0r=0 (l. 155). Further properties beyond recoverability (or alternatively to it) would entail trade-offs, each potentially deserving a dedicated study.

In general, the chain-rule factorization of zt=(ct,ht)z_t = (c_t, h_t) (Sec.3.1) makes the concepts an input to the residual pathway. This permits the residual to adapt after an intervention whenever an unobserved factor is related to the intervened concepts and, in stochastic settings, to account for the realized concept values (Remark 3.1). We therefore deliberately remain general and retain this dependence in favor of responsiveness to interventions.

As the reviewer points out, however, a residual that is aware of the concepts can also suppress the effect of interventions transmitted through the admissible transition. Guaranteeing that this cannot occur would be an additional desideratum. We see two possible paths:

  • Keeping our general methodology and diagnosing post-hoc whether cancellation occurs. Specifically, consider an intervention ctˉ↦c~tˉc_{\bar t} \mapsto \tilde c_{\bar t}, and let r(h,c)r(h, c) denote the residual decoded by Decr\mathrm{Dec}_r. By the additive form of Propc\mathrm{Prop}_c, the change in the next concept state splits into two path-specific effects: c~tˉ+1−ctˉ+1⏟Δc=fϑ(c~tˉ)−fϑ(ctˉ)⏟Δf: mechanism path+r(h~tˉ,c~tˉ)−r(htˉ,ctˉ)⏟Δr: residual path\underbrace{\tilde c_{\bar t+1} - c_{\bar t+1}}_{\Delta^{c}} = \underbrace{f_\vartheta(\tilde c_{\bar t}) - f_\vartheta(c_{\bar t})}_{\Delta^{f}\text{: mechanism path}} + \underbrace{r(\tilde h_{\bar t}, \tilde c_{\bar t}) - r(h_{\bar t}, c_{\bar t})}_{\Delta^{r}\text{: residual path}} We then measure the component of the total effect Δc\Delta^{c} along Δf\Delta^f: β:=⟨Δc,Δf⟩∥Δf∥22,Δf≠0.\beta := \frac{\langle \Delta^{c}, \Delta^{f} \rangle}{\|\Delta^{f}\|_2^2}, \qquad \Delta^{f} \neq 0. If the residual neither suppresses nor reinforces the mechanism, the complete model follows the change prescribed by fϑf_\vartheta, and β=1\beta = 1. This permits detecting whether the effect of the mechanism remains faithful to the original even after introducing the residual. When the residual suppresses (reinforces) the mechanism, then β<1\beta < 1 (β>1\beta > 1); this permits detecting when the model is altering the effect of how the mechanism fϑf_\vartheta responds.

  • By construction. Removing the concepts as an input to the residual (i.e., an ablation of our approach) guarantees that interventions act with the same effect as in the original model (β=1\beta = 1 by construction). However, one would trade off the responsiveness properties discussed above.

We measured β\beta for the complete model (IntDSSM-D) and for the new ablation (mechanism-faithful IntDSSM-D) on N-CMAPSS. We trained both variants first with a mechanism fϑf_\vartheta. Then, training with −fϑ-f_\vartheta, we test whether the residual learns to override a mechanism whose concept effect contradicts the data. The table reports β\beta (mean ± 2 SE over seeds) for a ground-truth intervention at tˉ=0\bar t = 0 on the test set.

β\betafϑf_\vartheta−fϑ-f_\vartheta
IntDSSM-D0.987 ± 0.001-0.815 ± 0.046
IntDSSM-D (mechanism-faithful)1.000 ± 0.0001.000 ± 0.000

With fϑf_\vartheta, IntDSSM-D leaves the effect along the direction of the mechanism nearly unchanged (β=0.987\beta = 0.987). With −fϑ-f_\vartheta, β<0\beta < 0: the residual largely restores the original direction of the concept effect. The mechanism-faithful ablation keeps β=1\beta = 1 under both mechanisms, as guaranteed by construction; with −fϑ-f_\vartheta, its interventions keep propagating with the inverted effect. This illustrates the trade-off between the two paths: the ablation guarantees faithfulness to the exposed mechanism, whereas IntDSSM-D can correct it, at the cost of the concept effect possibly being no longer attributable to fϑf_\vartheta alone.

In the revised manuscript, we now discuss both options in a new self-contained Appendix C. If the reviewer agrees, we would use the additional page to add the mechanism-faithful ablation to the main experiments. We thank the reviewer for raising this, which improved the paper.


We hope to have clarified all of the reviewer's concerns and are happy to provide further details if needed.

Thanks so much to the authors for the reply!

Overall, I find this to be a technically solid paper and the only one in my batch that is readable. That being said, I still believe that while my points are addressed, that there are way too many different things floating around in this paper, making it a bit too bulky and difficult to digest how each component interacts with the main takeaway.

While I do not have any specific comments, I would be happy to consider raising this to a strong accept if the authors are able to streamline and polish the exposition of the paper to 1) highlight what should the core takeaway be 2) what is the mathematical insight that truly needs to be written and fleshed out and presented to the reader 3) overall better articulation of each section and how it relates to the central story. Thanks so much again!

Reviewer_aLLeOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a general methodology for building intrinsically interpretable deep state-space models by decomposing the semantic flow into an expert-defined concept-based function and an unconstrained residual hidden state flow. The two pathways are coupled across levels and time, so concept interventions can influence subsequent rollouts rather than only the current prediction. Evaluations are done on Lorenz-96, Lotka-Volterra, Relative Rotation Graphs, and N-CMAPSS. The experiments show forecasting accuracy comparable to opaque DSSMs, stronger semantic align, and robustness of predictive accuracy to mechanism misspecification.

Critical strengths
  1. A meaningful dynamical extension of concept-based interpretability. The strongest contribution is not the concept-bottlenect + residual concept by itself, but to generalize the idea from static concept to semantic flow.

  2. experiment covers many different cases and testified the idea from different angles

Critical weaknesses
  1. Mechanism alignment is not yet the same as mechanistic faithfulness. The N-CMAPSS stress test shows that, after retraining, the residual can compensate for persistence or even reversed-degradation priors while preserving similar forecasting accuracy. This is a valuable result, but it also weakens the central interpretability claim: the exposed transition may be a human-readable scaffold that the opaque residual cancels, rather than a mechanism genuinely used by the complete model. Residual magnitude or intervention responsiveness diagnoses compensation but does not by itself establish the causal or computational importance of the exposed mechanism. This creates a structural trade-off: when the expert mechanism explains nearly everything, the need for a deep residual model is less clear; when it is weak or wrong, the residual may dominate the effective computation.

  2. The novelty and related-work positioning should be sharpened. The main novelty appears to be the dynamical propagation of expert-defined concepts together with an admissible transition, rather than concept-residual decomposition per se. In addition to concept-residual and temporal concept-bottleneck work, the manuscript should discuss two closely related works Function Alignment: A New Theory for Mind and Intelligence, Part I: Foundations (arXiv:2503.21106, 2025), and Bridging Perceptual and Analytic Dynamics via Function Alignment (ICLR 2026 Re-Align Workshop, OpenReview: TuuyJZrDm5). Both formalize heterogeneous autoregressive processes at different representational levels that influence one another bidirectionally across time. The present recursion seems structurally close: the concept state affects residual/hidden dynamics, while the hidden state corrects the next concept state.

Questions for authors
  1. Mechanism quality. The current N-CMAPSS result suggests that predictive accuracy alone cannot distinguish a meaningful mechanism from a poor one when the residual can compensate. What evidence would justify treating human concept as a real explanation rather than an arbitrary interpretation?

  2. Necessity of the deep residual pathway. On RRG and N-CMAPSS, where the concept-only mechanism explains much of the forecast, what concrete advantage does the deep state-space residual provide over a simpler structured dynamical model with a learned residual correction?

2 comments
AuthorsOpenReview ↗

rebuttal 1/2

We thank the reviewer for their positive feedback and comments. We address the main points below.


W1 – Mechanistic faithfulness

Mechanism alignment is not yet the same as mechanistic faithfulness.

We agree that the two properties differ, and we find this a very interesting discussion.

A contribution of our paper is to formalize and construct an aligned concept-only interpretable model in a sequential setting, which, as the reviewer notes, is sound w.r.t. both D1 and D2. Once we step away from this ideal model and require the complete model to be also fully expressive, an open question arises on which design principles should guide this extension. We believe there are several possible approaches to this question, depending on the properties one demands of the complete model. For example, in our paper, we demand recoverability, and achieve it by combining architectural (additive residual) and training choices (separate losses with gradients stopped on the concepts pathway while training the residual). This guarantees we can always recover the aligned interpretable model when needed. Beyond recoverability (or alternatively to it), one might demand further properties of the complete model. Yet, we expect this to lead to trade-offs, and each set of properties may deserve its own dedicated study.

One simple example we can think of is to require that the intervention effect of the concepts on the predictions is the same in both the complete model and the ideal concept-only model (i.e., an instance of 'mechanistic faithfulness'). This can be achieved by an ablation of our approach: by removing all the connections from the concepts to the residual pathway. However, it would also prevent the residual from updating based on the actual concept realizations (in probabilistic settings) or updating after an intervention whenever an unobserved factor is related to one of the intervened concepts (Remark 3.1), reducing responsiveness.

We kindly refer to our response W2 to Reviewer uhN2, where we discuss how we can (i) keep our general methodology and diagnose post-hoc whether cancellation occurs, or (ii) add an assumption that prevents cancellation by construction. We thank the reviewer for raising this aspect, which improved the paper.


W2 – Positioning w.r.t. function alignment

[...] the manuscript should discuss two closely related works [...]

Thank you for these references. We agree they share some methodological aspects with the decomposition we adopt, which should definitely not be intended as a methodological novelty per se. As you note, our main contribution is the dynamical extension of the expert-defined concept bottleneck using an SSM formulation, and we have revised the contribution statement to avoid possible confusion (ll. 62-67). However, the line of work on function alignment fundamentally differs in scope from ours and, arguably, from concept-based interpretability in general.

Function alignment aims at forecasting accuracy and stability under the availability of a given analytic model. Our goal is instead that of intrinsic interpretability: we extract expert-defined concepts from high-dimensional observations (the only input to the model), constrain the model to reason through them, and pursue accuracy only subject to this interpretability constraint. Furthermore, the mechanism constraint can be much more relaxed (e.g., a broad class of functions).

Since this is similar to the distinction with hybrid physics-neural models (raised by W2 to Reviewer 6nkz), we now discuss both lines of work together in an additional paragraph to the Related Work (ll. 459-477).

AuthorsOpenReview ↗

rebuttal 2/2

Q1 (related to W1) – Evidence for a real explanation

[...] the exposed transition may be a human-readable scaffold that the opaque residual cancels [...]

[...] What evidence would justify treating human concept as a real explanation rather than an arbitrary interpretation?

Our proposed methodology does not demand to preserve the effect the concepts have on predictions from the ideal interpretable model to the complete model. As from the discussion above, we believe this would constitute an additional desideratum, with its own trade-offs. The same cancellation concern was also raised by W2 to Reviewer uhN2. We kindly refer the reviewer to this response, where we discuss how we can (i) keep our general methodology and diagnose post-hoc evidence on whether such a cancellation occurs, or (ii) add an assumption that prevents cancellation by construction.


Q2 – Necessity of the deep residual pathway

On RRG and N-CMAPSS, where the concept-only mechanism explains much of the forecast, what concrete advantage does the deep state-space residual provide over a simpler structured dynamical model with a learned residual correction?

We are not sure we fully understand the question, please allow us to consider two possible interpretations.

  • if the reviewer refers to a learned correction on the concepts, e.g., ct+1=fϑ(ct)+g(ct)c_{t+1} = f_\vartheta(c_t) + g(c_t), the resulting transition would remain a function of the concepts only, so it cannot recover the predictive information missing from them (hence accurate only under dynamical closure). Closing the gap requires access to the observation history and a state that carries this information forward during the rollout, which is what our residual pathway provides.

  • if the reviewer refers to a correction that is a function of the input yet simpler than a neural network, such corrections are perfectly compatible with our framework. Our deep residual pathway is just one possible instantiation, which we chose to mimic the opaque DSSM counterpart while exposing the concept interface. On RRG and N-CMAPSS, simpler corrections may work as well. Note how our residual pathway is lighter than it may appear. In the deterministic instantiation, the hidden state is propagated by an affine map (Eq. 25), and the residual decoder is a small MLP (e.g., a single 16-unit layer on RRG, Table 4). The only deep component is the encoder Ench\mathrm{Enc}_h, which is needed to extract the missing information from the high-dimensional observations.

Please let us know if we have misinterpreted the question.


We hope to have clarified all of the reviewer’s concerns and are happy to provide further details if needed.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their time and constructive feedback. We uploaded a revised manuscript with all changes highlighted in blue. We summarize them below.


Summary of changes

1. Positioning and related works (Revs. aLLe, uhN2, 6nkz). We have added a new Related Work paragraph (ll. 459–477) discussing hybrid physics-neural models (including function alignment). These models share our additive structure but differ in scope. We detail the consequences for both model variables (D1) and mechanisms (D2).

2. Post-hoc diagnosis of mechanism faithfulness (Revs. uhN2, aLLe). We have added a new Appendix C that discusses a post-hoc diagnostic of the faithfulness of the residual model to the concept-only pathway.

3. Contribution statement (Rev. aLLe). We have revised a small portion of the abstract and introduction to clarify our core contribution in the dynamical propagation of the expert-defined concept bottleneck.

4. Meaning of interventions (Rev. 6nkz). We have added a preamble to Section 3.3 on interventions in concept-based interpretability to clarify their meaning. Accordingly, we renamed the next paragraph Ground-truth concept interventions, the form used in our experiments.

5. Minor polishing. We have fixed a typo in the Proof of Proposition B.4 (App. B.2). We have also made a minor specification in App. E.4.

We hope these changes address the reviewers' concerns, and we are happy to provide further details during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.