acceptodds
Under review as a conference paper at ICLR 2027

RelationFirst: What Composition Requires from Learned Representations

Abstract

Accurate local predictions do not establish whether a learned representation will remain reliable within a larger computation. The open question is what information the consuming computation needs and how errors in that information affect its result. RelationFirst makes both requirements precise. Composition sufficiency identifies the distinctions a component must preserve across contexts, while propagation geometry determines how local approximation errors affect the assembled result. Stationary elliptic PDEs make these requirements testable. Classical elimination exposes an exact interface for the discrete problem, and the assembled equilibrium reveals how response errors propagate. We train models on isolated components, freeze them, and reuse them in held-out assemblies. With the output decoder and composer fixed, changing only the supplied interaction shifts the global equilibrium. A single propagation law predicts these effects across distinct nonlinear systems, including cases where a learned response helps and where it harms. Perhaps counter-intuitively, in a controlled intervention, even a local error with only half the magnitude yields a larger global error across all 72 tested assemblies, driven purely by the shift in error direction. These results show why reusable representations must be evaluated through the behavior they preserve under composition and the consequences of their errors in use. These criteria can guide the design and evaluation of learned components in numerical solvers and coupled simulators.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
representation learning, compositional generalization, reusable representations, modular learning, representation evaluation, scientific machine learning, neural PDEs

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_RwaHOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when a learned local representation remains reliable after being reused as a component of a larger computation. The central argument is that local output accuracy alone is insufficient for compositional reuse: a component must preserve the information required by its surrounding computation, and errors in this interaction information should be evaluated according to how they propagate through the assembled system. The paper formalizes these ideas as composition sufficiency and propagation geometry.

Critical strengths
  1. The controlled intervention is clean and convincing. Instead of training two complete systems and attributing their differences to “better representations,” the authors freeze the output model and composer and change only the interaction response. This isolates the phenomenon much better.
  2. The propagation result explains the effect.
Critical weaknesses
  1. The main conceptual claim appears substantially broader than what is theoretically or empirically established.
  2. The empirical evidence is narrow relative to the broad representation-learning framing. The headline controlled experiments are based primarily on two nonlinear elliptic reaction families with 32 and 16 held-out 2D cases respectively. The controlled error-direction study uses 72 assemblies, but these are again 1D cubic elliptic systems rather than a qualitatively different compositional learning problem
Questions for authors

Could you please explain or address the weaknesses raised above?

Reviewer_TkgZOpenReview ↗
Rating
1
Confidence
1
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies whether learned local components remain reliable when they are composed into a larger computation. As accurate local representations do not necessarily lead to good global performance, the authors propose a framework to focus on "composition sufficiency", i.e., what information a component must preserve for composition and "propagation geometry", i.e., how local approximation errors affect the assembled result. Experimental results and theoretical analysis demonstrate that the global errors depend on both the direction and magnitude of the local errors.

Critical strengths
  1. This paper addresses an important problem and helps explain why simply combining accurate local models into a larger computational system can still produce unreliable results.

  2. The experiments are designed with well-controlled settings. For example, the field model, physical inputs, mesh, port, initialization, and global Newton solver are fixed, while only the local response and its consistent tangent are exchanged.

Critical weaknesses
  1. The paper is generally very difficult to follow as the authors introduce many technical terms without first explaining the intuition behind them.

  2. Concerns regarding the experimental results: For example, as shown in Table 2, Analytic-3 achieves the lowest errors across all three PDE families. Thus, the advantage of Analytic-2+NN over the analytic methods is not very clear to me. The paper does not explain when a learned evaluator would be useful.

Questions for authors
  1. Could the authors explain why the learned relation improves the results for cubic but substantially increases the errors for sinh, as shown in Figure 1?

  2. For the experiments on changing error direction in Section 4.3, the number of components (N) was set to 8, 16, and 32. Figure 3b shows that the global errors grow with increasing N. Could the authors provide more explanation on this trend?

  3. Could the authors discuss for which application scenarios the learned evaluator pipeline can be faster than FEM?

Reviewer_Ku8yOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This manuscript, I think, observes the well known fact that errors propagate to downstream tasks. It notes that understanding how that happens for learned models is important. Nominally all this is only addressed in the context of neural network solvers for PDEs.

Critical strengths

None

Critical weaknesses

It is not at all clear that this manuscript has any (substantive) contribution. Irrespective of that point, even if a clear contribution could be identified, the manuscript in its current form is inscrutable. There is no narrative structure and no clear message; in fact, the intro does not even clearly and precisely state what the problem being considered is. Terms are introduced without being defined and commentary appears out of nowhere. Many paragraphs read as distinct (and often hard to understand) statements.

An illustrative example: a Newton solver is repeatedly referenced (the first time in the main text as "Newton code"), yet it is never clear what problem this solver is applied to. Perhaps there are interesting ideas buried in there, but substantial work needs to be done to revise the manuscript to clearly articulate them.

Relevant to the above is that the manuscript is ripe with "Cladueisms" (replace with whatever model was actually used) to the point of being distracting. I have no issue with the use of LLM tools if the resulting manuscript is well written and structured, but this one is not.

Questions for authors

None

Comments on the paper

2 comments

Dear Reviewers,

The authors have posted their rebuttals. As the author-reviewer discussion period is coming to a close soon, please read their responses and engage in the discussion. At a minimum, please acknowledge that you have read the authors’ responses. Thank you for your time and effort.

Best,

AC

Thank you for taking the time to read our responses. We understand your position and appreciate your candid assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.