acceptodds
Under review as a conference paper at ICLR 2027

SAME ERROR, DIFFERENT GEOMETRY: RELATIONAL GENERALIZATION ON COMBINATORIAL LANDSCAPES

Abstract

Standard predictive error can be insufficient on structured combinatorial spaces because models with similar aggregate error can organize residuals differently over the underlying graph. This matters when predictions are used not only to estimate individual fitness values, but also to recover mutational effects, interactions, landscape structure, and search-relevant relationships. We formalize this distinction as relational generalization, separating pointwise predictive risk from first- and second-order graph-relative fidelity. Across four empirical RNA fitness landscapes and seven model classes, natural matched-risk comparisons showed limited first-order separation, motivating a controlled intervention that exactly matched held-out pointwise error while preserving learned residual organization. Despite identical held-out pointwise risk, learned residual directions retained substantial differences in first- and second-order relational error, with second-order error differing by up to 2.8× within the same system and split. These differences altered reconstructed local-optimum and accessibility structure. Mutation-sign disagreements were concentrated among low- to moderate-margin effects, while high-margin effects were largely stable, consistent with a margin bound linking relational error to sign errors above a specified effect margin. Controlled spectral constructions on Domingo tRNA and Tetrahymena thermophila provided a complementary demonstration that equal aggregate error can coexist with different relational geometry and reconstructed topology. Relation-aware training further reduced relational error in both tested biological systems, although pointwise equivalence was not established. Sensitivity analysis showed that the main joint-preservation result was robust across a broad range of first- versus second-order weightings. Overall, our results show that aggregate predictive error alone does not determine relational geometry or structural fidelity, motivating relational generalization as a complementary criterion for evaluating and training models intended for structured biological inference and sequence search.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Relational Generalization, Combinatorial Generalization, Representation Learning, Inductive Biases

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FMBXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This manuscript argues that pointwise prediction error is insufficient for evaluating surrogate models on combinatorial fitness landscapes. Given a true fitness function ff, prediction f^\hat f, and residue e=f^−fe = \hat f - f, it distinguishes pointwise error R0R_0 from errors in first-order mutational effects R1R_1, and second-order mutation-square interactions R2R_2. The key math observation is that fixing ∣∣e∣∣||e||, and hence pointwise risk, doesn't determine quantities such as eTLee^TLe; therefore residual fields with equal magnitude can have very different geometry relative to the mutational graph. Further, the manuscript derive margin-based bounds connecting R1R_1 and R2R_2 to mutation/inter-action sign errors and a corresponding bound on changes in high-margin local-optimum status.

Critical strengths

The concept is useful and the math formulation is clean. In the science surrogate modelling, we often care about derivatives, orderings, interactions, optimization trajectories rather than only scalar prediction RMSE. The paper turns the intuition into a precise graph-based formulation. The argument that two predictors with comparable fitness RMSE may induce different mutational gradients is very insightful. The experiments are well-controlled and convincing as mechanism demo. Plus, the metrics are made to be actionable. Relational-aware training is important because otherwise the contribution risks being only diagnostic. The intervention shows that the relational quantities can be changed by the learning objective, with order-specific effects rather than a trivial uniform smoothing effect.

Critical weaknesses
  1. The strongest empirical phenomenon is counterfactual rather than naturally occurring. The predefined natural matched-R0R_0 exp is negative, where the hypothesized large separation doesn't occur. The strongest results instead come from taking already-trained residuals and using GT labels to construct f~=f+ce\tilde f = f + ce so that test R0R_0 is exactly equal. This means the exp demonstrates possibility conditional on residual direction, not that realistic independently trained models with comparable generalization error routinely exhibit significant relational divergence. Relation-aware training improves RrelR_{rel}, but pointwise equivalence fails under the prespecified criterion. So I think the result doesn't yet demonstrate that "better relational fidelity at the same predictive accuracy"; a pointwise/relational trade-off is still possible.
  2. Measurement noise for second differences. A second-order finite difference combines 4 experimental measurements and can amplify assay noise. The manuscript reports that about 62.5% of Domingo mutation squares are assay-ambiguous, although the model ordering survives when restricting sign recovery to assay-confident squares. This highlights the problem that noise-aware evaluation is unavailable across most datasets and is not built directly into R2R_2 or the training loss.
Questions for authors
  1. Can you demonstrate a substantial relational difference between independently trained models that naturally achieve equivalent R0R_0, without any post-hoc residual rescaling?
  2. What is the intended practical use of the learned-direction calibration?
  3. Can relation-aware training improve R1/R2R_1/R_2 while enforcing actual pointwise equivalence rather than only a validation admissibility threshold?
  4. What are the exact split geometries? In particular, how do the conclusions change under Hamming-distance extrapolation, held-out mutation/site splits, or other splits (which may prevent near-neighbor leakage between training/testing genotypes)?
Reviewer_rDEhOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper argues and shows that pointwise predictive error cannot capture higher-order errors in fitness landscapes. E.g. two models might have the same pointwise error but different local/global optima in the fitness landscape graph. They define first (edge-)order and second (square-)order errors and relate these to each other and to the number of edges/squares that have reversed signs copared to the true fitness landscape.

Critical strengths
  • Clear Motivation: The motivation that different pointwise errors can lead to markedly different residual geometries which might have a big effect on downstream analysis (e.g. search) is clear and convincing.
Critical weaknesses
  • The emprical analysis is not convincing:
    • it lacks clarity: often it is hard to understand what is being tested and why, it is very hard to read and lacks organization.
    • Experiment 5.1. shows that similar pointwise R0R_0 error seldomly occurs in practice between models, and if it occurs the higher-oder R1R_1 error does not differ much between these models. This somewhat contradicts the central motivation of the paper that in practice only consdering the R_0 error is not enough and can come with different higher-order errors
    • The following experiments artifically rescale the residual error such that all models attain the same pointwise R0R_0 error. They then show that this can lead to substantially different higher-order errors. However, fixing the R0R_0 error, seems like a fundamental limitation of this analysis: It is quite clear/not surprising that a worse point-wise model will also have a higher R1R_1 and R2R_2 error even if the residual is rescaled. As I understand it, these experiemnts do not capture any effects of practical trained models (which are not rescaled) and therefore cannot support the central motivation of the paper.
    • The analysis of relation-aware objectives is hard to understand (for me) and seems incomplete. The aouthors themself acknoledge that this is only "explorative".
  • Theoretical results are not interesting on their own. It is not hard to see that different pointwise errors can lead to differerent global error structures, and the theoretical results (e.g. Prop 1 and Corollary) border on being trivial.
Questions for authors

The most important question to me is Question 1:

  1. Which concrete exprimental results establish in practical trained models that similar pointwise errors can lead to markedly different higher-order errors? So which experiments actually support your motivation that in real practically trained models it is not enough to only consider the R_0 error?
  2. How is a mutation square and its "four-corner second difference" defined?
  3. How exactly is the relation-aware training performed? I did not understand this based on your description.
  4. I am having trouble understanding the smooth/high-frequency experiment: You decompose the residual into a high-frequency and a low frequency component. Then, you rescale it to the same R_0 error. Then you observe different R_1, and R_2 errors between those. Is my understanding correct? If yes, what is this supposed to show?
Reviewer_fogGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether pointwise prediction accuracy is sufficient to evaluate models on structured combinatorial landscapes. The authors introduce relational generalization, which measures how well a model preserves first-order mutation effects and second-order interactions, rather than only individual fitness values. They provide theoretical analysis connecting relational errors to mutation-sign recovery and local landscape structure. Experiments are conducted on four RNA fitness landscapes using seven model families. The authors compare models with similar prediction errors, perform controlled residual normalization, and evaluate differences in relational accuracy and reconstructed landscape structure. They also investigate relation-aware training objectives and their potential benefits for downstream sequence search.

Critical strengths
  • The paper highlights a meaningful limitation of conventional prediction metrics. For structured prediction problems, accurately predicting individual values does not necessarily imply accurate recovery of local differences or interactions. The distinction between pointwise, first-order, and second-order errors is clearly motivated and may be useful for evaluating models in scientific applications.

  • The authors evaluate multiple RNA landscapes and model families, combining theoretical analysis with empirical comparisons and controlled constructions. In particular, the matched-error experiments demonstrate that different residual patterns can produce different relational errors even when their pointwise errors are identical. The paper also discusses negative findings and limitations, which I appreciate.

Critical weaknesses
  1. The novelty and significance of the main contribution are somewhat limited. The central observation that similar prediction errors may correspond to different relational structures is not particularly surprising. In general, the magnitude of prediction errors does not fully characterize their spatial or structural distribution. The proposed first- and second-order metrics are closely related to finite-difference and graph-based error analysis.

  2. The experimental evidence is not strong enough to establish practical significance. The most convincing results come from the controlled matched-error experiments, where residuals are rescaled after training to produce identical pointwise errors. While this demonstrates the proposed phenomenon, it does not establish how frequently or significantly such differences arise between independently trained models. In fact, the natural matched-risk comparison in Section 5.1 fails to demonstrate the hypothesized large relational separation. More importantly, the practical benefits remain unclear.

Questions for authors
  • Novelty: What is the main conceptual or theoretical advantage of the proposed relational generalization framework compared with existing finite-difference, gradient-matching, or graph-based structural analyses?

  • Natural model comparisons: The natural matched-risk experiments show limited separation, whereas the main positive results depend on post-hoc residual normalization. Can the authors provide stronger evidence that substantial relational differences occur naturally among independently trained models with comparable pointwise accuracy?

  • Practical benefits: Can the authors demonstrate a setting where optimizing the proposed relational metrics consistently improves downstream performance while maintaining comparable pointwise prediction accuracy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.