acceptodds
Under review as a conference paper at ICLR 2027

Where Does Protein Structure Emerge? Layer-Wise Native 3D Organization Across Protein and General Language Models

Abstract

Protein language models (PLMs) are trained primarily from amino-acid sequences, yet their representations support tasks closely tied to protein structure. We ask where, at what scale, and how consistently native three-dimensional organization becomes visible across network depth. We study dedicated PLMs spanning Transformer, convolutional, and state-space architectures, together with general-purpose language models and protein-oriented checkpoints. We analyze structure from two complementary perspectives: how the geometry of whole-protein representations changes across layers, and whether residues that are nearby in native 3D space remain nearby in representation space. We find strongly depth-dependent and model-specific organization. Whole-protein shape complexity often compresses across depth, but its trajectory varies across model families. In dedicated PLMs, Local Topological Fidelity (LTF) consistently peaks before the final layer, whereas general-purpose LLMs show more heterogeneous behavior. Secondary-structure conditioning further reveals that intermediate-scale neighborhoods (–) centered on helical residues exhibit the strongest fidelity. Finally, LTF enables downstream-label-free, model-adaptive layer selection, with particularly strong utility on structure-sensitive evaluations. Across ten PLMs and four evaluations, it is the only downstream-label-free selector in the statistically top-ranked group alongside supervised layer-selection methods. The selected region remains stable across structural reference sets, supporting reuse without task-specific layer search.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Protein Language Models, Large Language Models, Protein Structure, Representation Geometry, Layer-wise Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rmYAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper analyzes the emergence of structure in protein and textual language models. Specifically, the authors look at the intrinsic manifold dimensionality (IMD, a measure of dimensionality of the embedding through layers) and local topilogical fidelity (LTF, a measure of embedding similarity in a given neighbourhood) as a function of the layer index. As with previous works in this field, the authors find that protein language models broadly expand and then compress the information in the embeddings. They also show that there are dinstinct 3D structure patterns in the 2 and 8 nearest neighbours, particularly for alpha helices. Finally, they show that IMD can be used in an unsupervised way to select the best layer for training supervised structural models.

Critical strengths

The authors expand the knowledge base for understanding emergence of structure in protein language models. They test the method on a large number of models and introduce two new variants of dimensionality measurement. They resolve a small open question from (Beshkov & Malthe-Sørenssen) about the peak in features near 8 nearest neighbours, being related to alpha helices. An unsupervised method to select the best embedding layer for downstream training is a potentially useful contribution to the field, and I appreciated the comparisons to different selection strategies.

Critical weaknesses

As acknowledged by the authors, the emergence of structure in LLMs is a well-studied phenomenon, and this paper provides limited additional insight. IMD is very similar to intrinsic dimensionality as explored in (Valeriani et al.) and LTF is very similar to the graph filtration moment (GFM) introduced in (Beshkov & Malthe-Sørenssen). Just because 3D coordinates get an average IMD of 3.88, doesn't mean that a value near 3.88 correponds to an internal representation of 3D structure. Additionally, the two LTF peaks near k=2 and k=8 can be trivially explained as the adjacent residues and the 4 nearest residues in a helix. The unsupervised selection method is interesting to me, but the tests in the main text mostly directly measure structural awareness, which the method claims to be directly selecting for anyway. This undercuts the "unsupervised" claim. I would like to see expanded testing of other labels such as EC and DeepLoc compared to strong baselines like the second last layer. Also, a small point, but the figures and labels/captions are very small and hard to read without massively zooming in.

Questions for authors

Have you checked if a simple model like sinusoidal Transformer positional encoding reproduces the LTF results? I am curious if this fully explains the k=8 peak.

Is there some more direct way to test if the IMD near 3.88 actually corresponds to improved structural understanding? Maybe reframing the layer selection for measuring contacts as an explicit test for this?

There seems to be a mismatch between Table 10 and Table 17 regarding worst regret for the final layer. I may be misunderstanding something, but can you explain why the average is greater than 1 while the individual values are all less than 1?

Do you plan to release public code to verify these results?

Does LTF fundamentally measure something different from GFM?

3 comments
AuthorsOpenReview ↗

R1 (Q2). Physical-IMD Proximity versus Native Structural Correspondence

ModelFinal LayerIMD-SelectedLTF-Selected
ESM2-35M9.809.609.20
ESM2-150M8.908.608.10
ESM2-650M7.978.757.29
Ankh Base8.608.408.00
Ankh Large8.208.107.60
ProtGPT29.409.309.00
CARP-38M10.1010.009.60
CARP-76M9.509.408.90
CARP-640M8.808.708.20
ProtMamba9.709.609.30
Panel Mean9.109.048.52

LTF-based selection achieves lower contact-distance MAE than IMD-based selection across all ten PLMs, reducing the panel mean from 9.04 to 8.52 Å. IMD selection also improves modestly over the final layer in nine of ten models, suggesting some practical utility as a layer-selection heuristic, but not establishing that proximity to the physical IMD reference directly measures native structural correspondence.

A particularly informative case is ESM2-650M, where the different structural metrics do not favor the same representation.

Table rmYA-R1-A2. ESM2-650M structural evaluation at selected layers. Mean LTF is computed on SCOPe over k = 1–19. CATH distance MAE evaluates residue pairs with |i−j| > 6, while long-range contact P@L uses |i−j| > 24.

RepresentationLayerMean LTF ↑Distance MAE (Å) ↓Long-Range P@L ↑
IMD-Selected232.608.750.52
LTF-Selected253.127.290.51
Final Layer331.487.970.50

The IMD-selected layer (L23) yields substantially higher distance error than both the LTF-selected layer (L25) and the final layer (L33), despite being closest to the physical IMD reference. Its MAE of 8.75 Å exceeds the corresponding values of 7.29 Å and 7.97 Å, respectively.

At the same time, IMD selection achieves a slightly higher long-range P@L (0.52) than LTF selection (0.51). This small descriptive difference illustrates that representation utility depends on the particular structural endpoint; it does not establish a statistically reliable advantage for either selector in long-range contact precision.

Taken together, these results demonstrate that matching the physical IMD reference does not guarantee superior native structural prediction. IMD proximity can identify useful intermediate representations, but it does not consistently identify the layers that minimize distance-prediction error. Conversely, LTF-based selection performs better on contact-distance MAE across the evaluated panel without implying superiority on every structural metric.

Importantly, the selected-layer comparison tests the practical utility of the two selection criteria, but does not establish how IMD proximity relates to structural performance throughout the complete representation hierarchy.

We therefore extend this analysis across every hidden layer of all ten PLMs, directly testing whether proximity to IMD ≈ 3.88 tracks contact-distance MAE and sequence-nonlocal native-neighbor retrieval, and comparing these associations with those obtained from LTF.

3. Does Physical-IMD Proximity Predict Structural Performance Across Depth?

To directly test whether proximity to the physical IMD reference predicts structural utility, we extend the selected-layer comparison to every hidden layer across all ten PLMs. This analysis evaluates the relationship between global spectral dimensionality, native-neighborhood correspondence, and downstream structural prediction without restricting the comparison to layers chosen by either selector.

For model m at layer ℓ, we define physical-IMD proximity as:

D(m, ℓ) = |IMD(m, ℓ) − 3.88|

A smaller D indicates closer agreement with the physical IMD reference. If this proximity reliably reflects structural utility, D should correlate positively with contact-distance MAE (lower is better) and negatively with nonlocal native-neighbor overlap (higher is better). Conversely, higher LTF is expected to associate with lower MAE and greater native-neighbor overlap.

3.1. Full-Depth Evaluation Protocol

We compute layer-wise associations using the original structural calibration set and the same downstream evaluation protocol as the manuscript.

Table rmYA-R1-0. Protocol for the full-depth structural correspondence analysis.

AuthorsOpenReview ↗

R1 (Q2). Physical-IMD Proximity versus Native Structural Correspondence

ComponentEvaluation Protocol
Model coverageAll ten PLMs; every hidden layer, excluding the initial token-embedding layer
Structural calibrationOriginal 1,098-domain SCOPe reference for layer-wise IMD and mean LTF
Downstream evaluationOriginal CATH test set of 75 protein domains
Contact-distance MAEFrozen, layer-specific representations evaluated using the shared contact probe; results averaged over seeds 42, 2024, and 7
Nonlocal Overlap@8Fraction of the eight nearest eligible native Cα neighbors recovered in representation space, excluding
Score aggregationDistance MAE is pair-weighted within each seed; Overlap@8 is averaged within proteins and then equally across eligible proteins
Statistical analysisSpearman correlation across hidden layers within each model; model-wise medians summarize the ten-model panel

Importantly, the correlations are computed separately within each PLM, rather than pooling layers from architectures with different depths and representation scales. The model-wise median then gives equal weight to each backbone, regardless of its number of hidden layers.

IMD and LTF are evaluated using the SCOPe structural reference, while contact-distance prediction and nonlocal neighbor retrieval are evaluated on CATH. The latter provides a direct, probe-free geometric diagnostic alongside the supervised structural prediction results.

3.2. Physical-IMD Proximity Shows Weak and Inconsistent Structural Associations

Table rmYA-R1-B. Full-depth Spearman correlations between geometric indicators and structural performance. D denotes |IMD − 3.88|. Lower MAE and higher nonlocal Overlap@8 indicate better structural performance. Each correlation is computed across the hidden layers of the corresponding model.

ModelLayersρ(D, MAE)ρ(D, Nonlocal Overlap)ρ(LTF, MAE)ρ(LTF, Nonlocal Overlap)
ESM2-35M12+0.18−0.14−0.52+0.61
ESM2-150M30+0.09−0.08−0.68+0.75
ESM2-650M33+0.12−0.06−0.74+0.81
Ankh Base48−0.07+0.03−0.59+0.70
Ankh Large48+0.04−0.02−0.65+0.78
ProtGPT236+0.29−0.22−0.47+0.57
CARP-38M16−0.16+0.12−0.43+0.52
CARP-76M33+0.06−0.10−0.62+0.72
CARP-640M56+0.21−0.17−0.71+0.80
ProtMamba16−0.09+0.07−0.56+0.65
Model-wise Median—+0.075−0.070−0.605+0.710
Physical-IMD proximity exhibits weak and inconsistent associations with structural performance across the ten PLMs. Its median correlation with contact-distance MAE is only +0.075, while its median correlation with nonlocal Overlap@8 is −0.070. The associations also change sign across architectures, indicating that closer agreement with the physical IMD reference does not consistently identify layers with better native structural correspondence or downstream distance prediction.

Some models nevertheless show modest associations in the expected directions, including ProtGPT2 (ρ = +0.29 with MAE) and CARP-640M (ρ = +0.21). Thus, the results do not suggest that global dimensionality is entirely unrelated to structural utility. Rather, proximity to the particular physical reference value of 3.88 is not a reliable monotonic predictor of structural performance across depth.

In contrast, LTF exhibits substantially stronger and directionally consistent associations across all ten models. Its correlations with contact-distance MAE range from −0.74 to −0.43 (median −0.605), while correlations with nonlocal Overlap@8 range from +0.52 to +0.81 (median +0.710).

These results show that layers with stronger native-neighborhood fidelity generally support more accurate distance prediction and recover more sequence-distant native neighbors. The association with supervised contact-distance MAE is particularly informative because it evaluates the utility of frozen representations through a downstream predictive task, rather than solely comparing alternative neighborhood statistics.

AuthorsOpenReview ↗

R1 (Q2). Physical-IMD Proximity versus Native Structural Correspondence

5. Scientific Interpretation and Relation to Prior Work

Together, our results distinguish global shape-spectrum organization from the preservation and downstream accessibility of native structural relationships.

IMD measures the effective dimensionality of variation across protein representation shapes. Its layer-wise evolution provides a useful description of geometric expansion and compression, but proximity to the physical reference of 3.88 does not establish whether particular native residue relationships are preserved.

This distinction is supported by two direct observations. First, physical-IMD proximity exhibits weak and inconsistent associations with structural performance across all ten PLMs, whereas LTF shows substantially stronger and directionally consistent associations with both contact-distance MAE and nonlocal native-neighbor retrieval (Table rmYA-R1-B). Second, layers with similar IMD proximity can retain different levels of structural fidelity and predictive utility, as demonstrated by the matched-proximity comparisons (Table rmYA-R1-C).

These findings are consistent with prior studies examining the relationship between representation dimensionality and model behavior. Ansuini et al. [1] investigate intrinsic dimensionality in deep neural networks, while Valeriani et al. [2] analyze dimensionality patterns in Transformer representations. Their work motivates the study of geometric reorganization, but does not imply that matching a particular physical shape-spectrum statistic guarantees residue-level structural correspondence. Moreover, our SRV tangent-covariance participation ratio differs from the TwoNN-based intrinsic-dimensionality estimator used by Valeriani et al.

Beshkov and Malthe-Sørenssen [3] further connect protein representation geometry with SRV-based shape analysis and neighborhood-graph measurements. Our additional experiments clarify the empirical relationship between these levels of analysis: global dimensionality describes representation-shape variation, whereas native-neighborhood fidelity provides a more direct indicator of residue-level structural correspondence and its association with downstream structural prediction.

We do not claim that IMD is unrelated to structural information, that LTF causally determines predictive performance, or that LTF is fundamentally distinct from closely related neighborhood-based measures such as GFM. Rather, the present experiments establish why matching global dimensionality is insufficient as a standalone test of native structural understanding, and why direct structural correspondence must be evaluated separately.

Overall Response to Q2

Our results directly address the reviewer's proposed test: an IMD value near the physical reference of 3.88 does not, by itself, establish improved native structural understanding.

The existing ten-model contact evaluation shows that LTF-based selection achieves lower mean distance MAE than physical-IMD-based selection (8.52 versus 9.04 Å). Our additional full-depth analysis further demonstrates that physical-IMD proximity has weak and inconsistent associations with structural performance (median Spearman ρ = +0.075 for MAE and −0.070 for nonlocal overlap), whereas LTF shows substantially stronger and directionally consistent associations (−0.605 and +0.710, respectively).

The matched-proximity analysis provides complementary evidence: even among layers with closely matched distances to the physical IMD reference, native-neighborhood fidelity can distinguish structural correspondence and, in three of four evaluated models, is associated with better contact-distance prediction.

The resulting scientific distinction is clear: IMD characterizes global geometric organization, LTF measures native residue-neighborhood correspondence, and supervised contact prediction evaluates the accessibility of structural information for a downstream task. These properties are related but not interchangeable.

We will incorporate the full-depth and matched-proximity analyses into the revised manuscript and clarify that physical-IMD proximity is a descriptive comparison criterion, not a sufficient indicator of native structural understanding.

References

[1] Ansuini, A., et al. (2019). Intrinsic Dimension of Data Representations in Deep Neural Networks. Advances in Neural Information Processing Systems, 32.

[2] Valeriani, L., et al. (2023). The Geometry of Hidden Representations of Large Transformer Models. Advances in Neural Information Processing Systems, 36.

[3] Beshkov, K., and Malthe-Sørenssen, A. (2026). Towards Understanding the Shape of Representations in Protein Language Models. International Conference on Learning Representations.

Reviewer_4G3mOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how native protein structure is reflected across language-model layers using global geometric analysis and Local Topological Fidelity (LTF). It uses LTF to select layers without downstream task labels and evaluates the selected representations on contact/distance prediction, fold classification, and EC classification. The main finding is that pre-final layers often provide stronger structural correspondence and better performance on structure-sensitive tasks.

Critical strengths
  1. Broad empirical coverage. The study spans several model architectures and includes substantial robustness checks. This makes the observed preference for pre-final representations more convincing than a single-model result.
  2. A useful layer-selection application. LTF offers a concrete use for structural analysis without requiring downstream labels. Its lower worst-case regret is encouraging, and the negative DeepLoc results help establish its limitations.
Critical weaknesses
  1. Substantial overlap with existing geometric analysis. SRV-based shape analysis and graph filtrations [1] already revealed stronger structural fidelity before the final layer. The broader experiments are useful, but the scientific question remains shallow and the practical contribution to the community is unclear. The study emphasizes expanding experiments more than developing new understanding.
  2. Small gains over simple alternatives. LTF achieves 8.52 Å contact MAE, compared with 8.62 Å for kNN-overlap, 8.56 Å without null normalization, and 8.61 Å for the best reported fixed-depth rule. Better worst-case regret is encouraging, but the added complexity needs stronger justification.
  3. Structural correspondence needs deeper biological analysis. Most analyses concern generic neighborhoods and secondary-structure classes. Binding sites or antibody CDR/interface regions, using resources such as BioLiP or SAbDab, would offer a more meaningful biological test. The current evidence describes structural correspondence more than it explains emergence.
  4. Key improvements lack uncertainty estimates. Key results lack consistent error bars and direct paired comparisons. Aggregate ranking tests do not establish whether the small gains over the closest alternatives are reliable, especially given related models and correlated EC endpoints.
  5. Structural generalization remains narrowly tested. Direct structural prediction relies mainly on a small CATH subset, while SCOPe is used for both reference selection and fold classification. Cross-resource homology controls need clarification. EC gains are indirect evidence of structural utility, and DeepLoc results show limited transfer.
  6. Missing code makes the results difficult to verify. Code and exact experimental artifacts are unavailable during review. The hyperparameter-selection protocol also leaves unclear whether selected and final layers receive an equally favorable comparison.

[1] Kosio Beshkov, Anders Malthe-Sørenssen. Towards Understanding the Shape of Representations in Protein Language Models, ICLR, 2026.

Questions for authors
  1. Are the gains over kNN-overlap, the no-null variant, and fixed-depth selection significant across seeds? Please provide per-seed results or paired uncertainty estimates.
  2. Does the helix-associated peak around (k ≈\approx 6–9) persist after removing sequence-local neighbors?
  3. How were cross-resource overlap and homology controlled? Which layer representations were used to select probe hyperparameters, and can code and exact splits be provided?
  4. Could you plz provide the code and raw data?
5 comments
AuthorsOpenReview ↗

R2 (W2/W4/Q1). Statistical Reliability and Practical Value of Full LTF

2. Direct Paired Evidence for Statistical Reliability

To directly address the reviewer's question regarding the reliability of the observed performance gains, we conduct paired protein-level bootstrap analysis and report complete per-seed comparisons across all ten PLMs. These analyses distinguish statistically supported improvements over simpler layer-selection strategies from the smaller differences among closely related multi-scale neighborhood criteria.

2.1. Paired Bootstrap Confidence Intervals

We perform 10,000 paired bootstrap resamples over the 75 CATH test proteins. Each resample uses the same sampled protein indices across all selectors, models, and probe seeds, preserving within-protein residue-pair dependencies and ensuring a direct paired comparison.

For each resample, we recompute the pair-weighted contact MAE, average across the three probe seeds within each model, and then average across the ten models. The paired improvement is defined as the comparator's MAE minus Full LTF's MAE, with positive values favoring Full LTF.

We report both pointwise 95% confidence intervals and Bonferroni-adjusted 99% confidence intervals for the five prespecified comparisons. The latter account for multiple comparisons at a family-wise confidence level of at least 95%.

Table 4G-R2-B. Paired contact MAE improvements across ten PLMs (Å, higher gain is better). Confidence intervals are obtained from 10,000 paired protein-level bootstrap resamples. Positive gains favor Full LTF. The 99% intervals apply Bonferroni correction across five comparisons.

ComparatorComparator MAE ↓Full LTF MAE ↓Paired Gain ↑95% CIAdjusted 99% CI
kNN-overlap@88.62108.5190+0.1020[+0.028, +0.173][+0.003, +0.194]
LTF without null8.56008.5190+0.0410[−0.015, +0.093][−0.033, +0.107]
Fixed 75%8.61008.5190+0.0910[+0.017, +0.161][−0.009, +0.179]
Fixed 80%8.64608.5190+0.1270[+0.074, +0.177][+0.056, +0.192]
GFM-M/mean8.50308.5190−0.0160[−0.046, +0.013][−0.055, +0.023]

The paired analysis establishes statistically supported improvements over single-scale overlap and selected fixed-depth heuristics. Full LTF improves mean contact MAE over kNN-overlap@8 by 0.1020 Å and Fixed-80% by 0.1270 Å, with both gains remaining positive after Bonferroni correction.

Against the stronger Fixed-75% heuristic, Full LTF achieves a 0.0910 Å mean improvement, supported by the pointwise 95% CI of [+0.017, +0.161]. The adjusted 99% CI includes zero, indicating that this comparison does not meet the more conservative multiple-comparison criterion.

In contrast, the improvements over LTF without null normalization (+0.0410 Å) and the difference relative to matched GFM-M/mean (−0.0160 Å) are not statistically distinguishable from zero under the reported bootstrap intervals.

These results clarify the contribution of the complete LTF construction. Its advantages over simpler neighborhood and fixed-depth selection strategies are supported by paired evidence, while the closely related no-null and matched GFM formulations exhibit comparable average predictive performance. This distinction is consistent with our direct GFM analysis in the response to Reviewer Z41H: native-neighborhood preservation provides the principal selection signal, whereas specific normalization and aggregation choices produce more modest differences.

2.2. Per-Seed Consistency and Model-Level Outcomes

We additionally examine whether the observed improvements are consistent across the three probe-training seeds rather than being driven by a single favorable run.

Table 4G-R2-C. Per-seed paired contact MAE gains and model-wise comparisons (Å). Positive gains favor Full LTF. Mean ± SD summarizes the three panel-level seed gains. Wins/ties/losses (W/T/L) are counted from Full LTF's perspective using seed-averaged model performance; ties denote identical selected-layer runs.

ComparatorSeed 42Seed 2024Seed 7Mean ± Seed SDLTF W/T/L
kNN-overlap@8+0.0779+0.1202+0.1079+0.1020 ± 0.02188 / 2 / 0
LTF without null+0.0465+0.0382+0.0383+0.0410 ± 0.00486 / 3 / 1
Fixed 75%+0.1026+0.0824+0.0880+0.0910 ± 0.01047 / 1 / 2
Fixed 80%+0.1555+0.1033+0.1222+0.1270 ± 0.02646 / 3 / 1
GFM-M/mean−0.0317−0.0156−0.0006−0.0160 ± 0.01551 / 7 / 2

The direction of the average improvement is consistent across all three seeds for four of the five comparisons. Full LTF achieves positive panel-level paired gains against kNN-overlap@8, LTF without null, Fixed-75%, and Fixed-80% at every evaluated seed.

AuthorsOpenReview ↗

R4 (W5/Q3). Structural Generalization and Cross-Resource Homology Controls

  • CATH: 65 of the original 75 test domains.
  • ECPred: 96 of the original 120 test proteins.

Importantly, the original training and validation partitions, fitted probes, hyperparameters, and random seeds remain unchanged. We do not retrain predictors, retune hyperparameters, or select layers using performance on the restricted test subsets.

Every selector is evaluated on the same retained examples within each benchmark. Where different selectors choose the same model layer, they reuse the identical fitted probe and its corresponding predictions, preserving the paired comparison protocol established in our response to W2/W4/Q1.

These restricted cohorts provide a test-partition sensitivity analysis rather than a newly constructed, fully homology-disjoint benchmark. Specifically, they contain no detected test-to-training or test-to-validation matches satisfying the stated identity, coverage, and E-value criteria. They do not establish the absence of shorter local similarities, remote evolutionary relationships, or potential pretraining exposure. The original training–validation relationships also remain unchanged.

We retain the complete original test sets as the primary benchmark evaluations and report the restricted-cohort results separately. This distinction allows us to examine whether performance differences persist after excluding the identified related test proteins without changing the underlying supervised learning protocol.

Together with the SCOPe reference-exclusion analysis in Section 2, this provides two complementary controls: reference filtering tests sensitivity of structural layer selection to cross-resource overlap, while test-cohort screening tests whether downstream performance depends on detectable sequence similarity to training or validation proteins.

We next evaluate the combined effects of these controls on selected layers, CATH contact prediction, and fine-grained ECPred classification.

4. Robustness of Layer Selection and Downstream Performance Under Homology Controls

The central question is whether removing detectable cross-resource and cross-partition homology materially changes the selected representation layers or their downstream utility. We examine this through three complementary evaluations: layer-selection stability under the filtered SCOPe reference, contact-distance prediction on full and homology-screened CATH test cohorts, and fine-grained enzyme classification under the corresponding ECPred controls.

4.1. Layer-Selection Stability After Reference Filtering

We recompute Full LTF and matched GFM-M/mean using the common 837-domain SCOPe reference, obtained by excluding the 261 domains with qualifying matches to CATH or ECPred. All neighborhood construction, null-reference settings, and scale-aggregation procedures remain unchanged. Importantly, downstream labels and prediction performance are not used to determine the filtered selections.

To isolate the effect of structural-reference filtering, we evaluate each selected layer using the same frozen-backbone representations and corresponding cached probe runs. If the filtered reference selects an unchanged layer, its downstream predictions remain identical to those obtained with the original reference.

Table 4G-R4-E. Layer-selection stability and CATH contact MAE after SCOPe reference filtering. Original selections use 1,098 SCOPe domains; filtered selections use 837 domains. Contact MAE (Å, lower is better) is evaluated on the unchanged 75-domain CATH test set.

ModelFull LTF: Original → FilteredGFM-M/mean: Original → FilteredOriginal LTF MAEFiltered LTF MAE
ESM2-35M9 → 99 → 99.2009.200
ESM2-150M24 → 2324 → 238.1008.200
ESM2-650M25 → 2525 → 257.2907.290
Ankh Base39 → 3939 → 398.0008.000
Ankh Large38 → 3738 → 377.6007.670
ProtGPT229 → 2928 → 289.0009.000
CARP-38M15 → 1515 → 159.6009.600
CARP-76M23 → 2323 → 238.9008.900
CARP-640M40 → 4041 → 418.2008.200
ProtMamba11 → 1110 → 109.3009.300
Panel Summary8/10 unchanged8/10 unchanged8.51908.5360

The selected layers remain unchanged in eight of ten PLMs for both Full LTF and GFM-M/mean. Only ESM2-150M and Ankh Large shift, each by one layer toward earlier representations. The mean absolute layer shift is therefore 0.20 layers across the ten-model panel.

Crucially, these limited shifts produce only a small change in downstream contact prediction. Full LTF's ten-model mean MAE increases from 8.5190 to 8.5360 Å, a difference of 0.0170 Å. Both selectors also continue to identify identical layers in seven of the ten models, consistent with their shared native-neighborhood selection signal.

AuthorsOpenReview ↗

R4 (W5/Q3). Structural Generalization and Cross-Resource Homology Controls

Table 4G-R4-H. Evidence supporting structural and functional generalization, with explicit applicability boundaries. Results retain their respective reference configurations and evaluation panels; they are not pooled into a single generalization metric.

EvaluationScope and Key ResultSupported ConclusionApplicability Boundary
CATH contact prediction75 test domains; 65 homology-screened domains; ten PLMs and three probe seedsDirect residue-distance prediction and sensitivity to reference-overlap and test-cohort screeningDoes not establish performance on all unseen folds or de novo structure prediction
BioLiP binding-site analysis240 nonredundant chains; 3,780 matched anchor pairs; selected-layer site LTF gains of +0.17 to +0.43 over FinalNonlocal intrachain structural correspondence around annotated small-molecule binding regionsNot direct ligand-contact, binding-affinity, or biochemical-activity prediction
ProteinGym mutation-effect prediction15 assays; 898-domain filtered reference; five-model mean Spearman: 0.3990 (Final) → 0.4191 (LTF)Transfer of structural layer selection to mutation-effect scoring with held-out mutation positionsWithin-assay position generalization; not protein-disjoint or full-ProteinGym evaluation
ECPred fine-grained classificationThree-model EC-L4 accuracy: 0.4467 (Final) → 0.4667 (LTF), original referenceFunctional representation utility beyond direct geometric targetsDoes not establish that classification gains are caused solely by native-distance preservation
PPB-Affinity prediction8,079 affinity records in 1,528 interaction groups; three-model mean Spearman: 0.3413 (Final) → 0.4037 (LTF)Transfer from monomer structural calibration to supervised intermolecular affinity predictionInteraction-group separation does not guarantee full chain-disjoint or pretraining-disjoint evaluation
DeepLoc localizationThree-model macro-F1: 0.670 (Final) versus approximately 0.646 (LTF)Negative control demonstrating task dependenceDoes not support universal superiority of structurally selected layers

These evaluations support complementary, not interchangeable, conclusions.

CATH directly tests the usefulness of selected representations for native residue-distance prediction, while the BioLiP analysis examines nonlocal structural correspondence around biologically annotated ligand-binding residues. ProteinGym, ECPred, and PPB-Affinity assess downstream targets that are not themselves native residue-distance matrices.

The ECPred results further illustrate the importance of distinguishing evaluation configurations. The three-model EC-L4 accuracy of 0.4467 → 0.4667 refers to the full 120-protein test set with the original SCOPe reference. Under the 837-domain filtered reference, mean LTF accuracy on that same test set is 0.4633. On the 96-protein homology-screened subset, the corresponding Final and filtered-reference LTF accuracies are 0.4267 and 0.4433, respectively (Section 4.3).

These comparisons retain positive average differences while preserving the model-specific exceptions documented in Section 4.

Importantly, the DeepLoc result remains an explicit limitation: structurally selected intermediate layers are not universally preferable for downstream tasks. The practical benefit of native-neighborhood-based selection depends on whether the target prediction benefits from the structural relationships retained in those representations.

Thus, our generalization claim is specific: the observed structural and selected functional advantages persist under the evaluated overlap controls, while their magnitude and applicability remain dependent on the model, task, and reference configuration.

5.3. Distinguishing Structural Reference Configurations

To ensure consistency across our manuscript and reviewer responses, we explicitly distinguish the structural reference configurations used for different evaluation resources.

Table 4G-R4-I. SCOPe reference configurations and their corresponding overlap controls. Each configuration serves a specified evaluation purpose. Different target collections yield different exclusion sets.

Reviewer_Z41HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how protein structure is reflected across the layers of protein language models (PLMs) and general-purpose LLMs. It uses two measures: IMD, a dimensionality score for whole-protein representation shapes, and LTF, which scores whether residues that are neighbors in 3D are also neighbors in the embedding. It describes how these vary across models, scales, training stages, and secondary-structure types, then proposes using LTF to select a layer without downstream labels.

The paper contains a large amount of analysis, but it adds up to observations rather than a clear insight. The descriptive part largely extends Beshkov & Malthe-Sørenssen (2025) using standard measures under new names. The LLM and secondary-structure comparisons don't lead to conclusions. The layer selector mostly confirms that a layer around 75% depth works well on tasks that are themselves about 3D structure.

Critical strengths

The model panel is broad, the robustness appendix is extensive, and the authors do report a case where their selector does worse.

Critical weaknesses
  1. After reading the paper I could not identify its takeaway, because most of the findings report the absence of a pattern: geometric compression recurs across models; the preferred layer depth is not consistent, and post-training redistributes structural fidelity across depth without increasing it. Most of this is exploratory observations without a clear explanation
  2. Including Llama and Qwen raises the question of what training only on protein sequences buys (see question below)
  3. The comparison is confounded in two further ways, by scale (7-8B LLMs against PLMs up to 1.5B) and by input format (space-separated amino acids). Because the LLMs show the same global compression as the PLMs, that result undercuts IMD as a structural measure, rather than saying much about LLMs vs PLMs.
  4. The secondary-structure analysis is similarly unclear and is again descriptive of variations without a takeaway.
  5. Much of the descriptive core follows Beshkov & Malthe-Sørenssen (2025), using the same square-root velocity curves and effective dimension across layers, and reaching the same conclusion that structural fidelity peaks just before the final layer.
  6. Both measures essentially boil down to well-established metrics: IMD being the participation ratio of the nonzero spectrum and LTF reducing to shared k-nearest-neighbor overlap scored against a random baseline. For the latter, plain k-NN overlap at k = 8 reaches 8.62 Å contact MAE against 8.52 Å for full LTF. This suggests the measures don't buy much over the standard metrics.
  7. The layer selector is evaluated almost circularly, since LTF picks the layer whose neighbors best match 3D structure and the headline task is predicting 3D residue distances. To their credit, the authors do show that on DeepLoc, which is not defined by structure, the final layer beats the selected layer, and a fixed rule of 75% depth comes within 0.1 Å of LTF.
Questions for authors
  • What is the main takeaway of the descriptive analysis beyond "the patterns are model-dependent"?
  • What should a reader conclude from the LLM comparison: are dedicated PLMs better at capturing protein structure, equivalent, or replaceable by general models? Can you support this with downstream results for the LLMs?
  • What is the takeaway of the secondary-structure analysis, and can you quantify the helix/strand/coil differences?
  • What does the paper add beyond Beshkov & Malthe-Sørenssen (2025)?
  • Does LTF selection help on tasks that are not themselves defined by 3D structure?
4 comments
AuthorsOpenReview ↗

#R4 (W5/W6/Q4). Scientific Contributions and Practical Value Beyond Prior Geometric Analyses

We thank the reviewer for raising this important question. Our contribution extends beyond identifying pre-final structural fidelity or introducing alternative geometric metrics. We advance the understanding of native structural organization by explaining its depth-dependent behavior, identifying its biological and model-specific characteristics, and establishing its practical value for downstream-label-free representation selection.

Building on Beshkov and Malthe-Sørenssen [1], we provide new explanatory diagnostics, quantitative secondary-structure analyses, comparisons between protein-specialized and general-purpose models, and systematic downstream evaluations. Importantly, our additional GFM comparisons further distinguish the shared geometric foundations from the scientific and operational contributions of our study.

1. Scientific Advances Beyond Prior Geometric Analysis

Beshkov and Malthe-Sørenssen [1] establish an important foundation through SRV-based shape analysis, graph-filtration-based structural correspondence, and the observation that native structural fidelity often peaks before the final layer. They also identify an intermediate-neighborhood feature and suggest that structurally favorable representations may benefit downstream prediction.

Our work advances these observations by addressing four questions: why structural fidelity can diverge from prediction-oriented accessibility, how structural correspondence differs between protein-specialized and general-purpose models, which residue environments underlie the intermediate-scale feature, and when structural fidelity provides a useful representation-selection signal.

Table R4-A. Scientific advances beyond the prior geometric analysis. The comparison distinguishes previously established observations from the additional experimental evidence provided in our study.

Scientific QuestionPrior Work [1]Our Additional Evidence and Findings
Why can structural fidelity peak before the final layer?Identifies pre-final structural fidelity and discusses its possible relationship to prediction-oriented refinement.R1: Joint layer-wise diagnostics across three PLMs reveal diverging trajectories between native structural fidelity and masked-residue predictability. For ESM2-650M, selected-to-final nonlocal LTF decreases from 1.60 to 1.19 and distance MAE increases from 7.29 to 7.97 Å, while masked-residue NLL improves from 1.97 to 1.86. These results support an objective-related representation-reorganization explanation.
What distinguishes protein-specialized and general-purpose representations?Primarily investigates structural organization in dedicated PLMs, including ESM2 and Ankh.R2: Cross-model analyses and downstream evaluations distinguish generic geometric compression from nonlocal native structural correspondence. Despite Llama-3.1's higher unfiltered LTF peak, its maximum nonlocal LTF is 1.22, compared with 1.62 for ESM2-650M. CATH and ECPred evaluations further establish differences in structural and functional representation utility.
Which structural environments produce the intermediate-scale feature?Reports an intermediate-neighborhood feature near k = 8 and suggests a possible association with helical structure based on whole-protein classes.R3: Residue-level SS3 conditioning reveals a consistent Helix > Coil > Strand ordering across nine PLMs at k = 6–9. Additional locality controls show that excluding sequence separations of four or fewer residues reduces helix band prominence from 0.66–1.02 to 0.04–0.06 across four representative PLMs, identifying its strong sequence-local dependence.
When can structural correspondence guide representation selection?Motivates the potential use of structurally favorable intermediate representations for downstream structure prediction.Section 4 and Appendices E–F: We operationalize native-neighborhood fidelity as a downstream-label-free layer-selection criterion, evaluated across ten PLMs and four downstream evaluations. Comparisons with supervised layer optimization, fixed-depth heuristics, and geometric alternatives establish its practical utility and task-dependent limits, including the negative DeepLoc results. The additional direct GFM comparison further evaluates the contribution of the specific score formulation.

These additions establish a progression from geometric observation to scientific interpretation and practical validation.

First, the R1 diagnostics provide direct evidence that native structural fidelity and masked-residue predictability can evolve in different directions across network depth. This extends the earlier pre-final fidelity observation by examining how structural correspondence relates to prediction-oriented representation accessibility.

AuthorsOpenReview ↗

R4 (W5/W6/Q4). Scientific Contributions and Practical Value Beyond Prior Geometric Analyses

We directly test these distinctions below instead of assuming that mathematical differences necessarily produce superior prediction performance.

3. Direct GFM Comparison Under a Unified Evaluation Protocol

To determine whether Full LTF provides additional practical value beyond established graph-based measures, we conduct a direct comparison with the published GFM formulation [1] and simpler neighborhood-selection criteria.

All structural selectors use the same SCOPe reference set, residue-aligned frozen representations, candidate layers, and neighborhood range k = 1–19. Downstream evaluation follows the manuscript's CATH, SCOPe, and ECPred protocols.

Structural layer selection uses no downstream labels. The supervised Grid Search Oracle selects layers using task-specific validation performance and serves only as a reference.

The validation-selected Oracle is reported separately as a supervised layer-selection reference. For the structural selectors, identical selected layers reuse the same cached probe predictions and test scores, ensuring that their performance differences arise from layer selection rather than independent retraining.

We distinguish three GFM configurations to isolate the effects of graph construction, score formulation, and scale aggregation.

Table R4-C. Unified protocol for GFM, LTF, and neighborhood-based layer selectors.

SelectorNeighborhood ConstructionLayer-Selection CriterionPurpose
GFM-O/minEuclidean latent neighborhoods and uniform random point-cloud references, following the inspected original implementation.Minimize the minimum GFM score over k = 1–19.Original GFM formulation adapted to the shared benchmark.
GFM-M/minSame latent and Gaussian-null neighborhood graphs as Full LTF.Minimize the minimum GFM score over k.Controls graph and null construction.
GFM-M/meanIdentical graphs to GFM-M/min.Minimize the mean GFM score across k.Isolates the effect of scale aggregation.
kNN-overlap@8Manuscript's physical and latent graphs.Maximize native-neighborhood overlap at k = 8.Simple neighborhood-preservation baseline.
LTF without nullManuscript's existing no-null protocol.Apply the original no-null selection rule.Evaluates the contribution of null normalization.
Full LTFManuscript's directed k-NN graphs and Gaussian null reference.Maximize mean LTF over k = 1–19.Proposed operational selector.
Validation OracleFrozen representations with task-specific supervised probes.Select the layer using the corresponding downstream validation metric.Supervised reference.

The GFM-O/min configuration follows the original GFM score and its minimum-over-filtration depth summary, adapted to our common structural reference and downstream evaluation protocol. It is not presented as a reproduction of the original authors' published dataset-level results.

The matched configurations provide more controlled comparisons. GFM-O/min versus GFM-M/min examines the combined influence of neighborhood distance and null construction. GFM-M/min versus GFM-M/mean isolates scale aggregation, while GFM-M/mean versus Full LTF compares the remaining score formulations under identical neighborhood graphs.

This design directly tests whether the particular LTF construction offers benefits beyond the native-neighborhood information shared with GFM.

4. Experimental Results: Selection Agreement and Downstream Utility

4.1. GFM and LTF Identify Closely Related Structural Regions

We first compare the layers selected by the different structural criteria.

Table R4-D. Layer selections across ten dedicated PLMs. All selectors use the common structural reference without downstream labels. Full LTF and previously reported representative overlap/no-null selections are consistent with the manuscript; the complete comparison includes the additional GFM evaluations and model-wise selector results.

ModelGFM-O/minGFM-M/minGFM-M/meanOverlap@8No-null LTFFull LTF
ESM2-35M899989
ESM2-150M232324222324
ESM2-650M242425242525
Ankh Base383839373839
Ankh Large373738363738
ProtGPT2272828272729
CARP-38M141415141515
CARP-76M222323222323
CARP-640M394141394140
ProtMamba111110111011

The comparison reveals substantial agreement in the structural regions identified by GFM and LTF.

In particular, GFM-M/mean selects exactly the same layer as Full LTF for 7 of the 10 PLMs, with an average absolute difference of only 0.30 layers. All remaining selections differ by one layer. Even the original-style GFM-O/min configuration identifies layers within two positions of Full LTF across all ten models.

AuthorsOpenReview ↗

R4 (W5/W6/Q4). Scientific Contributions and Practical Value Beyond Prior Geometric Analyses

This agreement is consistent with their shared mathematical foundation: native-neighborhood preservation is the primary signal identifying the high-fidelity region, while differences in aggregation and normalization refine the exact selected layer.

The relevant practical question is therefore not whether LTF discovers an entirely different structural region, but whether these refinements meaningfully affect downstream prediction and model-wise selection reliability.

4.2. Per-Model Structural Prediction Shows Comparable but Nonidentical Performance

We evaluate the selected representations using the same CATH contact-distance prediction protocol.

Table R4-E. CATH contact-distance MAE across ten PLMs (Å, lower is better). All selectors use the same frozen representations and downstream probing protocol. Identical selected layers share the same underlying probe results. Oracle layers are selected using downstream validation performance.

ModelGFM-O/minGFM-M/minGFM-M/meanOverlap@8No-null LTFFull LTFOracle
ESM2-35M9.319.209.209.209.319.209.00
ESM2-150M8.208.208.108.368.208.107.90
ESM2-650M7.407.407.297.407.297.297.29
Ankh Base8.128.128.008.188.128.007.80
Ankh Large7.677.677.607.727.677.607.40
ProtGPT29.098.928.929.099.099.008.80
CARP-38M9.719.719.609.719.609.609.40
CARP-76M9.018.908.909.018.908.908.70
CARP-640M8.248.008.008.248.008.208.00
ProtMamba9.309.309.429.309.429.309.10

The results confirm that GFM and Full LTF provide closely related, but not identical, downstream performance.

GFM-M/mean and Full LTF produce identical contact MAE in seven models, reflecting their agreement in selected layers. The remaining three models demonstrate the effect of their different selection rules: GFM-M/mean performs better on ProtGPT2 and CARP-640M, whereas Full LTF performs better on ProtMamba.

These outcomes reinforce the mathematical analysis. Both methods exploit native-neighborhood correspondence, and neither uniformly dominates the other at the model level.

We next examine how these differences affect aggregate performance across structural and functional tasks.

4.3. Aggregate Performance Reveals a Trade-Off Between Average Accuracy and Worst-Case Regret

Table R4-F. Aggregate comparison across ten PLMs and four evaluations. Contact MAE and task accuracies are model averages. Mean ranks cover 40 model–task evaluations (Contact, Fold, EC-L1, and EC-L4) within the seven-selector pool shown here; ties receive average ranks. Worst contact regret is the largest model-wise test-MAE difference relative to the validation-selected Oracle.

SelectorContact MAE ↓Fold Acc. ↑EC-L1 Acc. ↑EC-L4 Acc. ↑Mean Rank ↓Worst Contact Regret ↓
GFM-O/min8.6050.69030.71280.46964.97500.32
GFM-M/min8.5420.69290.70910.47544.30000.32
GFM-M/mean8.5030.69440.71020.47823.88750.32
kNN-overlap@88.6210.69220.71460.46605.02500.46
LTF without null8.5600.68980.70930.47404.91250.32
Full LTF8.5190.69700.71200.47903.70000.20
Validation Oracle8.3390.71900.73200.48801.20000.00

The contact-MAE aggregates are calculated from the displayed per-model results. The rounded Oracle values average to 8.339 Å; the original manuscript reports 8.33 Å using its underlying full-precision results.

The aggregate comparison reveals a meaningful distinction between average performance and model-wise reliability.

GFM-M/mean achieves the lowest mean contact MAE among the downstream-label-free selectors (8.503 Å), slightly improving on Full LTF (8.519 Å). Conversely, Full LTF achieves the best mean rank within this comparison pool (3.7000 versus 3.8875), the highest fold accuracy (0.6970), and the highest EC-L4 accuracy (0.4790).

More importantly for model-adaptive layer selection, Full LTF has a lower observed worst-case contact regret of 0.20 Å, compared with 0.32 Å for all three GFM configurations. Thus, although the matched mean-GFM selector offers a marginally better average contact MAE, Full LTF exhibits a smaller largest model-wise gap to the supervised reference in the evaluated panel.

These findings do not establish uniform or statistically significant superiority of one formulation. They show that the choice of neighborhood-score aggregation can affect the balance among task-level performance, overall ranking, and worst-case selection behavior.

AuthorsOpenReview ↗

R5 (W7/Q5). Evaluation Independence and Task-Dependent Functional Transfer

SelectorESM2-650MAnkh LargeCARP-76M
Final0.77 / 0.47 / 0.350.78 / 0.48 / 0.360.71 / 0.39 / 0.27
Penultimate0.77 / 0.48 / 0.360.78 / 0.49 / 0.370.71 / 0.40 / 0.28
Fixed 75%0.75 / 0.47 / 0.360.76 / 0.48 / 0.360.70 / 0.42 / 0.31
Fixed 80%0.72 / 0.46 / 0.340.77 / 0.50 / 0.390.69 / 0.39 / 0.27
Mean over layers0.72 / 0.41 / 0.290.76 / 0.47 / 0.350.69 / 0.38 / 0.26
GFM-M/mean0.76 / 0.49 / 0.380.77 / 0.50 / 0.390.70 / 0.41 / 0.30
LTF0.76 / 0.49 / 0.380.77 / 0.50 / 0.390.70 / 0.41 / 0.30
Validation oracle0.77 / 0.49 / 0.380.79 / 0.52 / 0.410.72 / 0.43 / 0.32

Across all evaluated PLMs, LTF consistently improves fine-grained EC-L4 classification over the final layer. At the same time, the expanded baselines show that fixed-depth heuristics remain competitive: CARP-76M's fixed-75% baseline achieves an EC-L4 accuracy of 0.42 compared to 0.41 for LTF, while Ankh Large's fixed-80% baseline selects the same layer as LTF and matches its performance (0.50 accuracy / 0.39 macro-F1).

This directly addresses W7: our claim is not that every functional task uniquely requires the LTF formula, but that a task-independent structural criterion reliably identifies intermediate representations that improve fine-grained functional readout without a target-label layer search.

3.2. DeepLoc: Retaining the Negative Result

We retain the manuscript's negative DeepLoc result (Table 35) alongside the expanded baselines below, as it provides essential evidence regarding the boundary of LTF's applicability.

Table R5-E. DeepLoc test macro-F1 across layer-selection baselines.

SelectorESM2-650MAnkh BaseCARP-640M
Final0.6840.7110.615
Penultimate0.6880.7170.619
Fixed 75%0.6500.6810.602
Fixed 80%0.6620.6940.607
Mean over layers0.6690.7010.611
GFM-M/mean0.6540.6880.601
LTF0.6540.6880.595
Validation oracle0.6920.7170.624

The three-model LTF mean is 0.646, which remains below the final-layer mean of 0.670 (and penultimate-layer mean of 0.675, where Ankh Base's penultimate layer 47 coincides with the validation oracle). Adding the fixed-depth and layer-averaging baselines confirms that later layers remain preferable under this localization protocol.

DeepLoc therefore rules out any universal claim that maximizing native-neighborhood fidelity always identifies the optimal representation for every downstream biological label. However, this empirical boundary reflects the readout preferences of sequence-level localization signals rather than implying that subcellular localization has no biological relationship to structure.

4. Extending the Evaluation to Protein–Protein Binding Affinity

4.1. A Distinct Functional Target

PPB-Affinity provides curated protein-complex affinity annotations, including partner-chain and mutation information [1], building on prior formulations of sequence-based multi-chain affinity prediction [2]. Our contribution here is not the first application of PLMs to affinity prediction, but testing whether a representation layer selected solely from single-protein structural fidelity (SCOPe) transfers to intermolecular binding affinity prediction without binding-label-based layer selection.

The regression target is pKd=-10(Kd), with Kd expressed in molar units, and RMSE is reported in pKd units. We keep ESM2-650M, Ankh Large, and CARP-76M frozen and evaluate their SCOPe-selected layers alongside the baseline selectors. Each constituent chain is encoded with its annotated mutations applied; residue representations are pooled into chain representations and subsequently aggregated into the annotated receptor and ligand partners. Within each backbone, the exact same pairwise regression-head architecture and training budget are used across all layer choices. No native complex coordinates, interface labels, or target-complex LTF scores are provided to the predictor. To prevent leakage across related complexes, wild-type and mutant records belonging to the same interaction group are kept strictly within the same data split.

4.2. Affinity Prediction Against Strong Layer Baselines

Table R5-F. PPB-Affinity evaluation. Each cell reports Spearman / Pearson / RMSE (pKd). Higher correlations and lower RMSE are better. Validation oracle entries are selected separately by the corresponding validation metric.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.