acceptodds
Under review as a conference paper at ICLR 2027

Self-Evolving Hierarchical Trust Calibration for RBP-RNA Affinity Ranking

Abstract

Ranking RNA-binding protein (RBP)–RNA pairs by binding affinity is a foundational step toward characterizing post-transcriptional regulatory specificity, yet it remains difficult because the available computational evidence is heterogeneous, noisy, and context-dependent. Sequence-based predictors, physics-based energy functions, and structure-confidence metrics each supply useful but incomplete signals whose raw scores are not comparable across pairs. Conventional pipelines compound this by optimizing a single global objective without revisiting the ranking errors they produce, so recurrent failure patterns are discarded rather than used to refine subsequent calibration. The central challenge is therefore not to collect more scores, but to decide how much to trust each one, and when. We introduce BindGauge, a self-evolving agent that represents its knowledge as a compact hierarchy of interpretable importance weights over tools and over metrics within each tool, and that aggregates calibrated evidence through multi-round residual deliberation with cross-tool interaction terms. To make calibration adaptive, BindGauge maintains a Ranking Experience Memory that records past ranking outcomes, diagnoses recurrent failure signatures, and distills per-metric reliability profiles. This closed loop, separated from standard parameter training, enables BindGauge to evolve from a uniform aggregator into a task-specialized ranker while preserving full interpretability. With 145 interpretable scalars, BindGauge attains Spearman ρ = 0.832 on PRAB and 0.317 on KD on two homology-controlled benchmarks, against 0.500 (Random Forest) and 0.216 (SVM) for the strongest learned baselines on each, and it exposes per-round tool weights and failure-signature clusters that render each ranking decision auditable. Both test splits are small, so we report bootstrap intervals alongside point estimates. Our code is available at https://anonymous.4open.science/r/BindGauge-4F71/.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Self-Evolving Agents, Interpretable Calibration, Heterogeneous Tool Aggregation, Computational Biology, RBP-RNA Binding Affinity

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_1vVHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces BindGauge, a supervised framework for ranking RBP–RNA binding affinity by combining sequence, energy, and structural-confidence metrics from four computational tools. Its proposed contributions are hierarchical tool/metric weighting, optional cross-tool interactions, and a ranking-experience memory that supports reliability masking and difficulty-based reweighting across training cycles. Evaluation uses two benchmarks with reported homology-controlled splits, shared-feature baselines, component ablations, and tool-contribution analyses. BindGauge reports Spearman correlations of 0.832 on PRAB and 0.317 on KD, compared with 0.500 and 0.216 for the strongest learned baselines, respectively.

Critical strengths
  • A useful, inspectable approach to heterogeneous evidence integration. Learning explicit weights over tools and their metrics offers a practical way to inspect how different evidence sources affect affinity rankings. This is particularly relevant when computational tools provide signals with different scales and reliability.

  • Promising PRAB performance with meaningful component comparisons. The reported improvement over Random Forest is substantial, and learned baselines share the same features and split. Removing interactions or memory reduces PRAB Spearman correlation from 0.832 to 0.514 or 0.654, respectively, providing preliminary evidence that these components contribute beyond simply collecting more features.

Critical weaknesses
  • The evidence does not establish the robustness of the reported gains. The test sets contain only 41 and 39 pairs. Although the paper repeatedly claims to report seed variability, confidence intervals, and significance tests, Table 5 contains only placeholders. This matters because PRAB performance ranges from 0.350 to 0.832 across evolution cycles, while the KD improvement over standalone FoldX is only 0.034 (0.317 versus 0.283; Table 8). Validation-based selection is appropriate, but these results need numerical uncertainty estimates to support robust superiority.

  • The central adaptation mechanism has an apparent inconsistency, and its specific benefit is insufficiently isolated. Equation (9) assigns a violated pair’s difficulty to its lower-affinity endpoint, whereas Equation (5) weights that pair’s loss using the higher-affinity endpoint. Consequently, the written equations do not directly increase emphasis on the violation that generated the difficulty; with two misordered samples, they would downweight the sole violated pair. Separately, the uniform-to-first-cycle comparison in Section 4.5 introduces supervised fitting as well as memory, so it cannot establish that the improvement primarily comes from self-evolution. These issues affect a central contribution.

Questions for authors
  • Does the implementation weight a ranking pair using its higher-affinity endpoint, lower-affinity endpoint, or both? How are Equations (5) and (9) reconciled, and would correcting either equation change the reported results?

  • What controlled comparison isolates repeated self-evolution from ordinary supervised calibration under a matched optimization budget? PRAB peaks at the first evolution cycle, and Equation (3) computes each round’s correction without conditioning on the preceding prediction beyond additive accumulation.

Reviewer_ukiHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces BindGauge, an adaptive calibration framework for ranking RBP–RNA binding affinities. It integrates heterogeneous prediction metrics from RBPsuite, Rosetta, FoldX, and Boltz-2, combining hierarchical weight learning, cross-tool interactions, and a Ranking Experience Memory to iteratively identify and correct ranking errors. However, the current manuscript has important weaknesses in its core algorithmic definitions and experimental reliability, which limit the evidence supporting its main methodological contributions.

Critical strengths

None

Critical weaknesses
  1. The PRAB and KD test sets contain only 41 and 39 samples, respectively, and the results in Table 5 of Appendix A.8 remain placeholders.
  2. Equation (8) constructs ranking-error vectors using ground-truth labels and clusters samples in this space, whereas inference uses standardized metric vectors to retrieve failure signatures. The semantic meanings of these representations differ, and the manuscript needs to clarify how the query representation is mapped to the stored prototypes. Since memory retrieval is a core contribution of the paper, this ambiguity directly affects the validity of the reliability masks and the verifiability of the memory mechanism.
Questions for authors
  1. Appendix A.8 states that five-seed results, 95% bootstrap confidence intervals, and paired significance tests were computed, but Table 5 contains placeholders. Could you provide these results for BindGauge and the strongest baseline on each dataset, including confidence intervals for their paired Spearman differences?
  2. What exact query and prototype representations enter the cosine-similarity calculation, and how are they mapped into a common space using information available at inference?
Reviewer_oKgrOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes BindGauge, a method for ranking RNA-binding protein–RNA pairs by binding affinity using heterogeneous computational metrics. The method combines tool-level and metric-level softmax weights, optional cross-tool interactions, and a Ranking Experience Memory that records ranking errors, identifies failure clusters, and adjusts feature reliability and training-example weights. The authors evaluate the method on PRAB and KD using sequence-identity-controlled splits and report improvements over classical and neural baselines operating on the same feature vector.

Critical strengths
  1. Well-motivated application and inspectable model structure. Integrating sequence-based, energetic, and structural-confidence metrics is a reasonable approach to protein–RNA affinity ranking.

  2. Useful controls in the baseline evaluation. The learned baselines use the same standardized feature vector and data split. Appendix A.6 also states that the neural baselines use the same pairwise ranking loss. These choices help isolate the aggregation method from differences in upstream features.

Critical weaknesses
  1. The statistical evidence is incomplete, and several reported results are inconsistent. In Table 6, the default configuration reports Spearman 0.461 in the evolution-cycle sweep, but 0.521 in the other default-setting entries. I would like the authors to clarify whether these values come from different configurations or experimental runs. Appendix A.8 states that the authors conducted five-seed experiments and computed bootstrap confidence intervals and paired-difference significance tests. However, Table 5 contains placeholders for every numerical entry. This contradicts the abstract and main text, which repeatedly state that uncertainty estimates are reported. With only 41 PRAB and 39 KD test examples, these missing results materially limit the strength of the empirical conclusions.

  2. The memory-retrieval mechanism is not sufficiently specified. In Appendix A.4, DIAGNOSE clusters the label-dependent deviation vectors $ \delta_i $ and stores their centroids. RETRIEVE subsequently describes matching a query’s standardized metric profile $ \tilde f_*$ to a failure signature using cosine similarity. I could not determine the two representations actually used in this comparison.

Questions for authors
  1. Can the authors provide the missing numerical results in Table 5 and reconcile the inconsistent default and uniform-weighting results in Tables 4 and 6? Please identify the configuration, seed protocol, and checkpoint-selection rule associated with each result.

  2. What are the exact operands of the cosine similarity used by RETRIEVE? How is the query representation computed without its affinity label, and what happens when no cluster or multiple clusters match?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.