acceptodds
Under review as a conference paper at ICLR 2027

ReMatch: Learning to Rank Alternative Pairings for RNA–Protein Interaction Prediction

Abstract

Predicting RNA–protein interactions from sequence can help reveal how RNAs are processed, localized, and translated. Although pretrained sequence models and expressive interaction architectures provide rich molecular features, generalization to unfamiliar RNAs and proteins remains challenging. Training often relies on an uneven mix of evidence, that is, observed interactions have experimental support, but many negatives are untested pairs formed by re-pairing molecules. Treating these alternatives as confirmed negatives can suppress plausible interactions, especially when hard-example selection favors high-scoring pairs. These uncertain alternatives can still be useful, however, if they provide comparisons rather than additional negative labels. We introduce ReMatch, short for Ranking-enhanced Matching, which combines binary classification with bidirectional ranking. For each observed interaction, ranking teaches the model to prefer that pair over alternatives formed by replacing either molecule. The comparisons act on the classifier’s interaction log-odds, allowing an alternative to remain above the interaction boundary while the observed pair receives a higher score. Across five validation folds, adding ranking improves accuracy by approximately 4.7 and 2.5 percentage points on two benchmarks with unseen RNA and protein identities, with AUROC gains on both. Additional classification supervision under the same hard-pair selection policy does not reproduce these gains, supporting the value of learning relative preferences beyond selecting difficult examples. Ranking also improves preference for observed molecular assignments in existing predictors, extending the benefit of this supervision across interaction architectures. Code and preprocessing pipelines are available at https://anonymous.4open.science/r/rematch/.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
RNA–protein interaction, Pairwise ranking, Cross-modal representation learning, Contrastive learning, Out-of-distribution generalization, Biological language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_N2hAOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes ReMatch, a training objective for RNA–protein interaction prediction that combines standard binary classification with bidirectional ranking over alternative pairings. Instead of treating all re-paired RNA–protein alternatives as confirmed negatives, ReMatch uses them as relative comparisons: for an observed interaction, the model is trained to score the observed pair above alternatives formed by replacing either the RNA or the protein. This allows plausible unverified alternatives to remain above the interaction boundary while still giving higher score to experimentally supported pairs. The method uses frozen RNA and protein language-model features with a trainable interaction head, then applies ranking directly to the classifier’s interaction log-odds.

Critical strengths
  1. The paper correctly highlights that many “negative” RNA–protein pairs are generated by re-pairing molecules and may simply be untested rather than truly non-interacting. Using these alternatives as comparisons, rather than hard negative labels, is a sensible way to reduce the risk of suppressing plausible interactions.
Critical weaknesses
  1. The paper reports validation means rather than results on an untouched test set: within each fold, the checkpoint is selected by validation accuracy and then evaluated on that same validation split. This can introduce optimistic bias and makes the reported generalization performance less convincing.

  2. The statistical evidence is also not very strong. Although the paper reports gains from adding ranking, the paired confidence intervals are wide. Because these intervals cross zero, interpret the reported gains as promising point estimates rather than firm, statistically significant improvements.

  3. The baseline comparison is not fully matched. The reproduced baselines are reported as one run per setting, whereas ReMatch is reported as five-fold validation means. This makes it difficult to separate the effect of the proposed ranking objective from differences in training protocol, model selection, and fold averaging.

  4. The methodological novelty is somewhat limited. The ranking loss is based on standard pairwise preference/ranking ideas such as Bradley–Terry, RankNet, and BPR, and the in-batch cross-pair score matrix resembles common contrastive or retrieval-style training constructions. The main contribution is therefore not a fundamentally new ranking objective, but the application of this idea to RPI with unreliable negatives, the use of interaction log-odds, bidirectional RNA/protein replacement, and a careful set of controlled analyses.

  5. The biological interpretation could also be clearer. The method assumes that experimentally observed RNA–protein pairs should be ranked above re-paired alternatives, but this preference mainly reflects available experimental evidence rather than guaranteed biological incompatibility of the alternatives. The distinction between true non-interactions, untested alternatives, hidden positives, and experimentally supported observations is dense and may be difficult for readers to follow.

  6. There is also potential pretraining leakage that is not fully checked. The downstream splits avoid exact RNA/protein identity overlap, but the frozen RNA and protein language models may have seen validation-set sequences during pretraining. The paper notes that encoder-pretraining exposure would require a separate provenance check, so this remains an unresolved concern.

  7. Several design choices may require tuning, including top-k hard alternative selection, ranking temperature, replacement direction, ranking schedule, and the score location used for supervision. Although the paper includes useful ablations, it still does not fully establish practical robustness to these choices.

  8. On the Cas v2 transfer set, all models stay close to chance, and ReMatch only marginally beats the strongest baseline, with no uncertainty reported. This analysis also uses the earlier ReMatch + MI checkpoint rather than the final model, and it has no BCE-only control. It therefore does not show whether the ranking objective helps cross-domain transfer.

Questions for authors

My questions for the authors largely follow from the weaknesses discussed above. I would especially appreciate clarification on the lack of an independent test set, the statistical reliability of the reported gains, the mismatch between baseline and ReMatch evaluation protocols, the degree of novelty beyond standard pairwise ranking losses, the possibility of pretraining exposure to validation sequences through frozen encoders, and the biological interpretation of treating observed pairs as preferred over unverified alternatives.

1 comment

Thank you for helping us strengthen the study. We conducted 135 evaluations across seven predictors and two ReMatch variants, plus a full hyperparameter scan and tests of pretraining overlap and Cas transfer without MI. We include one table for space, and full results and procedures will accompany the revision.


Response to Q1:

We appreciate your suggestion to make evaluation more rigorous. Our original protocol followed the RPI baselines for comparability. We now use separate training, validation, and test data in each of five folds, following broader ML practice. We reserve 15% of each training partition for validation and checkpoint selection, then evaluate on the original held-out fold. These are revised runs on existing data. ReMatch's gains persist, and the clearer protocol should also help future RPI comparisons.

Table 1. Five-fold mean +/- standard deviation. Classification denotes BCE training of the same network without ranking.

DatasetModelAccuracyAUROC
CW30_50Classification0.636+/-0.0240.784+/-0.041
CW30_50ZHMolTopoRPI0.606+/-0.0050.756+/-0.043
CW30_50ReMatch0.721+/-0.0340.829+/-0.021
CW60_80Classification0.632+/-0.0750.748+/-0.085
CW60_80ZHMolTopoRPI0.650+/-0.0540.763+/-0.083
CW60_80ReMatch0.668+/-0.0520.791+/-0.091

Response to Q2:

Your careful assessment helps us state the evidence more precisely. We will describe the original CW gains as promising point estimates, since their paired intervals cross zero. We reran four matched controls with separate checkpoint selection: BCE, BCE-H, BCE+Rank, and BCE-H+Rank.

BCE+Rank improves mean accuracy and AUROC over BCE on all three benchmarks, with accuracy gains of 8.5 and 3.6 percentage points on CW30_50 and CW60_80. Hard-negative classification improves CW AUROC but lowers NPInter performance, while adding it to ranking gives no consistent further benefit. This supports learning preferences from sampled alternatives while retaining standard BCE for classification. We will include these controls in the revision so readers can assess the size and consistency of the gains.


Response to Q3:

Thank you for emphasizing fair comparisons. We reran all six existing predictors and report five-fold means and variation. ReMatch achieves the highest mean accuracy and AUROC on both CW benchmarks while remaining competitive on NPInter. The matched BCE control further shows that adding ranking improves the same network under this evaluation protocol.


Response to Q4:

Your comment helped us clarify the contribution. We use established ranking losses with alternatives formed by replacing either molecule. With BCE retained under the revised protocol, we compared cosine similarity and interaction log-odds. CW30_50 accuracy is 0.639 versus 0.721 for individual comparisons and 0.626 versus 0.723 for joint comparisons. Both favor supervising the score used for prediction.


Response to Q5:

We appreciate the distinction you raise: an observed interaction need not bind more strongly than an unobserved one. To test how this affects learning, we withheld known interactions and used them as alternatives. When withheld positives form 10.65% of selected alternatives, ReMatch retains 22.7% of the withheld set above the prediction threshold, versus 11.3% when BCE and ranking are retained but alternatives also receive negative labels. Retention is also higher at the other two rates. This supports using comparisons to preserve plausible interactions.


Response to Q6:

Thank you for encouraging this direct check. Unlabeled sequence exposure does not itself imply leakage of interaction labels. We split each test set into disjoint subsets with and without exact RNA matches to the checked RiNALMo pretraining source. On CW30_50 and CW60_80, ReMatch's AUROC is 0.787 and 0.750 with overlap, but 0.873 and 0.849 without overlap. Its predictive value thus extends beyond sequences present in that source.


Response to Q7:

We appreciate the practical concern. We tested every combination in our grid of top-k, temperature, replacement direction, and ranking schedule, alongside score-location controls. The results support k=5, temperature 0.25, bidirectional replacement, and two warm-up epochs. CW30_50 mean AUROC ranges from 0.819 to 0.832 across candidate counts, suggesting flexibility in this choice. The full results will provide a reproducible starting point and show where tuning matters.


Response to Q8:

Thank you for identifying the missing transfer control. We compared final ReMatch without MI against BCE-only on Cas v2. ReMatch has higher point estimates on both reported metrics. Although absolute scores remain modest, this is encouraging evidence that ranking benefits the final model without relying on MI. We will add these results to the revision.

Reviewer_pKNAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes ReMatch, a method that combines binary classification with ranking for RPI classification. The method presents alternative interactions (keeping one molecule fixed) to the scored interaction, supporting preference of observed data over unobserved, potentially negative interactions without the need to explicitly generate negative samples. ReMatch is evaluated across 2 datasets, outperforming existing methods. The authors provide ablation studies on the training objectives and the location of the ranking. In addition, ranking can be transfered to existing RPI methods, showing improved assignment preference.

Critical strengths
  • generation of negatives is potentially one of the hardest problems for RPI classification in my limited experience. I enjoyed reading the manuscript though I'm not completely familiar with the current approaches to RPI classification. A method that reduces/overcomes the need for generation of negative examples, however, is highly valuable for the community
Critical weaknesses
  • It was sometimes hard for me to follow the text, which, however, might at least in part be due to my limited knowledge of the current RPI classification approaches and literature.
Questions for authors

I was wondering whether additional/different constraints or criteria could be easily incorporated for the ranking of alternatives?

2 comments

Response to Weakness 1:

Thank you for this constructive feedback on improving the accessibility of our presentation for readers outside the RPI prediction field. We will prepare a simpler account below that separates the types of evidence and introduce the training steps before the notation. The central distinction is that benchmark pairs retain their class labels, while newly constructed alternatives provide comparisons without additional negative labels.

An observed interaction has experimental support. An alternative pairing is unverified and may still interact. A hidden positive is a known interaction deliberately withheld for a controlled test. Training first constructs alternatives and removes known training interactions from the comparison pool. It then selects the highest-scoring eligible alternatives and trains the observed pairs to score above them, alongside classification on the benchmark labels. We will use this sequence and consistent terminology in the manuscript so readers can follow how each type of evidence guides learning. Explaining the purpose of each step before the equations will also make the design choices and opportunities for extension easier to assess.


Response to Q1:

We appreciate this question since additional evidence may change which alternatives are useful and how confidently they should be compared. To examine this flexibility, we tested different comparison scores, compared alternatives individually or jointly, and applied ranking to three other RNA-protein interaction predictors. These tests show how the same idea can guide different predictors without changing how they make predictions.

The table gives a concrete example of reuse across predictors. Both columns come from the new five-fold evaluation. Base uses each predictor's original training objective, while With ranking adds ranking to that same predictor. Each row therefore compares objectives within one model. To keep within the response limit, we show one table and summarize the other comparisons below, while full results and settings will accompany the revision.

The metric is assignment preference, not classification accuracy. It measures how often a model gives a higher total score to two observed pairs than to the same molecules with their proteins exchanged. Higher is better, while exact ties do not count as successes. Values are means +/- standard deviations across five folds.

BenchmarkPredictorBaseWith ranking
NPInter2.0ZHMolTopoRPI0.872+/-0.0090.932+/-0.005
NPInter2.0ZeRPI0.799+/-0.0460.925+/-0.005
NPInter2.0BioLLMNet0.746+/-0.0160.866+/-0.008
CW30_50ZHMolTopoRPI0.693+/-0.0630.702+/-0.078
CW30_50ZeRPI0.662+/-0.0780.672+/-0.088
CW30_50BioLLMNet0.697+/-0.0520.705+/-0.059

Adding ranking improves the mean assignment-preference rate for all three predictors on both benchmarks. A separate comparison examines individual and joint alternatives in ReMatch under the same current evaluation protocol. Both variants retain benchmark binary cross-entropy. On CW60_80, individual comparisons at the final interaction score give accuracy of 0.668 and AUROC of 0.791, while joint comparisons give accuracy of 0.682 and AUROC of 0.788. Joint comparison has higher accuracy but slightly lower AUROC in this setting. Both variants retain the same prediction path, giving researchers different training choices within one architecture.

Additional evidence could enter at three points. First, it could determine which alternatives are eligible, while the method already excludes known training interactions. Second, a reliability weight could give uncertain comparisons less influence or omit an unsupported preference. Third, evidence for a stronger ordering could encourage a larger score gap. Reliability weights and evidence-dependent gaps are extensions we propose, building on the score and comparison choices already tested. When the added information is used only to construct training comparisons, individual pairs can still be predicted in the same way. This offers a practical route for using new evidence without redesigning the predictor.

Reviewer_pKNAOpenReview ↗

Response to author comments

I thank the authors for their clarifications and I'll keep my score.

Reviewer_DEzLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on RNA-protein interaction. In this setting, they propose to integrate classic classification with ranking to exploit "unknown interactions". To this end, they prpose ReMatch, which adds a pairwise ranking loss.

Critical strengths

The problem is well motivated and the approach is sound.

Results are promising and the experiments are well designed.

Critical weaknesses

In some cases (Table 9), statistical evidence is weak.

Since the authors introduce "ranking" as training techniques, I would expect the evaluation to also measure ranking metrics.

It looks like that checkpoint selection and testing are done on the same set of data. The authors should clarify this.

The approach reminds me of PPI Candidate Ranking (ICML 2026, Russo et al.). In that case the ranking is introduced for PPI, which is a similar setting wrt RPI. The authors do not mention this article in related work, which starts from similar assumptions: reframing classification as ranking and using unobserved pairs.

Questions for authors

Could the authors add some ranking metrics?

Could the authors argue better the problem of the checkpoint selection? If not, could they use a small testset portion for evaluation that was not seen during checkpoint selection?

Could the authors clarify whether PPI Candidate Ranking is comparable with ReMatch and in case propose a comparison?

3 comments

Thank you for helping us strengthen the evaluation. We added ranking metrics, reran classification comparisons with separate validation for checkpoint selection, and reviewed our controls and prior work. One ranking table appears below, and the full results and settings will accompany the revision.


Response to Q1:

Thank you for this helpful suggestion. We agree that classification performance alone does not show how well a model prioritizes candidate interactions. We therefore added mean reciprocal rank (MRR), mean average precision (MAP), Recall@10, and normalized discounted cumulative gain at 10 (nDCG@10) to examine this directly. Below, we compare ReMatch with ZHMolTopoRPI, the strongest retrieval baseline in our evaluation. Values are five-fold means +/- standard deviations.

DatasetMethodMRRMAPRecall@10nDCG@10
CW30_50ZHMolTopoRPI0.164+/-0.0360.131+/-0.0300.234+/-0.0420.148+/-0.033
CW30_50ReMatch0.158+/-0.0220.132+/-0.0210.227+/-0.0240.141+/-0.019
CW60_80ZHMolTopoRPI0.157+/-0.0630.121+/-0.0410.237+/-0.0940.141+/-0.058
CW60_80ReMatch0.172+/-0.0500.138+/-0.0320.252+/-0.0560.157+/-0.042

The clearest advantage appears on CW60_80, where ReMatch achieves higher mean scores on all four metrics. This is encouraging because the improvement extends to retrieving observed interactions among the top-ranked candidates. The results on CW30_50 are closer: ReMatch has slightly higher MAP, while ZHMolTopoRPI leads on the other three metrics.

We will include the full ranking results and document the query molecules, candidate pools, and averaging procedure in the revision. Your suggestion has helped us give readers a fuller picture of the model's practical value: how well it identifies interactions and how effectively it prioritizes candidates for further investigation.


Response to Q2:

Thank you for your recommendation for a more rigorous evaluation. We reran the comparisons with training, validation, and test partitions in each of five folds. Under this revised protocol, ReMatch (BCE plus ranking) has higher mean accuracy and AUROC than the same network trained with BCE alone on both CW benchmarks.

Our original protocol followed the RPI baseline studies to keep comparisons aligned with their reports. We agree that the broader ML practice of separating training, validation, and testing strengthens the assessment. We retain the original folds and reserve 15% of each training partition for validation and checkpoint selection. The original held-out fold is used for testing, not checkpoint selection, in these new runs. Those folds had been examined in the submitted study, so these are revised evaluations on existing benchmark data.

Both models have lower absolute scores in the revised evaluation than in the original report. On CW30_50, accuracy/AUROC changes from 0.723/0.813 to 0.636/0.784 for BCE alone, and from 0.770/0.844 to 0.721/0.829 for ReMatch. On CW60_80, it changes from 0.703/0.791 to 0.632/0.748 for BCE alone, and from 0.729/0.799 to 0.668/0.791 for ReMatch. These historical changes are distinct from the benefit of adding ranking. The latter requires a same-protocol comparison of BCE alone with BCE plus ranking. We compare means without claiming paired statistical significance: under the revised protocol, ReMatch exceeds BCE alone in both metrics on both benchmarks.


Response to Q3 and Weakness 4:

We appreciate the reference, it helps place our contribution more clearly. We examined Russo et al. (ICML 2026) alongside our objective. Both studies use incomplete interaction records to prioritize plausible pairs, and we will add the citation and discuss this shared motivation in Related Work.

The main distinction is what information ranking uses and when it is needed. Russo et al. retrieve candidates using a query protein's known partners and regions selected from predicted contact maps, then refine the ranking with additional evidence. ReMatch learns preferences during training at the classifier's interaction score, while prediction needs only the queried RNA and protein sequences. Our existing controls compare applying ranking to representation similarity or the final prediction score. They examine where to apply supervision, rather than reproduce the Russo pipeline.

Reference: Russo et al., PPI Candidate Ranking, ICML 2026. https://proceedings.mlr.press/v306/russo26b.html

Thank you for the detailed response and the additional experiments.

Checkpoint selection: I appreciate the rerun with a separate validation split and testing on the original held-out fold. It is good that you report the lower absolute scores openly, and that the gain of ranking over BCE alone persists under the new protocol. This resolves my main concern about the evaluation setup. Since the hyperparameters were chosen on folds that had already been examined, I would still describe the revised numbers as less biased rather than fully independent.

Ranking metrics: thank you for adding MRR, MAP, Recall@10 and nDCG@10. However, the comparison is only against ZHMolTopoRPI, and ReMatch is not better on CW30_50 (it leads on MAP only) and is within one standard deviation on CW60_80. The more informative comparison would be BCE-only versus BCE+rank, to show whether the ranking objective improves ranking itself. The query and candidate-pool protocol also needs to be specified, and the results are currently only partial.

Comparison with Russo et al.: I appreciate the commitment to cite it and discuss the shared motivation. However, my question was whether the two approaches are comparable and, if so, to propose a comparison. The response describes how the methods differ (what information ranking uses and when) but does not state whether a comparison is feasible, and no comparison or adapted baseline is provided or planned. Since both works start from the same assumption, reframing classification as ranking using unobserved pairs, I consider this a relevant missing baseline or, at minimum, a missing argument for why it cannot be compared.

Overall, the rebuttal addresses only part of my concerns, therefore I would still keep my score.

Thank you for the careful follow-up.


Response to AQ1 (Checkpoint selection)

We agree with your distinction and appreciate the suggestion: the revised estimates are less biased, rather than fully independent, because the original folds informed hyperparameter choices. As complementary evidence, we also evaluated final ReMatch without MI against BCE-only on the Cas v2 transfer set. ReMatch achieves higher scores on both reported metrics. Although absolute performance remains modest, this comparison provides additional support for ranking beyond the CW benchmarks.

We will continue seeking an additional dataset that has not informed model development. Preparing suitable data and completing the comparisons will require more time than remains in the rebuttal period. We will include the Cas results in the revision and report the additional evaluation in a subsequent version once complete.


Response to AQ2 (Ranking metrics)

Thank you for suggesting this direct test of the ranking objective. We included only ZHMolTopoRPI in our previous response because it was the strongest baseline on the reported ranking metrics. We now compare BCE-only with BCE+Rank in the same architecture and evaluation protocol. Both tables include all six baselines, values are five-fold means +/- standard deviations.

CW60_80

MethodMRRMAPRecall@10nDCG@10
ZeRPI0.115+/-0.0710.090+/-0.0440.179+/-0.0660.101+/-0.053
PRInter0.084+/-0.0320.057+/-0.0140.101+/-0.0220.060+/-0.018
CORAL0.081+/-0.0370.058+/-0.0210.132+/-0.0510.071+/-0.029
MHAM-NPI0.072+/-0.0280.055+/-0.0260.075+/-0.0470.047+/-0.031
ZHMolGraph0.119+/-0.0630.105+/-0.0520.207+/-0.1320.114+/-0.067
ZHMolTopoRPI0.157+/-0.0630.121+/-0.0410.237+/-0.0940.141+/-0.058
ReMatch (BCE)0.142+/-0.0600.110+/-0.0420.211+/-0.0690.124+/-0.052
ReMatch (BCE+Rank)0.172+/-0.0500.138+/-0.0320.252+/-0.0560.157+/-0.042

CW30_50

MethodMRRMAPRecall@10nDCG@10
ZeRPI0.128+/-0.0550.095+/-0.0290.160+/-0.0290.099+/-0.031
PRInter0.062+/-0.0070.042+/-0.0060.063+/-0.0100.039+/-0.008
CORAL0.122+/-0.0510.075+/-0.0320.112+/-0.0360.081+/-0.036
MHAM-NPI0.069+/-0.0450.056+/-0.0260.061+/-0.0250.044+/-0.032
ZHMolGraph0.124+/-0.0600.108+/-0.0520.188+/-0.0990.113+/-0.065
ZHMolTopoRPI0.164+/-0.0360.131+/-0.0300.234+/-0.0420.148+/-0.033
ReMatch (BCE)0.131+/-0.0290.104+/-0.0210.189+/-0.0280.110+/-0.024
ReMatch (BCE+Rank)0.158+/-0.0220.132+/-0.0210.227+/-0.0240.141+/-0.019

All four mean ranking metrics improve on both benchmarks, by approximately 19-28% over BCE-only using unrounded values. The matched results support better retrieval from adding ranking to the same model. We will include the complete results in the revision.


Response to AQ3 (Comparison with Russo et al.)

Thank you for emphasizing the need for an empirical comparison. We apologize for misunderstanding your question and not directly addressing the feasibility of the comparison in our previous response. Yes, a comparison is feasible, and we have now implemented and evaluated an RPI adaptation of Russo et al. We retained the full known-partner retrieval and reranking workflow and the D-SCRIPT scoring component, replacing only the contact predictor with RPcontact.

We evaluated transfer learning based on existing training data sources and on the RPI14K dataset (derived from RPI7317, which partially overlaps with our existing data) and retrained both methods under its two split rules.The frozen adaptation achieved AUROC 0.474 in the transfer tests. Results after retraining are shown below as means +/- standard deviations.

DatasetMethod30_50 ACC/AUROC60_80 ACC/AUROC
RPI14KAdapted Russo et al.0.542±0.040 / 0.482±0.1080.611±0.062 / 0.474±0.069
RPI14KReMatch0.529±0.075/ 0.580±0.0640.665±0.036 / 0.723±0.032
CWAdapted Russo et al.0.683±0.031 / 0.714±0.0640.691±0.014 / 0.719±0.038
CWReMatch0.721±0.034 / 0.829±0.0210.668±0.052 / 0.791±0.091

The adapted Russo method is competitive in accuracy, exceeding ReMatch on 30/50 (0.542 versus 0.529), which makes it a valuable addition to the comparison. ReMatch achieves higher mean AUROC on both splits and higher accuracy on 60/80. Together, these experiments acknowledge the strengths of the adapted framework while providing additional evidence for ReMatch’s predictive performance across the two RPI settings.

Your follow-up helped us turn a discussion of methodological differences into an implemented and evaluated baseline. We will include the complete transfer and retraining results, adaptation details, and evaluation protocol in the revision.

Comments on the paper

1 comment

We thank the reviewers for the careful reading and constructive suggestions. Your feedback led us to strengthen evaluation and test the ranking objective more directly. We summarize the new evidence below; values are five-fold means ± standard deviations unless stated otherwise.


  1. Revised evaluation and objective controls We reran six predictors, ReMatch, and two variants across three datasets and five folds (135 evaluations), reserving 15% of each training partition for checkpoint selection before testing on the original held-out fold. These estimates are less biased, rather than fully independent, because the folds previously informed hyperparameter choices.

BCE-H adds hard cross-pairs as negative labels. Adding ranking to BCE improves both mean metrics on all three datasets; hard-pair classification does not consistently reproduce these gains.

ObjectiveNPInter2.0 ACC/AUROCCW30_50 ACC/AUROCCW60_80 ACC/AUROC
BCE0.942±0.005 / 0.979±0.0010.636±0.024 / 0.784±0.0410.632±0.075 / 0.748±0.085
BCE-H0.924±0.005 / 0.966±0.0040.709±0.063 / 0.840±0.0150.651±0.055 / 0.791±0.072
BCE-H+Rank0.929±0.007 / 0.964±0.0090.674±0.026 / 0.823±0.0260.674±0.049 / 0.790±0.067
ReMatch (BCE+Rank)0.946±0.003 / 0.982±0.0010.721±0.034 / 0.829±0.0210.668±0.052 / 0.791±0.091

  1. Direct retrieval evaluation We evaluated all six baselines and matched BCE/BCE+Rank models. Below we retain the strongest reported retrieval baseline and the objective control. Adding ranking improves all four mean metrics on both benchmarks, by approximately 19-28% relative to BCE-only.

CW30_50

MethodMRRMAPRecall@10nDCG@10
ZHMolTopoRPI0.164±0.0360.131±0.0300.234±0.0420.148±0.033
ReMatch (BCE)0.131±0.0290.104±0.0210.189±0.0280.110±0.024
ReMatch (BCE+Rank)0.158±0.0220.132±0.0210.227±0.0240.141±0.019

CW60_80

MethodMRRMAPRecall@10nDCG@10
ZHMolTopoRPI0.157±0.0630.121±0.0410.237±0.0940.141±0.058
ReMatch (BCE)0.142±0.0600.110±0.0420.211±0.0690.124±0.052
ReMatch (BCE+Rank)0.172±0.0500.138±0.0320.252±0.0560.157±0.042

  1. Reuse in existing predictors Ranking increases assignment-preference rates for all three predictors on both benchmarks, supporting reuse beyond our architecture.
PredictorNPInter2.0 Base+RankCW30_50 Base+Rank
ZHMolTopoRPI0.872±0.0090.932±0.0050.693±0.0630.702±0.078
ZeRPI0.799±0.0460.925±0.0050.662±0.0780.672±0.088
BioLLMNet0.746±0.0160.866±0.0080.697±0.0520.705±0.059

  1. Score location and aggregation Interaction log-odds outperform embedding cosine on both CW metrics under either aggregation rule, supporting supervision at the prediction score.
Score / aggregationCW30_50 ACC/AUROCCW60_80 ACC/AUROC
Embedding cosine / pairwise0.639±0.033 / 0.800±0.0300.634±0.055 / 0.769±0.061
Embedding cosine / multi-candidate0.626±0.039 / 0.809±0.0340.635±0.061 / 0.772±0.077
Log-odds / pairwise0.721±0.034 / 0.829±0.0210.668±0.052 / 0.791±0.091
Log-odds / multi-candidate0.723±0.023 / 0.831±0.0230.682±0.067 / 0.788±0.084

  1. Controlled hidden positives ReMatch retains more hidden positives with positive scores than either hard-negative classification control at every tested contamination level. The table reports P(Sh>0).
Realized contamination (%)BCE-HRankBCE-H+RankReMatch
10.6500.122±0.1920.091±0.1520.113±0.1440.227±0.224
22.0490.122±0.1690.091±0.1530.106±0.1440.226±0.220
32.7040.116±0.1430.088±0.1570.103±0.1320.218±0.210

  1. Pretraining exposure and external transfer We evaluated seven predictors on subsets with and without exact RNA matches to the checked RiNALMo source. ReMatch's AUROC below shows useful prediction beyond exact matches.
DatasetOverlapNon-overlap
NPInter2.00.977±0.0030.989±0.003
CW30_500.787±0.0110.873±0.049
CW60_800.750±0.0910.849±0.075

On the 4,238-example Cas v2 transfer set, final ReMatch without MI exceeds BCE-only on both reported measures. Absolute performance remains modest, but this adds evidence for ranking beyond the CW benchmarks. These are point estimates.

MethodACCAUROC
ReMatch (BCE-only)0.50990.5344
ReMatch0.52360.5541

We appreciate the reviewers' help in sharpening these comparisons. The revision will include all baseline results, remaining metrics, and evaluation protocols.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.