- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
I think this paper presents an interesting approach to improving protein-RNA structure prediction using experimental binding signals. Compared with existing structure prediction models that mainly rely on structural data, this paper tries to use eCLIP data, which provides information about protein binding regions on RNA but does not contain atomic-level structural information. More detailed into the method, the authors propose ESFold, a two-stage post-training framework based on Protenix. In the first stage, the model learns binding-aware representations by aligning predicted RNA contact probabilities with experimental eCLIP signals, together with standard structural supervision from PDB. In the second stage, the authors adapt DiffusionNFT to optimize generated structures using experimental agreement and geometric constraints as rewards. The model does not require experimental signals during inference.
The experiments are conducted on the protein-RNA subset of FoldBench and a held-out eCLIP evaluation set. The method improves interface prediction success from 61.43% to 74.29% and experimental peak F1 from 0.332 to 0.459 compared with Protenix.
In all, I think the paper investigates a meaningful problem and reports some promising results. Using experimental data to improve pretrained structure prediction models is an interesting direction, especially when such data are much more available than experimentally resolved structures. However, after looking more carefully into the methods and experiments, I have several concerns regarding the evaluation metrics, experimental evidence, and the overall presentation of the paper. Some of these concerns are directly related to whether the reported results can fully support the main claims.
Critical strengths
-
I think the paper investigates an important problem in biomolecular structure prediction. Protein-RNA complex structures remain relatively limited, while experimental interaction data such as eCLIP provide additional information that could potentially improve structure prediction. Exploring how to incorporate such experimental signals into pretrained structure models is a meaningful research direction.
-
The proposed two-stage framework is reasonable, and the reported improvements are encouraging. The authors combine binding-aware representation learning with reward-based structure optimization and evaluate both structural accuracy and agreement with experimental signals. I think this is a useful direction, particularly because experimental binding information does not directly provide atomic-level structural supervision.
Critical weaknesses
-
I am not fully convinced that the current experiments sufficiently support the main contributions. First, the definition of Peak F1 in Appendix C allows predicted regions to match experimental peaks through any overlap, without one-to-one matching. Together with percentile-based thresholding, this can potentially give high scores to predictions with poor localization. Since Peak F1 is one of the main evaluation metrics, I think a more careful evaluation is necessary. Moreover, Stage 1 combines PDB structural supervision with eCLIP signals, but the paper does not include a PDB-only fine-tuning baseline or shuffled eCLIP controls. Therefore, it remains unclear how much of the improvement can be attributed specifically to meaningful experimental supervision.
-
I have significant concerns about the experimental analysis and overall presentation of the paper. Although Stage 2 introduces several reward components, the paper does not provide sufficient ablations to explain which components contribute to the final improvement. The structural evaluation set is also relatively small, making additional statistical analysis important. More importantly, I think the presentation quality is considerably below what I would expect for an ICLR submission. The main text does not include detailed quantitative result tables, with most important comparisons placed in the appendix. The figures are not vectorized, which affects readability, and some training details are not clearly reported. Given the complexity of the proposed framework, I think the paper needs a much clearer experimental presentation and more thorough analysis to support its conclusions.
Questions for authors
-
Could the authors provide additional evaluation of Peak F1 using stricter region matching or position-level metrics? I think it would also be helpful to report the fraction of predicted positive positions to better understand the reliability of the current metric.
-
Could the authors compare Stage 1 with PDB-only continued fine-tuning under a similar training budget? Also, would the improvements remain if the eCLIP signals were shuffled or replaced with mismatched profiles?
-
Could the authors provide more detailed ablations of the Stage 2 reward components? I am also interested in whether the experimental reward correlates with structural accuracy, such as DockQ.
7 comments
PDB-only fine-tuning and signal-control experiments
Under the same training budget, true-eCLIP Stage-1 supervision achieves higher structural accuracy and signal agreement than PDB-only fine-tuning and both signal controls.
- Shuffled-eCLIP: randomly permute the measured signal values across valid RNA positions, preserving each example’s signal-value distribution but disrupting positional correspondence.
- Random-signal control (labeled “mismatched-eCLIP” in the table): replace the measured profile with a random signal. This tests uninformative supervision rather than a real profile from another protein–RNA pair.
True-eCLIP and both signal controls retain the same PDB structural supervision; only the eCLIP labels differ. PDB-only excludes eCLIP supervision, and Protenix v1.0 is the pretrained baseline.
| Stage-1 supervision | DockQ success | iRMSD (Å) | lRMSD (Å) | lDDT |
|---|---|---|---|---|
| Protenix v1.0 | 61.43% | 7.61 | 18.66 | 0.74 |
| PDB-only | 64.29% | 6.46 | 16.86 | 0.76 |
| Shuffled-eCLIP | 62.86% | 7.06 | 17.41 | 0.76 |
| Mismatched-eCLIP | 61.43% | 7.34 | 17.82 | 0.75 |
| True-eCLIP (ESFold_rep) | 70.00% | 5.37 | 13.24 | 0.78 |
Structural results use the same 70 interfaces from 49 complexes. Signal-agreement results below use the 190-condition, 82-RBP eCLIPBench.
| Stage-1 supervision | Peak F1 [95% CI] | Jaccard | Spearman ρ | Enrichment |
|---|---|---|---|---|
| Protenix v1.0 | 0.332 [0.305, 0.361] | 0.126 | 0.051 | 1.732 |
| PDB-only | 0.339 [0.297, 0.379] | 0.119 | 0.035 | 1.797 |
| Shuffled-eCLIP | 0.338 [0.304, 0.371] | 0.104 | 0.056 | 1.763 |
| Mismatched-eCLIP | 0.315 [0.281, 0.344] | 0.113 | 0.047 | 1.772 |
| True-eCLIP (ESFold_rep) | 0.417 [0.388, 0.445] | 0.155 | 0.144 | 3.044 |
Manuscript revision. Section 4.5 and Table 2 present the supervision controls; Appendix N, Tables 29–30, provides the remaining metrics.
Stage-2 ablations and reward–DockQ association
The ablations indicate that reference loss provides the strongest stabilization, geometry helps preserve physical plausibility, and signal reward gives a modest Peak F1 gain in the evaluated run.
| Ablation setting | Mean Peak F1 ↑ | Mean physical penalty ↓ |
|---|---|---|
| All rewards + reference loss | 0.353 | 0.086 |
| No signal reward | 0.345 | 0.089 |
| No geometry | 0.355 | 0.089 |
| No reference | 0.273 | 0.171 |
| No reference + no geometry | 0.168 | 0.605 |
Values average nine validation checkpoints from steps 8,000–10,000, sampled every 250 steps, from one run per setting. Geometry terms are ablated as a group.
For reward–DockQ association, the available diagnostic uses native-derived pseudo-eCLIP profiles, not real experimental profiles paired with native structures:
| Native-derived diagnostic, 49 complexes | Pearson r [95% CI] | Spearman ρ [95% CI] |
|---|---|---|
| Enhanced eCLIP reward vs. mean interface DockQ | 0.473 [0.239, 0.662] | 0.394 [0.121, 0.615] |
| Peak F1 component vs. DockQ | 0.282 [−0.050, 0.553] | 0.202 [−0.105, 0.463] |
| Profile-rank component vs. DockQ | 0.631 [0.465, 0.766] | 0.655 [0.504, 0.764] |
Interface scores are averaged within each of the 49 complexes, with 2,000 bootstrap resamples. The enhanced reward shows a positive association with DockQ in this native-derived diagnostic; its relationship to DockQ using real experimental eCLIP profiles remains untested.
Manuscript revision. Section 4.5 and Table 3 report the five ablations. Appendix O defines the ablations and reports the native-derived reward–DockQ diagnostic in Table 31; the paper includes the trajectories in Figure 8.
Stricter localization metrics and positive-position coverage
We agree that overlap-based Peak F1 alone does not establish precise localization. We added nucleotide-level metrics, maximum-cardinality one-to-one peak matching, prediction coverage, and a threshold sweep. The comparison uses matched confidence-selected Top-1 predictions on 190 conditions from 82 RBPs.
ESFold improves position-level metrics and one-to-one F1 at IoU ≥0.10. Gains are smaller at IoU ≥0.25 and do not extend to IoU ≥0.50, indicating stronger regional localization rather than uniformly sharper peak boundaries.
Both models predict the same positive-position fraction. ESFold predicts fewer, wider regions, which motivates the additional one-to-one matching and threshold-sensitivity analyses below.
We additionally evaluate an absolute contact definition: minimum RNA–protein heavy-atom distance ≤8 Å, without percentile thresholding for the binary mask. Binary contact metrics and RNA-position PR-AUC are reported below. PR-AUC is the trapezoidal area under the RNA-position-level precision–recall curve; it is distinct from average precision (AP).
Together, these analyses show that the improvement persists in position-level evaluation, across prediction thresholds, and under an absolute distance-based contact definition. All signal metrics are averaged within each RBP and then across the 82 RBPs; paired intervals are 95% bootstrap CIs.
Manuscript revision. Section 4.4 and Appendix M, Tables 24–28, now give strict matching, coverage, threshold sensitivity, and distance-based results. Appendix Q defines AP and trapezoidal PR-AUC, including the ranking score and threshold direction.