- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses RNA–ligand interaction prediction in the structure-scarce regime. Rather than collapsing each RNA–ligand pair into a single fused representation, PIPER generates a small set of learned surrogate conformers by deforming a residue-level scaffold, ranks them using prior-guided geometric evidence and ligand-conditioned residue–atom interactions, and aggregates the top-two states for the final score. A source-aware screened residual acts as an auxiliary regularizer. Evaluations span multiple RNA–ligand benchmarks and controlled distribution-shift protocols (including RNA cold-start and scaffold-level shifts). Reported ranking metrics improve over the matched baselines, and the ablations indicate that both the sparse multi-conformer competition and the residual term contribute to the gains.
Critical strengths
- The problem formulation is well motivated. A single static representation can obscure state-dependent interaction signals; explicit competition among a compact set of surrogate geometric hypotheses is a reasonable inductive bias under limited tertiary-structure supervision.
- The empirical evaluation is broader than a single benchmark. Multiple datasets, several distribution-shift protocols, and targeted component ablations (conformer count, prior, cross-interaction, residual variants) make the comparative case more convincing.
- The authors state the limitations of the geometry and field clearly: the conformers are coarse surrogate geometries rather than atomistic ensembles, and the neural fields supply physics-informed regularization rather than calibrated electrostatic predictions. This framing is appropriate and should be retained.
Critical weaknesses
- The surrogate conformers are not shown to correspond to real conformational states. They arise from learned local deformations of a simple scaffold, and the paper provides no comparison with experimentally resolved RNA structures or ligand-specific binding modes/contact patterns. Consequently, the central “multi-conformer reasoning” claim is stronger than the evidence supports; the method is more accurately described as inference over a set of latent geometric hypotheses.
- The physics-informed component is less persuasive than the ranking results. While the screened residual has a plausible form, its contribution depends on several design choices (source assignment, probe sampling, screening) that are only partially controlled. Existing ablations (generic L2, homogeneous PDE residual, shuffled sources) show degradation, yet they do not fully isolate specificity from a generic smoothness or capacity effect of an auxiliary field regularizer.
- Evidence for consistent superiority is uneven. Some gains are modest relative to typical run-to-run variation, and the paper does not report clear target-level or paired statistical analysis. Claims that the method consistently outperforms strong baselines should therefore be stated more cautiously.
Questions for authors
- Can the learned surrogate conformers be compared with any resolved RNA structures or ligand-specific contact patterns to demonstrate that they capture meaningful interaction-relevant states?
- What additional control would more cleanly separate the contribution of the source-aware screened residual from a generic smoothness or capacity effect of an auxiliary field?
- Which performance gains remain significant across random seeds and individual RNA targets, and how sensitive are the results to the number of retained conformers (Top-1 / Top-2 / All-3)?
4 comments
Response to Reviewer Y3dF (Part 2 / 3)
W2 and Q2. Isolating Screening from Generic Regularization
We thank the reviewer for suggesting this targeted control. Following your advice, we conducted a source-preserving, no-screening experiment to isolate the contribution of screening from generic regularization and additional model capacity.
Matched screening control. As defined in Eq. (14), PIPER employs the residual
where denotes the surrogate source terms and controls screening. Setting removes only the screening term while retaining the source-aware Poisson constraint. Unlike the homogeneous-PDE ablation in Table 2, this control preserves the source assignments, Laplacian operator, and auxiliary-field capacity.
We independently retrained three variants on the SMRTnet fixed split, holding the data partition, architecture, source construction, probe configuration, regularization weight, optimizer, hyperparameters, and checkpoint-selection procedure fixed.
Table R3.2. Matched screening controls on SMRTnet.
| Screening configuration | AUC | AUPR |
|---|---|---|
| PIPER (screened residual, κ = 0.15) | 0.8667 | 0.7466 |
| Source-preserving, no screening (κ = 0) | 0.8584 | 0.7364 |
| Stronger screening (κ = 0.30) | 0.8582 | 0.7349 |
Contribution of screening. Removing screening alone reduces AUC/AUPR by 0.0083/0.0102, despite identical model capacity, source assignments, and regularization settings. Stronger screening (κ = 0.30) also performs below the default, indicating that increased screening strength does not necessarily improve predictive performance.
Distinction from generic regularization. The existing generic L2, homogeneous-PDE, and shuffled-source controls (Table 2; Appendix B.12) examine different aspects of field regularization, differential constraints, and source–geometry alignment. The new κ = 0 control specifically isolates screening while preserving the source terms. Together, these comparisons provide stronger evidence that PIPER's predictive benefit is associated with the source-aware screened formulation, rather than additional capacity or generic regularization alone.
These results support a positive contribution from screening under matched conditions. The screening-specific matched control and its implementation are now reported in Appendix B.12 of the revised manuscript.
Response to Reviewer Y3dF (Part 3 / 3)
W3 and Q3. Multi-Run Robustness and Conformer-Support Sensitivity
We thank the reviewer for emphasizing multi-run evidence and posterior-support sensitivity. We address both concerns through expanded OOD comparisons, paired RNA-clustered analysis, and three-run Top-1/Top-2/All-3 evaluation.
1. Robustness against strong baselines. To assess whether PIPER's performance advantages persist across independent runs, we extended the evaluation to three independent training runs for PIPER and all six external baselines across four OOD protocols, using matched splits and reporting mean ± SD. PIPER achieves the highest mean AUC/AUPR under RNA cold-start (0.8659/0.7480), scaffold-hopping (0.7327/0.5563), and dual cold-start (0.6827/0.4658).
We further conducted paired bootstrap analysis (5,000 resamples) to quantify the reliability of these improvements. Under RNA cold-start, PIPER outperforms the strongest baseline, RNAsmol, by +0.1769 AUC (95% CI [+0.1562, +0.1974]) and +0.2436 AUPR (95% CI [+0.2089, +0.2748]). Under scaffold-hopping, it surpasses SDNE by +0.0499 AUC (95% CI [+0.0240, +0.0754]) and +0.0494 AUPR (95% CI [+0.0174, +0.0839]). All four paired confidence intervals exclude zero, providing evidence that these improvements are not attributable solely to test-set sampling variability.
These evaluations support PIPER's generalization advantages beyond single-run comparisons. Complete three-run results and paired statistical analyses are provided in Appendices B.6–B.8 of the revised manuscript.
2. Matched ablations and RNA-level uncertainty. We compared PIPER against independently trained single-conformer and no-screened-residual variants across all four OOD protocols. To account for dependencies among pairs sharing the same RNA, we performed paired RNA-clustered bootstrapping (2,000 resamples per run), resampling complete RNA groups.
Table R3.3. Three-run OOD differences and RNA-clustered uncertainty.
| OOD protocol | PIPER vs. | ΔAUC | AUC CI excl. 0 | ΔAUPR | AUPR CI excl. 0 |
|---|---|---|---|---|---|
| Ligand cold | Single conformer | +0.0085 | 1/3 | +0.0006 | 1/3 |
| No residual | +0.0224 | 2/3 | +0.0258 | 1/3 | |
| RNA cold | Single conformer | −0.0042 | 1/3 | −0.0087 | 1/3 |
| No residual | +0.0263 | 3/3 | +0.0273 | 2/3 | |
| Scaffold-hopping | Single conformer | +0.0011 | 1/3 | +0.0012 | 1/3 |
| No residual | +0.0202 | 3/3 | +0.0004 | 0/3 | |
| Dual cold | Single conformer | +0.0301 | 2/3 | +0.0230 | 1/3 |
| No residual | +0.0166 | 1/3 | +0.0070 | 0/3 |
Δ denotes the three-run mean PIPER-minus-variant difference. CI counts indicate how many run-specific paired RNA-clustered 95% CIs exclude zero, regardless of direction; they do not establish cross-run significance.
The stronger evidence concerns screened-residual regularization. Removing it reduces mean AUC/AUPR across all four protocols. Under RNA cold-start, PIPER improves AUC/AUPR by +0.0263/+0.0273, with clustered CIs excluding zero in 3/3 and 2/3 runs. Under scaffold-hopping, the AUC improvement is +0.0202, with CIs excluding zero in all three runs. In contrast, multi-conformer gains are protocol-dependent: they are largest under dual cold-start, modest under ligand cold-start and scaffold-hopping, and slightly negative under RNA cold-start. Full paired and RNA-clustered analyses are included in Appendix B.8.
3. Posterior-support sensitivity. We extended the Top-1/Top-2/All-3 comparison to three independent runs. All variants generate three surrogate conformers and differ only in the number retained for posterior aggregation. Top-1 is distinct from the single-conformer ablation in Table 2, which generates only one conformer and disables conformer conditioning.
Table R3.4. Posterior-support sensitivity on SMRTnet (mean ± SD).
| Retained support | AUC | AUPR |
|---|---|---|
| Top-1 | 0.8708 ± 0.0133 | 0.7625 ± 0.0227 |
| Top-2 (PIPER) | 0.8781 ± 0.0031 | 0.7753 ± 0.0052 |
| All-3 | 0.8726 ± 0.0037 | 0.7668 ± 0.0064 |
Top-2 achieves the highest mean AUC/AUPR and lowest observed SD, improving by +0.0073/+0.0128 over Top-1 and +0.0055/+0.0085 over All-3. These results favor sparse Top-2 aggregation among the tested configurations, without establishing cross-run statistical superiority (Appendix B.16).
Overall, the expanded analyses provide stronger paired evidence for PIPER's advantages under RNA cold-start and scaffold-hopping, while identifying protocols where gains are modest or uncertain. The ablations further support screened-residual regularization, and the three-run support comparison favors Top-2 aggregation. We have incorporated these results into the revised manuscript and clarified the protocol-dependent scope of our conclusions.
We hope our responses have addressed your concerns and welcome any further feedback. We would be happy to incorporate these discussions into the camera-ready version.
Official Comment by Authors
Dear Reviewer Y3dF,
Thank you again for your thoughtful review and constructive questions. As the discussion period approaches its close, we would like to briefly follow up on our responses and the revised manuscript.
Following your suggestions, we have completed additional three-run OOD evaluations with paired uncertainty analyses (Section 4.3; Appendices B.6–B.8), screening and geometry controls (Appendices B.12–B.13), and an external HARIBOSS structural assessment (Section 4.5; Appendices B.16–B.17). We have also refined the Abstract and Conclusion to clarify the protocol-dependent contributions of multi-conformer reasoning and the interpretation of surrogate coordinates. All corresponding revisions are highlighted in blue.
We would greatly appreciate it if you could take another look and let us know whether these updates address your concerns. We would be happy to clarify any remaining questions before the discussion closes.
Best regards, The Authors
Response to Reviewer Y3dF (Part 1 / 3)
We sincerely appreciate the reviewer’s positive and insightful feedback, particularly the recognition of PIPER’s well-motivated formulation, multi-conformer reasoning, and comprehensive evaluation. Your encouragement further motivates our research.
We have carefully considered your constructive comments and questions, and strengthened the supporting evidence and methodological clarity. We provide detailed point-by-point responses below.
W1 and Q1. Structural Evidence for Multi-Conformer Reasoning
We thank the reviewer for suggesting validation against experimentally resolved RNA structures. Following your suggestion, we conducted an external structural assessment on HARIBOSS to examine whether PIPER's surrogate conformers capture interaction-relevant information. As defined in Section 3.1, these conformers represent latent geometric hypotheses rather than recovered molecular structures.
External structural assessment. We evaluated 114 RNA–ligand complexes with experimentally resolved contact information and 123 complexes for geometric assessment. Experimental structures were used exclusively for post hoc evaluation, without influencing model training or conformer selection. We compared PIPER with a matched no-screened-residual ablation, examining whether posterior preferences align with structure-derived RNA–ligand contacts.
For this diagnostic analysis, the selected conformer is defined by the highest posterior probability; PIPER's original Top-2 aggregation for interaction prediction remains unchanged, as defined in Eq. (13).
Table R3.1. External structural assessment on HARIBOSS.
Contact-based analyses use 114 complexes; geometric evaluation uses 123 complexes.
Evidence of contact-consistent reasoning. PIPER selects the candidate most consistent with experimental contacts in 87.7% of complexes, compared with 40.4% for the no-residual variant. Among discordant paired outcomes, 64 favor PIPER versus 10 favor the ablation (two-sided exact sign test, p = 8.96 × 10⁻¹¹). Moreover, the Spearman correlation between posterior preference and contact-derived candidate ranking increases from −0.10 to +0.50. These results provide external structural evidence that PIPER's posterior preferences better align with experimentally observed interaction contacts, with screened-residual regularization contributing to this alignment.
Structural interpretation. This improvement concerns the relative selection of interaction-relevant hypotheses rather than atomistic reconstruction. The selected candidates show comparable absolute contact AUPR (0.2996 vs. 0.2998) and pocket RMSD (15.65 Å for both models). Thus, the findings support contact-consistent hypothesis ranking within PIPER's multi-conformer framework, consistent with its design of evaluating competing surrogate geometries through interaction evidence rather than recovering physical conformational states.
The complete HARIBOSS assessment and its interpretation are included in Appendix B.17 (Table 27) of the revised manuscript.