- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces RiboSeer, an agentic framework for RNA-binding pocket prediction. The framework integrates 15 existing tools across four categories: complex structure prediction, pocket detection, binding residue prediction, and molecular docking. An LLM-based agent analyzes target characteristics and selects appropriate tools, and a LightGBM-based fusion model combines their predictions to produce residue-level binding probabilities. RiboSeer is evaluated on a benchmark of 107 held-out targets. It achieves a Pearson correlation of 0.593 and an AUROC of 0.899, outperforming the best individual predictor (Pearson r = 0.439) and a weighted ensemble baseline (0.515). Ablation studies show that most improvements come from learned prediction fusion, while LLM-based orchestration provides an additional but relatively modest contribution (~7% improvement). Overall, the paper demonstrates the practical benefits of adaptively combining heterogeneous scientific prediction models.
Critical strengths
-
The paper presents an interesting and practical idea: dynamically selecting and combining different models based on the characteristics of each problem, rather than relying on a single model. Given the complementary strengths and limitations of existing predictors, I believe this is a meaningful and potentially generalizable direction for scientific applications.
-
The proposed framework achieves good empirical performance. On 107 held-out targets, RiboSeer achieves a Pearson correlation of 0.593, outperforming the best individual predictor Boltz-2 (0.439), as well as a simple weighted ensemble method (0.515). These results demonstrate the practical value of combining heterogeneous predictors.
Critical weaknesses
-
While the overall framework is well-motivated, the technical contributions is relatively limited. The proposed method mainly integrates existing techniques, including LLM-based tool selection, bandit-based exploration, and LightGBM-based prediction fusion. The paper focuses more on system integration than on introducing fundamentally new algorithms or methodological advances.
-
Although the paper emphasizes agentic orchestration, the actual contribution of the LLM agent is relatively modest. The ablation results show that adding the agent improves Pearson correlation from 0.554 to 0.593, corresponding to an approximately 7% relative improvement. Moreover, the agent primarily acts as a model router rather than performing more critical scientific reasoning or decision-making. This raises questions about the necessity of an LLM-based agent compared with simpler model selection strategies, particularly given that most of the performance gains come from learned prediction fusion. Additionally, the paper's presentation could be improved. Figure 1 is overly dense, and its text and visual elements are too small to read comfortably.
Questions for authors
The proposed framework seems potentially applicable to a broader range of scientific prediction tasks. Do the authors have plans to extend this approach to other tasks? More importantly, do the authors believe the current framework can be readily generalized to different scientific problems, or would substantial task-specific adaptations be required?
2 comments
Response to Reviewer A3CN (2/2)
(continued from part 1)
Q. Generalizing to other scientific tasks
Reusable: the adapter contract that standardizes incompatible predictor outputs, the fallback for failed tools, the feature construction over tool agreement and context, and the learned fusion. In our experience these are the parts that take the time and are the parts that transfer as a recipe.
Not reusable without work: a new task needs its own predictors and adapters, its own label definition, its own fusion features and training data, and, if online feedback is used, a task-appropriate surrogate reward. We have run preliminary extensions toward RNA–ligand, protein–ligand and protein–protein interface prediction, and each required task-specific tools and inputs; we do not present those as a controlled cross-task demonstration, so they are not in the paper.
The conclusion now states this explicitly: the framework is a reusable recipe for domains where incompatible predictor families coexist under separate protocols, not a zero-shot capability. We would rather scope it that way than imply generality we have not measured.
We think the revision is now an accurate account of where the value sits: a heterogeneous benchmark and a fusion result that reproduces exactly and holds up under paired tests and a corrected pocket protocol, with the orchestration component reported honestly as interface rather than mechanism. If a further experiment would help your assessment, we are glad to run it during the discussion period.
Thank you again for the constructive feedback. If the revised contribution statement and new comparisons address your concerns, we would appreciate it if you would consider raising your score.
Best regards,
Authors
Response to Reviewer A3CN (1/2)
Dear Reviewer A3CN,
Thank you for the review, and for seeing the per-target selection idea as a direction worth pursuing beyond this task. Your two criticisms both landed, and the revised PDF (changes marked in blue) acts on them rather than argues with them.
W1. Limited technical novelty
We agree that LLM tool selection, bandit utilities and LightGBM fusion are established, and we do not claim a new learning algorithm. What we claim, and now state that way in the contributions, is the integration and comparative evaluation: 15 predictors from four paradigms produce scores on different scales with different failure modes and cannot simply be averaged, so RiboSeer standardizes residue-level outputs through per-tool adapters, handles failed predictions, and fuses per-tool scores together with cross-tool agreement, sequence-neighbourhood context and a target profile.
The evidence that this is not a trivial wrapper is the gap between having the tools and using them: best single tool 0.439, simple mean over the same 15 tools 0.446, train-weighted mean 0.515, learned fusion 0.593. On the 104 targets where Boltz-2 is valid, the paired gain over it is +0.155 (95% CI [0.121,0.188], 10,000 PDB-grouped bootstrap resamples), which we added in revision so that the headline comparison is made on a common set of targets.
W2. Necessity of the LLM agent, and Figure 1
You and the other reviewers converged on the same point, so we tested it directly instead of re-interpreting the 0.554→0.593 ablation. On the same 107 targets (Table 29 in the revision):
No interval excludes zero, so none of these comparisons supports an accuracy benefit specific to LLM reasoning. The original ablation could not isolate it either, because the mandatory core is present in both arms. The abstract, Sec. 2, Sec. 4.4 and the conclusion now say that learned fusion supplies the gain and that LLM selection is on par with a fixed core and a non-LLM selector; we present it as a usable interface rather than a source of accuracy. Your own reading, that the agent acts as a router, matches what the numbers show, and the paper now reads that way.
For completeness, the router is not arbitrary: its selected set contains the retrospectively best single tool for 86.0% of targets against 80.4% for the core alone, and the per-target effect runs both ways (on
8inf_u_xadding GraphBind lifts fused r from 0.307 to 0.480; on8cdr_NN_Bbthe extras drop it from 0.093 to −0.001). We report this as coverage behaviour, not as a demonstrated benefit.Figure 1. Agreed, it is too dense. The revision adds an enlarged two-panel version in Appendix A.17 so the labels are legible now, and the final version will replace Fig. 1 with a simplified redraw with fewer boxes and larger type. The caption also corrects a panel label: the fusion is a regressor producing continuous scores, not a classifier (Sec. 3.7).
Two further corrections from this round may interest you, since they affect how the results should be read: the pocket-level evaluation now fixes pocket size from training data instead of from test labels, and with that correction RiboSeer leads Boltz-2 on every pocket metric (Table 3); and the headline reproduces exactly (107/107 targets, mean 0.59308) from one hashed frozen checkpoint, documented in Appendix A.13.
(continued in part 2)