acceptodds
Under review as a conference paper at ICLR 2027

Generalizable RNA 3D Structure Prediction with RNA-Adapted Protenix and Agentic Tree Search

Abstract

Predicting RNA three-dimensional structure remains challenging because of limited experimental data, conformational flexibility, and reduced accuracy on long and topologically complex targets. We adapt Protenix for RNA-focused structure prediction using nucleotide-specific inputs, RNA structural supervision, and multi-conformation training. Our approach has three main components. First, multi-conformation supervision retains up to five experimentally observed reference conformations per RNA sequence and optimizes primarily against the best-matching valid state while retaining an auxiliary reference loss. Second, we augment the curated experimental corpus with quality-filtered synthetic RNA structures generated using a diffusion-inspired structural generation pipeline. Third, we use Agentic Tree Search to select five structures from 15 candidates generated by RNA-adapted Protenix, template-based matching, and Boltz, balancing source-model reliability, conformational diversity, and structural priors. RNA-adapted Protenix is trained with MMseqs2-derived evolutionary information, while benchmark inference is performed without inference-time MSA under the competition environment. The strongest standalone configuration reaches TM-scores of 0.489 on the public benchmark and 0.463 on the hidden private benchmark of the Stanford RNA 3D Folding Challenge. Agentic Tree Search increases performance to 0.684 public and 0.632 private. Synthetic augmentation improves the public score further but does not improve the private score, highlighting the importance of evaluating structural generalization on hidden RNA targets.

Primary area:
generative models
Keywords:
RNA 3D structure prediction, multi-conformation learning, Agentic Tree Search, Protenix, diffusion models, multiple sequence alignment, structural bioinformatics, template-based modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3GMXOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a framework for RNA 3D structure prediction that combines RNA-adapted Protenix, multi-conformation supervision, synthetic structural augmentation, and Agentic Tree Search (ATS) for selecting structural predictions.

The authors fine-tune Protenix using curated RNA structures, MSA-assisted training, and a multi-conformation objective that selects the best-matching experimental reference. At inference time, Protenix, template-based modeling (TBM), and Boltz each generate five candidate structures. ATS then selects five predictions from this pool using source-model reliability weights, structural diversity, and fixed structural priors.

The proposed framework achieves public/private TM-scores of 0.684/0.632 on the Stanford RNA 3D Folding benchmark, compared with 0.601/0.591 for the reported Kabsch-based selection baseline. The paper also provides ablations on multi-conformation supervision, MSA-assisted training, and ATS parameter sensitivity.

Critical strengths

1. Important and practically relevant problem.

RNA 3D structure prediction remains challenging due to limited experimental structural data, conformational flexibility, and complex tertiary interactions. The paper addresses these difficulties in a relevant benchmark setting where multiple structural predictions are required.

2. Promising empirical improvements from combining complementary predictors.

The integration of learned predictors and template-based modeling is well motivated. The reported improvement from 0.601/0.591 to 0.684/0.632 when replacing the simpler selection strategy with ATS is substantial and suggests that candidate selection can meaningfully affect prediction accuracy.

3. Useful ablation studies for multi-conformation supervision and MSA-assisted training.

Appendix N.1 provides matched ablations showing that both multi-conformation supervision and MSA-assisted training contribute to prediction accuracy. In particular, multi-conformation supervision improves the private TM-score from 0.381 to 0.412, and combining it with MSA-assisted training further improves the score to 0.424. These experiments provide useful evidence for the effectiveness of the proposed training choices.

Critical weaknesses

W1. The methodological novelty and independent contribution of Agentic Tree Search are insufficiently established.

ATS is presented as a central methodological contribution. However, Algorithm 1 describes a greedy, diversity-aware candidate selection procedure rather than a conventional tree search algorithm. The method initializes the selected set with fixed structural anchors and iteratively adds the candidate maximizing a weighted combination of structural diversity and distance from these anchors. The authors explicitly acknowledge that the current implementation does not maintain multiple search branches. This procedure appears closely related to existing greedy quality-diversity selection methods, including Maximal Marginal Relevance (MMR)-style re-ranking. While incorporating structural priors and source-model reliability weights may be practically useful, it remains unclear what fundamentally distinguishes ATS from these established approaches.

More importantly, the current experiments do not adequately isolate the contribution of the proposed selection procedure from its heuristic components. The comparison with Kabsch-based top-confidence selection is informative, but it does not establish that ATS outperforms stronger diversity-aware selection baselines under matched conditions. For example, Appendix N.2 demonstrates substantial sensitivity to source-model weights and anchor choices. Since the reference configuration initializes the selected set with Protenix and TBM, two of the final five predictions are effectively predetermined. It is therefore important to disentangle improvements arising from anchor selection, reliability weighting, and diversity-aware greedy selection.

The absence of a TBM-only baseline also makes the contribution of the individual components difficult to assess, particularly given the substantial improvement observed after introducing TBM into the ensemble.

I believe the paper would be considerably strengthened by:

  • Comparing ATS against standard diversity-aware selection methods, such as MMR-style re-ranking, farthest-point selection, and clustering-based representative selection, using the same 15-candidate pool.
  • Reporting TBM-only performance and component-wise ablations of the ATS scoring function.
  • Providing an oracle best-of-15 analysis to quantify the gap between candidate quality and the effectiveness of the selection strategy.

These experiments are important for determining whether ATS represents a distinct methodological contribution or primarily an effective combination of existing selection heuristics.

W2. The experimental evidence does not fully establish the claimed structural generalization.

The title and conclusions emphasize generalizable RNA 3D structure prediction. However, the experiments do not sufficiently distinguish generalization to unseen RNA sequences from generalization to evolutionarily distant sequences or previously unseen structural folds. In Section 4.1 and Appendix A.2, the authors explicitly acknowledge that they do not apply a global sequence-identity decontamination threshold between training and evaluation sequences. Although known evaluation structures are excluded using their PDB identifiers, closely related sequences and structural homologs may remain in the training corpus. This concern is particularly relevant because TBM makes an important contribution to the final framework. Template-based prediction is a legitimate and useful approach, but strong performance on targets with closely related structural templates does not necessarily demonstrate comparable performance on RNA families or folds lacking suitable templates.

The external evaluations on CASP15, RNA-Puzzles, and YangLab30 provide additional evidence, but only aggregate TM-scores are reported, without detailed comparisons against strong reference methods under matched conditions. Similarly, the post-training PDB analysis demonstrates complementary behavior between TBM and Protenix but does not directly evaluate the complete ATS framework. Therefore, I would like to see performance stratified by sequence similarity to the training corpus, structural template availability, and RNA length. In particular, evaluating targets with low sequence similarity and limited template support would help clarify the extent to which the proposed framework generalizes beyond previously observed structural families.

W3. Important aspects of the evaluation protocol and experimental controls require clarification.

The paper reports private benchmark results for numerous configurations, including ATS weight and anchor sensitivity experiments. However, the provenance of these private evaluations and whether private feedback influenced model or hyperparameter selection are not sufficiently clear. Without a precise description of which test set and submission procedure were used, it is difficult to assess the independence of the reported comparisons.

There is also ambiguity surrounding the MSA-free inference setting. The authors attribute the absence of inference-time MSA to the lack of internet access in the Kaggle environment. However, the competition's official dataset description states that precomputed MSA files are made available for hidden test sequences. The inability to perform online MSA generation therefore does not, by itself, establish that inference-time MSA was unavailable.

These issues do not invalidate the reported results, but clarifying the evaluation protocol and the rationale for MSA-free inference is important for interpreting the findings and ensuring reproducibility.

Questions for authors

Q1. Private benchmark evaluation and hyperparameter selection. Could the authors clarify how the reported private benchmark scores were obtained? Specifically, were they produced through official final Future Data evaluation, post-competition late submissions, or another evaluation procedure? Were all configurations evaluated on the same private target set? Additionally, were private benchmark results used to select ATS source-model weights, structural anchors, or other hyperparameters? Clarifying the tuning procedure and evaluation chronology would strengthen confidence in the reported results.

Q2. MSA availability and inference-time evaluation. The competition's official data description indicates that precomputed MSAs are available for hidden test sequences, even though online sequence database searches are unavailable. Could the authors explain why these precomputed MSAs were not used during inference? Was this a deliberate methodological choice, a model compatibility issue, or a computational constraint? If feasible, comparing inference with and without the provided MSAs would help clarify the significance of MSA-assisted training followed by MSA-free deployment.

Q3. Multi-conformation supervision and the definition of distinct conformational states. The proposed multi-conformation objective minimizes the structural loss over available references, with an auxiliary loss on the first reference. While this allows predictions to match alternative experimental states, it does not explicitly encourage different generated predictions to cover distinct conformations. Could the authors evaluate whether multi-conformation supervision actually improves conformational diversity or coverage, rather than only best-of-five prediction accuracy? Relatedly, Appendix A reports that 3,009 of 3,203 filtered unique sequences exhibit multiple experimentally observed conformations. This proportion appears unusually high and would benefit from further explanation. How does the criterion based on structural variation distinguish biologically meaningful alternative conformations from minor experimental or geometric differences? Reporting the distribution of pairwise TM-scores among reference conformations would help clarify the nature of the multi-conformation training data.

Q4. Synthetic structural augmentation and controlled comparisons. The paper reports generating 3,982 synthetic structural candidates and retaining 400 for training. However, the generation procedure and quality filtering criteria are not described in enough detail for independent reproduction. Could the authors provide the generation architecture, training procedure, structural filtering criteria, and selection thresholds? Moreover, the transition from Merged-v1 to Merged-v2 simultaneously introduces synthetic augmentation and removes approximately 40 CASP16-derived sequences. A controlled experiment changing only the synthetic augmentation component would be necessary to isolate its contribution. Such an experiment could also help explain why synthetic augmentation improves the public score but not the corresponding private score.

Q5. DRfold and ResidueGNN evaluation. The paper motivates ResidueGNN by stating that DRfold does not directly provide the C1′ coordinates required by the benchmark. However, the original DRfold framework includes an atomic coordinate reconstruction procedure, and its final structural outputs can contain C1′ coordinates. Could the authors clarify which DRfold representation is used as input to ResidueGNN? More importantly, have they compared ResidueGNN against directly extracting C1′ coordinates from the final structures generated by the official DRfold pipeline? This comparison is necessary to establish the independent contribution of ResidueGNN.

Q6. Positioning relative to recent methods and reproducibility. Recent methods such as DRfold2 and trRosettaRNA2 are particularly relevant to RNA 3D structure and conformer prediction. Although fully matched comparisons may be challenging, additional discussion or comparisons under a common evaluation protocol would help establish the significance of the proposed approach. For ATS, could the authors also provide the exact values and tuning procedure for the diversity and structural-prior coefficients, together with the computational overhead of generating and selecting the 15 candidates? Releasing the candidate structures, prediction files, and ATS selection implementation would substantially improve reproducibility.

1 comment

We thank the reviewer for recognizing our empirical improvements and for the constructive feedback. We address W1–W3 and Q1–Q6 below.

W1: ATS novelty and baselines

ATS uses greedy, state-dependent selection rather than multi-branch tree search. It selects five of 15 Protenix/TBM/Boltz candidates using source reliability, Kabsch diversity, and structural priors, updating scores after each selection.

Matched comparisons on the same candidate pool:

MethodPublicPrivate
Full ATS0.68400.6320
No anchors0.67490.6351
Equal weights0.67970.6331
No prior penalty0.67780.6315
Anchored MMR0.68200.6203
Anchored FPS0.65280.6244
Kabsch selection0.60100.5910
FPS0.59480.5934
K-medoids0.55550.6073
MMR0.55190.5271

ATS exceeds anchored MMR by +0.0020/+0.0117. TBM-only achieves 0.570/0.530 versus ATS 0.684/0.632. Anchor removal and equal weights slightly improve private results, so reference parameters are not universally optimal. Oracle best-of-15 cannot be calculated from aggregate private scores without native structures.

W2: Homology-aware generalization

We analyzed 75 external records: YangLab30 (30), RNA-Puzzles (20), CASP15 (12), and post-cutoff PDB (13).

BLAST comparisons requiring ≥80% bidirectional coverage found only 1/75 records at ≥70% identity and none at ≥95%.

US-align comparisons normalized by evaluation-reference length found 9/75 matches above TM=0.45 and 4/75 above TM=0.50 after retrospective filtering. Thus, 66/75 (88%) have no detected match above 0.45.

Rfam/Infernal identified 11 shared-family records, 40 annotated without detected overlap, and 24 unassigned. Among annotated records, 40/51 (78.4%) have no detected shared family.

These results measure detected similarity, not evolutionary independence or novel folds. Retrospective filtering does not alter trained checkpoints. Details appear in Appendix N.1. Prediction stratification by similarity and length remains desirable.

W3/Q1: Private evaluation

Original models, ATS weights, and anchors were selected using public feedback and local validation before future private evaluation. Organizers evaluated fixed submissions on future targets unavailable during development.

Additional ablations were retrospective, following release of aggregate private scores, and did not influence original configurations.

Q2: MSA-free inference

Organizers supplied precomputed MSAs. We tested them but observed no meaningful accuracy improvement, while loading and featurization increased overhead. MSA-free inference was selected under P100 runtime constraints, while training remained MSA-assisted.

Q3: Multi-conformation supervision

Experimental coordinates were grouped by sequence; multiple records may represent chains, models, or deposits rather than distinct biological states.

Alternatives were ordered by C1′ TM-score relative to a fixed primary reference. The objective is:

L_multi = min_k L(pred, GT_k) + 0.1 L(pred, GT_1).

Supervision improves TM from 0.420/0.381 to 0.431/0.412 without MSA. It accommodates alternative references but does not explicitly maximize output diversity. Reference-pair TM-score distributions remain unreported.

Q4: Synthetic augmentation

We used public UW synthetic structures, not a generator trained by us. Of 3,982 candidates, 400 were retained through BLAST, RMSD/TM-score comparisons, and Rfam-informed selection of non-redundant examples from existing training families.

This assesses diversity, not experimental correctness. Merged-v1 achieves 0.448/0.424 versus Merged-v2 0.489/0.463, but Merged-v2 also removes approximately 40 CASP16-derived sequences. A controlled augmentation-only experiment remains necessary.

Q5: DRfold2 and ResidueGNN

ResidueGNN converts intermediate DRfold2 MSA2XYZ coordinates (P, C4′, N1/N9) directly into C1′ coordinates.

Official DRfold2 can reconstruct C1′, but we avoided full reconstruction/refinement due to computational and dependency overhead in the offline P100 environment. ResidueGNN is a separate baseline, not part of ATS. Direct comparison with official reconstruction remains necessary.

Q6: Coefficients and reproducibility

We expanded discussion of DRfold2 and trRosettaRNA2; matched comparisons remain desirable.

ATS uses source weights (0.75, 0.15, 0.10), diversity coefficient 1.0, prior penalty 0.5, and anchors Protenix #2 + TBM #1.

Score(c|S) = w_model(c)[1.0 D(c,S) - 0.5 P(c)]

Protenix/Boltz generation dominates runtime; ATS adds limited CPU overhead. We intend to release implementation details and permitted artifacts.

We believe these additions strengthen the methodology and evaluation and respectfully invite reconsideration.

Reviewer_xryhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a novel approach to RNA 3D structure prediction. The proposed method is based on an adapted Protenix as main predictor that is specifically trained on a new PDB-derived RNA corpus, while the final predictions are selected from multiple predictions of multiple sources, protenix, template matching, and Boltz, using an agentic tree-search procedure while samples are weighted by source. The authors further propose several training advancements like synthetic RNA data, and multi-conformation training. The main evaluation follows the Stanford RNA 3D structure prediction Kaggle competition protocol, using a public and a held-out (unseen) private test set. Further evaluations on CASP15, CASP16, RNA-Puzzles, and YangLab30 targets are shown in the Appendix. The performance for the main evaluation on the Kaggle competition data is quite impressive. The full ensemble including ATS selection improves over AlphaFold 3 by nearly 0.2 TM. Overall the performance appears strong, also for the standalone protenix version.

Critical strengths
  • The performance of the presented approach is strong. The standalone adapted protenix version surpasses AF3 TM on the public Kaggle set, while being on par on the held-out data. Adding ensembling with ATS selection strongly improves performance by nearly 0.2 TM.
  • The ATS seems to work very well.
Critical weaknesses
  • The main weakness is the missing homology aware data processing pipeline. The authors use a temporal cutoff, and explicitly exclude/remove specific test targets (CASP16 removed from v2) from the training set. The authors state this issue clearly several times in the manuscript, however, sequence and structure aware splitting is known to be required to avoid data leakage between train and test data. While this might be less important for the main evaluation using the original Kaggle protocol (and since the AF3 baseline also does not consider homology aware data splitting), at least the additional evaluations should consider train/test homology. Otherwise the "Generalizable" in the title is not justified in my opinion. I would rather prefer fewer evaluations in the Appendix on correct data splits than many evaluations that are non-informative for generalization performance. I would first suggest that the authors test to remove test samples (e.g. from CASP15/16, Puzzles, YangLab30) based on similarity (sequence and structure) to see how many samples would survive. If there remain some, even a small generalization experiment would be much better than evaluations within distribution only. This could avoid expensive re-training from scratch. Additionally, at least an analysis of train/test similarity would help to understand the performance gains better.
  • There are no limitations in the main paper. Actually Limitations is the very last section of the Appendix.
Questions for authors
  1. In the loss formulation in section 3.3, how is the order determined to decide which sample contributes to the auxilliary loss L1\mathcal{L}_1?
  2. Maybe I missed it, but how are the initial structural priors for the ATS procedure obtained/selected?
  3. How were the weights for the sample sources determined for the ATS? Are the values shown in the sensitivity study in Table 8 all tested configurations?
Reviewer_vsKiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adapts Protenix for RNA 3D structure prediction through multi-conformation training and synthetic structural data. It also introduces Agentic Tree Search to select among candidates from multiple sources, and evaluates the approach on public and private RNA folding benchmarks.

Critical strengths

None

Critical weaknesses
  1. The quality of the synthetic training data is not sufficiently characterized. The paper needs a more systematic evaluation of the generated structures and their effect on generalization. Without that analysis, it is difficult to assess whether synthetic augmentation provides reliable improvements.

  2. The benchmark comparison does not sufficiently establish the method’s performance relative to strong RNA structure prediction systems. Comparisons with the original Protenix and AlphaFold 3, under comparable evaluation conditions, would help establish the benefit of the proposed adaptations.

Minor presentation comment: The arrows overlap in panel 4 of Figure 1, labeled “RNA-Adapted Protenix + Agentic Tree Search,” making the workflow difficult to follow. This is a presentation issue rather than a primary reason for rejection.

Overall, I recommend rejection because the quality of the synthetic data and the comparative benefit of the method are not sufficiently established.

Questions for authors

How is synthetic structure quality measured, and how does it relate to downstream prediction accuracy? What comparisons with the original Protenix and AlphaFold 3 are available under the benchmark’s input constraints?

1 comment

We thank the reviewer for the constructive feedback. We have clarified synthetic-data provenance and family-aware selection, added an original-Protenix baseline, distinguished organizer-provided AlphaFold 3 references from our submissions, and improved Figure 1.

W1/Q1: Synthetic structure quality and generalization

We used the publicly released University of Washington Synthetic RNA Structures dataset, not a generator developed or trained by us. From 3,982 candidates, we selected 400 through a two-stage procedure: (i) BLAST sequence comparisons and RMSD/TM-score structural comparisons to assess similarity and redundancy against experimental training structures; and (ii) Rfam covariance-model annotation to preferentially retain structurally non-redundant candidates from RNA families already represented in training. The objective was to expand conformational diversity within existing families.

We agree that structural non-redundancy and family membership do not establish physical correctness or experimental accuracy. Accordingly, revised Section 3.4 characterizes this as family-aware, diversity-oriented selection, not experimental validation of generated structures.

For downstream performance, Merged-v1 Protenix with multi-conformation supervision and training-time MSA reaches 0.448 public / 0.424 private TM, while Merged-v2 reaches 0.489/0.463, a gain of +0.041/+0.039. Crucially, Merged-v2 both adds 400 synthetic examples and removes approximately 40 CASP16-derived sequences. This comparison therefore does not isolate the causal effect of augmentation, which would require a controlled run changing only synthetic inclusion.

At the ensemble level, ATS configurations score 0.684/0.632 and 0.692/0.620, indicating that higher public performance need not transfer to private performance. We have explicitly qualified our augmentation claims accordingly.

W2/Q2: Original Protenix and AlphaFold 3 comparisons

We evaluated unmodified pretrained original Protenix through the competition submission framework, under its inference and resource constraints, separately from ATS. AlphaFold 3 scores were provided by the organizers, not generated by our pipeline; their full-sequence evaluation is not strictly compute-matched to our submissions.

MethodPublic TMPrivate TM
Original Protenix (zero-shot)0.3800.370
RNA-adapted Protenix0.4890.463
AlphaFold 3, organizer (MSA)0.3490.464
AlphaFold 3, organizer (no MSA)0.2990.376
TBM-only0.5700.530
Kabsch selection0.6010.591
Full ATS0.6840.632

Compared with original Protenix, RNA-specific adaptation improves public/private TM by +0.109/+0.093, independently of ATS. The adapted standalone model exceeds the MSA-enabled organizer AlphaFold 3 reference on public data and performs nearly identically on private data (0.463 versus 0.464).

Full ATS exceeds the MSA-enabled AlphaFold 3 reference by +0.335/+0.168. It also improves over TBM-only by +0.114/+0.102 and over Kabsch selection by +0.083/+0.041, demonstrating the benefit of combining complementary candidate sources and adaptive selection.

Conclusion

These revisions directly address the requested baseline comparisons and clarify both the motivation for synthetic augmentation and the limits of the available evidence. We believe they strengthen the empirical evaluation and respectfully invite reconsideration of the assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.