- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces CryoASTRA, a framework for building atomic models from cryo-EM maps, polymer sequences, and ligand identities. It combines sparse semantic voxel features, sequence-aligned EM points, and an atomic diffusion model with local atom-to-voxel attention.
Critical strengths
- The framework integrates protein, nucleic-acid, and ligand modeling and supports whole-map encoding without sliding windows.
- Evaluation on 103 temporally held-out, sequence-filtered maps demonstrates improved protein model completeness and a statistically significant TM-score gain over CryoAtom2.
Critical weaknesses
- Missing core ablations. The paper does not isolate the contributions of semantic voxel conditioning and Heterogeneous BallTree Attention. Consequently, the improvements cannot be clearly attributed to these proposed designs versus pretrained structure-prediction components, MSA information, or other architectural changes.
- Incomplete validation of atomic model quality. The reported backbone and side-chain accuracy and RMSD results are informative, but complementary assessments of stereochemical quality and model–map agreement are missing. Representative geometry and density-fit measures, such as MolProbity and Q-score or local map–model correlation, would strengthen the evidence for improved atomic model quality. Such comparisons should use consistent refinement protocols across methods.
- Limited validation beyond proteins. Mean Accuracy@3 Å is approximately 47% for both nucleic acids and ligands, indicating substantial remaining errors. The absence of comparisons with dedicated ligand-fitting methods further limits conclusions about the practical advantage of unified modeling.
Questions for authors
- With the pretrained backbone and training setup held fixed, how does performance change when semantic voxel features are removed or replaced with raw density, and when Heterogeneous BallTree Attention is replaced with simpler local attention?
- Can the authors report stereochemical quality, local map–model agreement, and stricter atom-position accuracy metrics, including comparisons on commonly modeled atoms, to distinguish improvements in completeness from improvements in coordinate accuracy?
4 comments
2. Common-Atom Accuracy, Q-score & Stereochemistry (W2, Q2)
"Evaluate accuracy on commonly modeled atoms, map fit (Q-score), and stereochemistry / real-space refinement."
- Commonly modeled atoms (§4.2; App. H.5, Table 12): On the intersection of reference protein heavy atoms modeled by both methods in the map frame:
- All 101 protein targets vs. CryoAtom2 (98.26% atom coverage): Common-atom Accuracy@3 Å is 93.16% vs. 91.84% ( pp, 95% CI [−1.72, 4.64]), and common-atom RMSD is significantly lower for CryoASTRA (3.20 Å vs. 6.64 Å, Å, CI [−6.63, −0.72]), while CryoASTRA builds 100.00% vs. 92.48% of polymer residues (TM-score 0.972 vs. 0.855).
- 50 CryoACE-completed targets (Table 12): Common-atom Accuracy@3 Å is 95.00% vs. 95.63% against CryoAtom2 (RMSD 2.40 vs. 2.45 Å), 96.21% vs. 94.59% against EMProt (RMSD 2.02 vs. 2.66 Å), and 94.05% vs. 85.54% against CryoACE ( pp, CI [2.50, 15.62], RMSD 2.78 vs. 4.72 Å).
- Q-score, active-voxel filtering & Phenix refinement (§4.2, §4.4, Table 1; App. H.3, Table 10; App. H.6, Tables 13–14):
- Unfiltered CryoASTRA (MSA) places 100% of sequence residues (including disordered loops without map density), giving full-protein Q-score 0.4691 vs. 0.5230 for CryoAtom2, which narrows to 0.5089 vs. 0.5350 on commonly modeled atoms (Table 13). Filtering to atoms within Å of an active voxel center (91.53% completeness; Tables 1 & 10) lifts full-protein Q-score to 0.5198 (MSA) / 0.5318 (ESM) and conditional Accuracy@3 Å to 94.92%, while lowering RMSD from 3.47 → 2.73 Å.
- Before refinement on 101 targets (Table 13), raw diffusion outputs have higher clashscore (211.68 vs. 32.23), Ramachandran outliers (5.47% vs. 0.58%), and rotamer outliers (19.71% vs. 4.51%) than CryoAtom2. Running paired
phenix.real_space_refineacross 9 targets (45 CryoASTRA samples and 9 CryoAtom2 models; Table 14) reduces CryoASTRA's mean clashscore from 237.0 → 21.5 and raises Q-score from 0.424 → 0.456, while keeping Accuracy@3 Å at 87.3% → 87.2%.
3. Nucleic Acids and Ligands (W3)
"Provide deeper analysis of nucleic-acid and ligand performance and tone down overclaims."
We toned down wording in the Abstract and Conclusion ("models nucleic acids and the input ligands") and added sub-component and reference-aligned evaluations in Table 1, App. H.3 (Table 10), and App. H.4 (Table 11):
- Nucleic acids (14 targets, Tables 1 & 11): CryoASTRA builds 100% of nucleotides (vs. 66.62% CryoAtom2, 50.49% ModelAngelo) with the highest joint NA TM-score (0.561 MSA, 0.608 ESM, vs. 0.424 CryoAtom2, 0.274 ModelAngelo, 0.143 CryoREAD, 0.133 EM2NA), and leads all baselines in Accuracy@3 Å across backbone (46.17% / 50.99% vs. 41.78% CryoAtom2, 16.86% AF3†, 8.93% CryoREAD, 8.69% EM2NA), phosphate (44.35% / 49.46%), sugar (46.80% / 51.52%), and base (48.90% / 53.02%; Table 11).
- Ligands (66 targets, Tables 1 & 10): Among the evaluated builders, only CryoASTRA models proteins, nucleic acids, and input ligands jointly, reaching 47.35% Accuracy@3 Å (53.42% at Å filtering) and 0.3312 Q-score (0.4121 filtered; 0.4270 ESM filtered), ahead of PDB-aligned AF3 (40.72%, Q-score 0.2026) and Protenix-Mini (31.53%, Q-score 0.1349). As noted in §4.2 and §5, ligand accuracy remains limited by lower local resolution and the single ligand semantic channel, so placements should undergo downstream refinement.
Thank you for the additional experiments. The HBA benchmarks clarify its efficiency advantage, and the common-atom comparisons, Q-scores, and stereochemical checks show both the strengths of the protein models and where refinement is needed. These results largely address my concerns, and I will raise my rating from Weak Reject to Weak Accept. Please retain one caveat in the revision: the experiments do not isolate whether retaining voxel-level semantic features during coordinate generation improves accuracy over using density alone.
Thank you very much for re-evaluating our work, for the helpful suggestions, and for raising your score. We agree completely and will ensure that this caveat—that our experiments do not isolate the accuracy gain of retaining voxel-level semantic features during coordinate generation over density alone.
Thank you for the clear and actionable review. To stay well within OpenReview's 5,000-character limit per comment, we split our response into Part 1 (W1, Q1) and Part 2 (W2, Q2, W3).
1. HBA Benchmarks & Sequence-Prior Controls (W1, Q1)