acceptodds
Under review as a conference paper at ICLR 2027

VenusGenome: Genome-Aware Masked Autoencoding for Microbial Genome Representation Learning

Abstract

Genomic-context models have shown that contextualizing protein embeddings can improve predictions of microbial function.We observe that a gene’s protein embedding is partially predictable from the embeddings of neighboring genes in the genome, motivating high-ratio masking that requires reconstruction from sparser genomic context. Genomic modeling must also distinguish biological relationships from assembly conventions, since the ordering of separate contigs does not imply adjacency and strand labels depend on the reference orientation. We therefore introduce VenusGenome, a gene-resolution masked autoencoder that combines visible-only encoding with genome-aware attention incorporating contig boundaries, relative strand orientations, and gene positions. We pretrain this model on a curated corpus of over 1.7 million microbial genome assemblies from isolate and metagenomic sources, using frozen protein embeddings as gene-level inputs. Across genome-level phenotype prediction and gene-level tasks, the resulting frozen representations achieve the highest mean performance among the evaluated models on nine of eleven primary metrics. An exploratory gene-input perturbation analysis further links representation sensitivity to experimentally measured fitness defects. Ablations support the contribution of genome-aware attention and show the benefits of masked autoencoding over BERT-style masked modeling, with further gains from higher masking ratios.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
microbial genome modeling, masked autoencoder, self-supervised learning, foundation models, computational biology

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_HRXzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VenusGenome learns gene- and genome-level representations by reconstructing masked, frozen protein embeddings with a visible-only encoder. The framework combines high-ratio masking with contig-restricted attention, global tokens, relative-strand biases, and within-contig positional information. Across six genome-level phenotype tasks and five gene-level metrics, its frozen representations achieve the highest mean performance among the evaluated models on nine of eleven primary metrics; the reported ablations favor masked autoencoding and the genome-aware attention design on zero-shot PPI ranking.

Critical strengths

1.The paper is well organized and readable. 2. The full model yields competitive results on both organism-level phenotype and gene-function benchmarks.

Critical weaknesses
  1. The contribution of the genome-aware architecture could be further clarified. For example, gLM (Hwang et al., 2024) reconstructs masked protein features from genomic context, BacFormer (Wiatrak et al., 2025) contextualizes ordered proteins across bacterial genomes, and MAE-LM (Meng et al., 2024) excludes masked inputs from the encoder. Against these precedents, explaining what contig-restricted attention and relative-strand biases add would make the architectural contribution easier to assess.

  2. The experimental analysis could better establish which design choices drive the downstream gains. Most masking and attention ablations use STRINGDB PPI ranking, so their effects on phenotype and BGC prediction are less directly tested. Selected ablations on these tasks, with pretraining data and model size held fixed, and genomic-distance controls for PPI would strengthen the attribution of the reported improvements.

Reference:

  • Hwang et al. (2024). Genomic language model predicts protein co-regulation and function. Nature Communications.
  • Wiatrak et al. (2025). A contextualised protein language model reveals the functional syntax of bacterial evolution. bioRxiv.
  • Meng et al. (2024). Representation Deficiency in Masked Language Modeling. ICLR.
Questions for authors
  1. Relative to gLM’s masked protein-feature prediction and BacFormer’s genome-wide contextualization, what is the main architectural contribution of VenusGenome? In particular, how do the contig restrictions and relative-strand biases change the relationships the model can represent compared with their existing structural encodings? It would be useful to distinguish these genome-specific changes from the visible-only encoder–decoder arrangement established by MAE-LM.

  2. Would the conclusions from the masking and attention ablations hold on representative phenotype and BGC tasks when pretraining data and model size are kept fixed? For the PPI results, which evidence channels enter the STRING target score, and do the gains persist after matching pairs by genomic distance or using experimental evidence alone? These results would make the connection between the pretraining rationale and the downstream functional benefits more convincing.

2 comments

Thank you for your positive assessment of the paper's clarity and competitive results, as well as your constructive questions on architectural positioning and gain attribution. We address your two questions below.

[Q1, W1] While visible-only continuous reconstruction builds on gLM, BacFormer, and MAE-LM, VenusGenome's genome-specific contribution is encoding assembly invariants directly into the attention graph via contig-restricted connectivity, global-token routing, and pairwise relative-strand biases.

Thank you for asking us to sharpen this distinction. We do not claim to originate continuous protein-feature reconstruction (gLM), genome-wide protein contextualization (BacFormer), or visible-only encoder–decoder masked autoencoding (MAE-LM). While the visible-only encoder–decoder layout is an adaptation of MAE-LM for compute efficiency and representation utilization, our genome-specific design changes which biological relationships can be represented without assembly-induced confounding:

  • Contig-restricted connectivity + global tokens vs. BacFormer's soft encodings: BacFormer concatenates contigs with contig embeddings, separators, RoPE, and order shuffling, which still permits direct cross-contig gene–gene attention and assigns spurious 1D relative distances across arbitrary contig boundaries in each forward pass. VenusGenome instead decouples local synteny from genome-wide composition in the attention graph: within-contig RoPE and attention strictly model physical gene neighborhoods, whereas cross-contig communication is routed exclusively through order-invariant global tokens (depending only on contig-membership equality rather than contig IDs or concatenation order).
  • Pairwise relative-strand bias vs. gLM's per-gene orientation feature: Because the reference strand of each assembled contig is arbitrary, flipping a contig reverses every individual gene's strand label in gLM. VenusGenome instead injects learned per-head same- vs. opposite-strand biases directly into within-contig attention logits, explicitly distinguishing co-directional pairs (e.g., co-transcribed operons) from opposite-strand pairs while remaining invariant to whole-contig strand flips.

Both encoder and decoder apply these genomic attention mechanisms with separate learned parameters, and our ablations (detailed in Q2) confirm that removing them substantially degrades local gene-organization tasks (e.g., BGC AUPRC drops from 0.660 to 0.503 and PPI from 0.293 to 0.203 when removing contig restriction).

Manuscript change: Updated Section 2 (Related Work) to explicitly credit gLM, BacFormer, and MAE-LM and contrast their encodings with VenusGenome's enforced contig-restricted connectivity, global-token routing, and pairwise relative-strand biases.

[Q2, W2] With model scale and pretraining corpus held fixed (512M, 40k steps), genome-aware priors and 75% masking drive gains on gene-organization tasks (BGC, operon, PPI), and architectural PPI advantages persist across non-proximity channels and all 12 distance/strand strata.

Thank you for suggesting these controls to connect our pretraining rationale with downstream functional benefits. We address all three parts at fixed 512M scale and 40k steps:

(1) Cross-task ablations (fixed 512M scale, corpus, and probe protocol). We evaluated frozen 512M checkpoints on genome-disjoint BGC detection, species-disjoint binary phenotype, and zero-shot operon and PPI ranking (five probe seeds for supervised tasks):

512M / 40k configurationBGC AUPRCPhenotype AUROCOperon AUPRCPPI ρ
Full MAE-M750.6600 ± 0.00190.7893 ± 0.00230.38970.2930
w/o contig attention0.5033 ± 0.00320.7903 ± 0.00400.34680.2029
w/o global tokens0.6768 ± 0.00290.7890 ± 0.00540.37080.2645
w/o strand bias0.6588 ± 0.00330.7944 ± 0.00280.35720.2609
MAE-M150.6344 ± 0.00240.7919 ± 0.00380.39340.2819
BERT-M150.6376 ± 0.00110.7896 ± 0.00350.39700.1920

This pattern directly links pretraining design to task structure: tasks governed by multi-gene spatial organization (BGCs, operons, and PPI) rely strongly on contig boundaries and strand biases (removing contig restriction drops BGC by 0.157 and operon by 0.043), 75% masking boosts BGC over 15% masking (+0.026), and MAE sharply improves zero-shot embedding geometry over BERT on PPI (0.2819 vs. 0.1920). Conversely, coarse genome-level binary phenotypes depend mainly on global gene-repertoire composition (mean-pooled across thousands of genes) and are therefore largely insensitive to local structural ablations.

(2) STRING channels and experimental-only feasibility. The main PPI target is STRING combined confidence across 8,525,221 within-contig pairs in 53 genomes (integrating neighborhood, fusion, co-occurrence, coexpression, experiments, curated databases, and text mining to measure functional-association ranking rather than physical binding). Experimental-only evaluation is infeasible here because only six pairs have nonzero experimental scores (and coexpression has none). Evaluating the five supported non-transferred channels independently (≥20 nonzero pairs per genome; shared sets per column) shows that the full model's gains extend beyond genomic neighborhood to fusion, co-occurrence, and curated database associations:

512M-M75 variantNeigh.FusionCooccur.DatabaseText
Full0.5090.2420.2890.2090.159
w/o contig attention0.4200.1600.2330.1370.122
w/o global tokens0.4850.2180.2810.1980.164
w/o strand bias0.4940.1960.2620.1910.155
Eligible genomes535253507
Included pairs664,84990,9124,106,45150,1993,693

(3) Genomic-distance and strand stratification. Stratifying the 8,525,221 combined-score pairs jointly by gene-index distance and strand orientation confirms that component gains persist within matched strata:

DistanceStrandPairsGenomesFullw/o contigw/o globalw/o strand
1–5Same384,829530.5400.4450.5250.486
1–5Opposite128,367530.2640.2410.2280.280
6–20Same249,379530.2830.1830.2550.230
6–20Opposite133,225530.1880.1550.1680.156
21–100Same386,110530.1260.0670.0850.096
21–100Opposite301,599530.1100.0750.0840.083
101–500Same949,407520.0970.0470.0650.074
101–500Opposite839,068510.0900.0590.0750.067
501–2,000Same1,591,146350.1000.0460.0660.073
501–2,000Opposite1,630,954350.1080.0500.0800.087
>2,000Same925,967130.1050.0770.0930.080
>2,000Opposite1,005,150130.1060.0680.0880.080

Full exceeds w/o contig and w/o global in all 12 strata and w/o strand in 11 of 12 (except distance 1–5, opposite strand). Although absolute correlations attenuate with distance and on short opposite-strand pairs (where co-transcribed operons are rarer), relative gains over all three ablations hold across every long-distance stratum (>100 to >2,000 genes), where local proximity cues are absent.

Manuscript change: Added cross-task ablations to Appendix B.5 (Table 12) and STRING channel/distance analyses to Appendix B.3 (Tables 10 and 11), and clarified the PPI target in Section 4.3.

Reviewer_8SVDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper suggests a model that combines the strengths of using biological sequence embedding with gene sequence ordering. It does this by using frozen mean protein embeddings of each gene, combining them with contig, within contig position and strand information. The models learns attention only within contig, strand specific attention bias and tokens (akin to class tokens) that can attend across all gene representations. Further it makes use of a non-masked-only token encoder structure and masks far beyond 15%. The paper compares results to several baseline models on 0-shot and supervised downstream benchmarks. Ultimately the paper claims to thereby learn functional relationships both from gene organisation and gene/protein representations.

Critical strengths

The architecture/general setup is well-motivated and meaningful to address a real problem in the field, namely how to model entire genomes in a way that incorporates genome structural information such gene ordering, strand orientation and contig boundaries into account. The paper also shows that particularly the contig aware attention contributes to performance. Further the model uses a visible MAE style encoder that reduces the compute spent on embedding the genome especially combined with the more aggressive masking strategy.

The paper also proves that contextualising the protein representations with genome organisational information actually improves performance (although there might be some ablation missing to make this claim more founded, I have addressed this below).

Critical weaknesses
  • This is actually a whole genome protein representation model, not a genome model. However, in prokaryotes did does amount to be similar since the non coding dan content is so low. By extension it also relies on accurate pre-identified gene boundaries. This is a limitation to the method that narrows it scope but I do not count this heavily against the paper since it is primarily concerned prokaryote genomes.
  • Missing ablations. Particularly for how much the genome structural information contribute vs training the model on the same data but without this info (ie. ESM2 embedding only model). This would distinguish what the model can learn from seeing all the protein representations vs what is contributed by the information that distinguishes venusgenome. I am aware some ablations are performed (e.g. removing contig aware information).
  • It is a bit unclear how much biologically meaningful information is learned from the gene organisation apart from sequential proximity. I think this concern could be helped by performing ablations that show performance for biologically meaningful functional relationships that are not stratified by apron co-occurrence and distance. A further test could be to compare Venusgenome against neighbourhood pooled ESM2 representations?
  • several details concerning the pretraining data split, baseline model use and evaluation details are either missing or not complete (eg. how are the models used for the gene level extraction and aggregation done for the NT level comparison models). It also seems different window-sizes are used without explanation.
  • most of the model ablations and pretraining comparisons uses PPI only as a downstream comparison metric. The appendix reports the reconstruction loss also which is fine and potentially better to rely on than a single downstream metric.
  • The 0-shot string PPI interactions: I need this clarified (and have thus also mentioned it below) before I count it as a critical weakness. but it is my understanding that the BacBench from which the PPI pairs are taken (which is from string db ultimately), rely on combined scores that might include interaction that extends to other evidence that only functional protein-protein interaction such as gene neighbourhood and co-expression? if this is the case it is hard to know if this downstream test shows that the model has learned something from he genome structure information other that proximity (which is why a distance/operon stratified results table would be needed).
Questions for authors

I have a list of questions here. Some have already been mentioned above.

  • are the genomes exclusively prokaryotes? if so then why call them microbial (although technically correct also implies a broader application)?

  • gene annotation: this is both a comment on the current setup and a limitation. the setup relies on having gene annotations and these are derived from computational(?) annotations and misannotation could have an impact. how robust is this method to misnnotated gene boundaries and assembly quality?

  • comparison models extraction and aggregation for downstream tasks should be clarified (also mentioned in weaknesses.

    • I might have missed it but i cannot find a clear explanation for how the gene-level extraction and aggregation for these models is done.

    • for the genome level i see the explanation both briefly in 4.1 and A.4. but it could be written more clearly and it also seems that the explanations are only for evo2 and ntv2?

  • for the operon 0 shot task since the model uses both strand direction and relative distance i would like to see the results stratified by strand agreement and distance between pairs. Otherwise it is hard to figure out if the model is simply using relatively simple information like the distance and strand rather than more complex things

  • I would like the data splitting clarified. especially in terms of homology clustering or results according to homology. also what justification is used for chosing the specific splits.

    • it seems the pretraining and downstream data sets/splitting are completely distinct?
    • the pretraining split is genome-split but not species split?
  • what is the specific justification for using changes in only nearby gene representation in section 4.4 rather than genome-level/global representation changes or at least full contig context (or up to 4096). If these are genes that affect entire organism fitness it would be reasonable to assume that this should be reflected in the genome level representation also? The EVO2 baseline seems so significantly differently done to the venusgenome experiment, that it is hard to compare. why are no other models used here like the baseline ESM2? or if a genome-ordering devoid version of venusgenome had been trained (as suggested in critical weakness section) this could be used as a comparison also.

  • PPI evaluation: the paper acknowledges earlier that functional similarity or interaction does not necessarily translate into cosine similarity and yet this is the measurement used to rank PPI. Also how are the string-db connected proteins filtered? many of the associations might be from from other evidence channels which could be related to e.g. operons which could favour this model At least this is my understanding. Please correct if I am wrong. But in any case I would like to see the PPIs results stratified by cross-contig and cross-operon/proximity.

  • figure 5: i would be interested to know how well this gene set corresponds with operon size. does the 33 gene neighbourhood correspond to functional units like the operons or the BGC. Or are nearby genes not in these units also contributing substantially?

  • cross contig attention/global tokens does not seem to have a strong effect on the performance (for PPI as opposed to validation reconstruction loss). could more experiments be done to establish its effect? e.g predicting functionally related genes that are not in the same contig.

8 comments

Thank you for your thoughtful review and detailed questions. We appreciate your positive comments on the motivation of genome-aware attention and the compute efficiency of visible-only MAE. Below, we address each of your ten questions (Q1–Q9) individually and map them to weaknesses (W1–W6) with new analyses and manuscript revisions.

[Q1, W1] We renamed "Microbial" to "Prokaryotic" in the title and explicitly restricted our scope and claims to bacterial and archaeal genomes.

Thank you for pointing out this naming ambiguity. Yes, our study focuses exclusively on prokaryotic microorganisms (bacteria and archaea) using isolate genomes and MAGs. We have replaced "Microbial" with "Prokaryotic" in the title, yielding VenusGenome: Genome-Aware Masked Autoencoding for Prokaryotic Genome Representation Learning, and made the bacterial/archaeal scope and lack of eukaryotic evaluation explicit in the introduction, methods, dataset descriptions, and limitations. Fitness Browser includes 46 bacteria and two archaea; all reported conclusions are limited to this scope.

Manuscript change: Title changed from "Microbial" to "Prokaryotic". Introduction, corpus description, and Conclusion updated with the bacterial/archaeal scope statement and lack of eukaryotic evaluation as an explicit limitation.

[Q2, W1] We added controlled terminal-truncation and contig-fragmentation perturbations alongside observational assembly-quality strata, and explicitly retained low-completeness degradation as a limitation.

Thank you for raising this important question. Because prokaryotic genomes have high coding density, continuous ORFs without complex alternative splicing, and conserved translation initiation signals, computational gene callers are highly reliable in this domain—for example, Prodigal achieves 99.8% gene-identification accuracy (2,437/2,443) and 96.7% exact start/stop accuracy (2,362/2,443) on experimentally validated annotations (Hyatt et al., BMC Bioinformatics 2010), supporting protein-coding genes as a sound modeling unit. To further quantify sensitivity to annotation errors and assembly quality without fine-tuning, we conducted controlled perturbations and observational quality-stratified evaluations on the submitted 1B-M75 checkpoint:

Boundary-shortening and fragmentation. We truncated up to 5 or 10 aa from each terminus of all 53 PPI test genomes (184,723 proteins), or split each contig into blocks of at most 129 or 33 genes (10,000 paired genome-bootstrap draws):

Input conditionPPI ρChange [95% CI]
Unperturbed reference0.2996—
Protein termini: up to 5 aa per end0.2937−0.0059 [−0.0071, −0.0048]
Protein termini: up to 10 aa per end0.2834−0.0161 [−0.0190, −0.0133]
Contig fragments: at most 129 genes0.2937−0.0058 [−0.0079, −0.0038]
Contig fragments: at most 33 genes0.2855−0.0141 [−0.0167, −0.0117]

Terminal truncation reduces correlation only modestly (by 1.97% and 5.39%), indicating reasonable sensitivity to mild boundary shortening and context loss.

Natural assembly-quality strata. Using existing fixed probes (five-seed means) with assembly metadata covering all 153 BGC, 795 temperature, and 2,898 of 3,660 phenotype test genomes:

TaskQuality stratumNESM2-35MBacFormerVenusGenome
BGC≤10 contigs770.3430.5210.650
BGC11–100 contigs340.3730.5540.666
BGC>100 contigs420.3740.5460.695
TemperatureCompleteness <90%110.9380.8880.737
Binary phenotypeCompleteness <90%110.5750.5440.550
Mixed phenotypeCompleteness <90%170.5390.5680.471
Binary phenotypeContamination >5%540.7760.7840.827
Mixed phenotypeContamination >5%580.5510.5080.609

BGC advantage persists across all three contiguity strata, and VenusGenome also leads on contamination >5% genomes, though low completeness (<90%) does not preserve a uniform advantage (in 11 low-completeness temperature genomes, MAE is 3.74 °C vs. 2.60/2.46 °C for ESM2/BacFormer). We explicitly note incomplete genomes as a limitation in the Conclusion.

Manuscript change: Controlled perturbation experiments and observational quality-stratified results added to Appendix B.7 (Tables 16 and 17), and Conclusion updated with annotation-dependence and low-completeness limitations.

Reference: Hyatt et al. (2010). Prodigal: prokaryotic gene recognition and translation initiation site identification. BMC Bioinformatics, 11, 119.

[Q3a, W4] We documented per-model gene-level feature extraction and alignment rules across all baselines in the appendix.

Thank you for pointing out that gene-level extraction and aggregation for the comparison models needed clearer documentation. Each evaluated gene is assigned its own frozen vector; we do not pool the genome first and reuse that vector for all genes. The table below summarizes audited gene-level extraction procedures; dimensions are native feature dimensions used by the downstream evaluations.

ModelDim.Gene-level feature construction
ESM2-35M480Layer-12 residue means excluding special/padding tokens, up to 1,022 residues (shared inputs across gene tasks and gene-context models).
ESM2-650M1,280esm2_t33_650M_UR50D, layer 33, residue means excluding BOS/EOS, up to 1,022 residues; BF16.
gLM1,280ESM2-650M inputs normalized with released statistics and ±0.5 strand features; nonoverlapping segments of up to 30 genes within each contig; final gene token states; FP16.
gLM21,280tattabio/gLM2_650M, protein + intergenic DNA + strand markers, 4,096-token windows (512 overlap), final amino-acid state means (excluding DNA/special tokens).
BacFormer480Frozen ESM2-35M inputs; native special/contig-type tokens; 5,000-gene windows with 512-gene overlap; final gene states, repeated states averaged; BF16.
NT-v2-500M1,024Strand-oriented CDS; final non-special, non-padding token means; BGC/PPI limit 2,048 tokens. (Distinct from the genome-level DNA-window pipeline.)
Evo2-1B1,920Forward contig DNA; 32,768-bp windows with 4,096-bp overlap; blocks.20 states; base states averaged over each gene interval. Unidirectional, no reverse-complement concatenation.
VenusGenome-1B1,408Unmasked ESM2-35M features; 4,096-gene windows with 512-gene overlap; final encoder gene states averaged over repeats; decoder not used.

For the essentiality benchmark, NT-v2 and Evo2 DNA-derived caches were aligned to the protein-label gene order using identifier matching with an order fallback and zero vectors for unmatched genes (167,752 matched out of 169,411 genes across 51 genomes). The Fitness Browser Evo2 baseline uses CDS likelihood rather than these hidden-state embeddings.

Manuscript change: Per-model gene-level extraction details (dimensions, pooling rules, window sizes, overlap handling, and essentiality alignment) added to Appendix A.4.

[Q3b, W4] We clarified genome-level readout aggregation and window-size settings for all evaluated models.

Thank you for highlighting that genome-level readout and window-size explanations were incomplete. We clarified the genome-level readout and windowing rules for all evaluated models rather than only Evo2 and NT-v2. Window sizes follow each model's native context limit or benchmark configuration (4,096 genes for VenusGenome, 5,000 genes for BacFormer, 30 genes per segment for gLM, 4,096 tokens for gLM2, 6,000 bp for NT-v2, and 8,192 bp for Evo2 genome caches vs. 32,768 bp for Evo2 gene caches).

  • VenusGenome-1B: First averages each gene's repeated states across overlapping 4,096-gene windows (512-gene overlap), then equally averages all gene means; global tokens participate in encoding but are excluded from the genome mean-pool readout.
  • ESM2-650M, gLM, and gLM2: Take unweighted means of their protein embeddings or contextual gene states, respectively.
  • NT-v2-500M: Uses nonoverlapping 6,000-bp windows over full contig DNA (omitting tails <50 bp), weighting window means by valid token counts.
  • Evo2-1B: Uses nonoverlapping 8,192-bp windows taking the last-token state from blocks.20 and averaging across windows.
  • ESM2-35M and BacFormer: Mean-pool precomputed protein or contextual gene features (exact legacy cache flags unarchived).

The exact overlapping-window extraction and evaluation scripts, along with the 1B-M75 weights, are also provided in our updated anonymous Zenodo package (see VenusGenome/README.md).

Manuscript change: Genome-level readout rules and window-size specifications for all models consolidated in Appendix A.4.

[Q4, W3] We added a distance- and strand-matched operon conditional pair AUROC diagnostic, showing gains beyond proximity in short-distance same-strand pairs.

Thank you for suggesting this stratification. We added a pair-level operon conditional pair AUROC diagnostic (distinct from group-ranking AUPRC). Each of the 16,288 annotated operon positives (across 11 genomes) is ranked by gene-embedding cosine similarity against 10 same-contig non-operon controls matched on exact gene-index distance, strand agreement, and midpoint DNA distance within 20% (disjoint from positive endpoints and annotated positives; averaged over five sampling seeds, positives, and genomes). Any predictor using distance or strand alone scores 0.500 by construction.

DistanceStrandGenomesPositivesVenusGenomeBacFormerESM2-35M
AllAll1116,2880.6240.5840.517
AllSame1015,4160.6380.5910.535
AllOpposite28400.5210.4990.476
1Same106,6030.6460.6000.541
1Opposite21770.4860.4830.442
2Same103,3610.6520.5610.526
2Opposite11370.4690.4910.501
3–5Same103,7570.6350.5640.525
3–5Opposite22620.5490.4880.512
6–10Same71,2400.5610.6820.509
6–10Opposite11520.5610.5320.388
11–20Same53850.5140.6820.484
11–20Opposite1820.5660.4180.367
21–50Same1280.8250.6440.436

Overall, VenusGenome outperforms both BacFormer (+0.041, 95% CI [0.002, 0.087]) and ESM2-35M (+0.108, [0.074, 0.139]). Gains are concentrated in short-distance same-strand pairs (distances 1–5), whereas BacFormer is higher at same-strand distances 6–20; opposite-strand bins cover only 1–2 eligible genomes and are reported for completeness rather than general conclusions.

Manuscript change: Conditional pair AUROC definition, structural-matching protocol, and distance-by-strand stratification table added to Appendix B.8 (Table 18) as a supplementary operon diagnostic distinct from the main group-ranking AUPRC.

Reviewer_CG4rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VenusGenome is a gene-resolution masked autoencoder for microbial genomes. Each gene becomes a token carrying a frozen 480-dimensional ESM2-35M protein embedding together with its contig, within-contig position, and strand, and the model learns contextualized representations by reconstructing masked gene embeddings. Two observations drive the design. A pilot analysis in Section 1 finds that a gene's embedding is partially predictable from its neighbors, where a linear predictor over the two flanking genes on each side cuts reconstruction error by 15.4 percent against a constant mean predictor versus 8.9 percent for four random genes on the same contig, which the authors read as grounds for masking aggressively. Separately, assembly conventions carry no biological meaning, since contig ordering does not imply adjacency and strand labels depend on the reference orientation. Section 3.1 therefore blocks cross-contig gene attention, adds a learned per-head bias distinguishing same-strand from opposite-strand pairs, applies rotary encoding over within-contig positions, and routes cross-contig information through 64 global tokens, all entering the logit in Equation 1. Only the 25 percent visible genes reach the encoder while an eight-layer decoder reconstructs mean-centered targets. Pretraining uses more than 1.7 million assemblies, combining 75,390 filtered isolate genomes with roughly 1.67 million metagenome-assembled genomes. The frozen 1B encoder is probed against ESM2, gLM, BacFormer, NT-v2, and Evo2 on six species-disjoint phenotype tasks and five gene-level metrics.

Critical strengths
  • (S1) The treatment of assembly structure is the most convincing contribution, and its evidence is unusually consistent. Contig order is an assembly artifact and absolute strand labels flip with the reference orientation, so Section 3.1 blocks cross-contig gene attention outright and encodes strand only as a relative same-or-opposite bias.

  • (S2) The probing protocol is disciplined where representation comparisons normally lose credibility. All encoders stay frozen, and Section 4.1 with Appendix A.2 confirms a shared one-hidden-layer MLP probe, identical optimization settings, fixed splits, and validation-based checkpoint selection, with only the input dimension adjusted. Genome-level tasks use species-disjoint splits keyed to NCBI taxon identifiers and average over five probe seeds.

  • (S3) The supporting analyses are informative and the engineering claims are measured rather than asserted. Figure 5 fixes the encoder and varies only the extraction window, showing that 33 genes already recover roughly 90 percent of the gain over the single-gene control, while Figure 4 adapts the MAE-LM diagnostic to show that masked-autoencoder representations spread variance over more principal directions than BERT-style ones.

Critical weaknesses
  • (W1) The headline comparison is not capacity-matched, so leading nine of eleven metrics does not show that the proposed design produces the gain. The reported model is a 1B encoder, whereas the strongest gene-context competitor in Tables 1 and 2 is BacFormer in its 26M base configuration, which consumes the same ESM2-35M inputs. The 300M Bacformer Large on ESM-C inputs has been public since early 2025 and is not evaluated, and gLM2 is discussed in Section 2 but never benchmarked despite operating at the same gene resolution. Scale alone therefore remains a live explanation, since Figure 3a shows PPI rising from roughly 0.26 at 85M to 0.293 at 512M, a gap of the same order as the margin over BacFormer.

  • (W2) The zero-shot PPI metric carries most of the evidential weight while being partly circular with the inductive bias it validates. Section 4.3 defines it as the Spearman correlation between embedding cosine similarity and the STRINGDB combined score, but that score aggregates neighborhood, gene fusion, and co-occurrence channels, all derived from genomic co-localization, so a model built on contig-restricted attention over ordered genes can score well partly by recovering synteny already present in its inputs. Two internal observations fit that reading, namely that the advantage saturates within a few hundred genes in Figure 5b and that contig-aware attention dominates the ablation in Figure 3c. The concern compounds because all four design questions in Section 5.1 are settled on this one metric, which then reappears as the leading gene-level result in Table 2, making selection criterion and headline evidence the same quantity. The experimental and database channels are independent of gene order and would settle the matter.

  • (W3) The evidence base supporting the reported margins is thin, and split validity is not established. The 75 percent masking ratio improves PPI from 0.282 to 0.293 at 512M but leaves 85M essentially unchanged at 40k steps in Figure 3b, and no 15 percent control exists at the 1B scale used for every number in Tables 1 and 2; since Figure 3 states that all curves are single pretraining runs, a margin of 0.011 cannot be separated from run-to-run variation. No significance test appears anywhere, the quoted spread covers probe seeds rather than pretraining seeds, and several leads are narrow. Table 3 further computes zero-shot PPI on 53 genomes, operons on 11, and essential genes on a 30/11/10 split, with no intervals and no account of how those 53 genomes were selected. Most importantly, the species-disjoint criterion constrains the probe but not the encoder, which may have seen test species among the isolate assemblies during pretraining, and because baselines were pretrained on different corpora any such leakage is asymmetric in VenusGenome's favor.

  • (W4) Each of the three motivating premises is only half closed, which weakens the problem-to-method chain. The step from local predictability to uniform 75 percent masking does not follow, since uniform masking thins neighbors only in expectation whereas span or block masking would remove exactly the cues the pilot identifies, and no such comparison is reported. The case for genome-scale modeling is incomplete, because Figure 5 varies only the extraction window of a model already pretrained at 4,096 genes and cannot separate long-context pretraining from long-context inference, which sits uneasily with 33 genes recovering most of the benefit and with gLM operating on 15 to 30 gene segments.

  • (W5) Notation, figure legibility, and a few omissions. The loss is labeled L_{MAE} in Figure 1a but defined as L_{MGM} in Equation 2, and d denotes the 480-dimensional protein embedding in Section 3.1 while Figure 4 uses it for the encoder hidden size. The ablations appear only as small line plots in Figure 3 and would be checkable as a numeric table, and Figure 2 carries 48 rotated organism labels that are hard to read without positive-gene counts annotated.

Questions for authors
  • (Q1) Can you decompose the zero-shot PPI evaluation by STRING evidence channel, including a variant restricted to the experimental and database channels? Survival of the advantage on evidence independent of gene order would substantiate the representation-geometry claim and raise my score; its disappearance would mean the result should be reframed as recovery of synteny structure.

  • (Q2) Can you add a capacity-matched comparison such as VenusGenome-85M against BacFormer base, and evaluate at least one stronger public gene-context model such as Bacformer Large or gLM2 under the same frozen protocol? Confirmation at matched capacity would address my main reservation about Tables 1 and 2; a reversal would indicate that scale drives the margins.

  • (Q3) How do the masking-ratio and architectural conclusions behave across repeated pretraining runs, and what does a 15 percent control look like at the 1B scale that all headline results use? Variance estimates keeping 75 percent masking ahead would resolve W3, while overlapping intervals would argue for presenting high masking mainly as an efficiency choice, which Table 6 already supports.

2 comments

Thank you for your detailed and constructive assessment. We appreciate your recognition of the assembly-aware attention design and disciplined probing protocol. Below, we address evidence specificity, model capacity, and masking reproducibility, and narrow our claims where the new controls do not support a general conclusion.

[Q1, W2] We decomposed zero-shot PPI across STRING evidence channels and show that VenusGenome-1B leads all baselines on fusion, co-occurrence, and the curated database channel.

Thank you for raising this important concern regarding potential circularity between STRING combined scores and genomic-proximity priors. The target is functional-association confidence, not physical binding. We mapped the original 8,525,221 pairs to STRING v12.0 and evaluated each evidence field itself on its supported subset. Results below are genome-macro Spearman correlations; models share pairs and eligible genomes within each column.

ModelNeighborhoodFusionCo-occurrenceDatabaseText
ESM2-35M0.1050.0590.1370.0060.053
ESM2-650M0.1240.0450.081-0.041-0.018
gLM0.2170.0890.0480.0820.112
gLM20.1960.0660.113-0.1060.113
BacFormer0.2820.1920.1660.0460.097
NT-v2-500M0.1590.1330.0730.0410.051
Evo2-1B0.5660.2000.1060.1620.064
VenusGenome-85M0.4500.1820.2830.1980.164
VenusGenome-1B0.5300.2540.2880.2190.135
Eligible genomes535253507

Experimental scores occur on only six original pairs, with no eligible genome, and coexpression has no supported pairs. Notably, VenusGenome-1B leads all competing baselines on fusion (0.254), co-occurrence (0.288), and the curated database channel (0.219 vs. 0.046 for BacFormer and 0.162 for Evo2-1B over 50 genomes)—a channel that does not rely on genomic co-localization. While these analyses remain conditional on the original combined-score pair universe, they show that the representation advantage extends beyond synteny recovery to curated functional associations.

Manuscript change: STRING evidence-channel and within-contig distance/strand tables added to Appendix B.3 (Tables 10 and 11), and the PPI target clarified as functional-association ranking in the main text.

[Q2, W1] VenusGenome-85M outperforms BacFormer-base on all 11 metrics, and VenusGenome-1B outperforms gLM2 on 9 of 11 main tasks.

Thank you for suggesting both a smaller-model control and a stronger public gene-context baseline. We address both in the combined benchmark table below: (1) comparing VenusGenome-85M (M75, 40k) against BacFormer-base under identical frozen ESM2-35M inputs (Q2a), and (2) evaluating gLM2 (tattabio/gLM2_650M) across all 11 main metrics alongside VenusGenome-1B (Q2b). All runs follow the main evaluation protocol; supervised entries report mean ± sample SD over five probe seeds, and zero-shot entries are point estimates.

TaskBacFormer-baseVenusGenome-85MgLM2VenusGenome-1B
Temperature, Pearson0.824 ± 0.0010.869 ± 0.0060.863 ± 0.0020.862 ± 0.001
Doubling time, Pearson0.185 ± 0.0110.190 ± 0.0420.267 ± 0.0070.336 ± 0.011
MIC, macro Pearson0.254 ± 0.0150.290 ± 0.0090.259 ± 0.0120.270 ± 0.016
AMR, macro AUROC0.614 ± 0.0180.685 ± 0.0070.602 ± 0.0390.682 ± 0.034
Binary phenotype, macro AUROC0.787 ± 0.0010.791 ± 0.0040.781 ± 0.0050.794 ± 0.004
Mixed phenotype, macro F10.421 ± 0.0230.431 ± 0.0150.396 ± 0.0230.457 ± 0.027
Essential genes, AUPRC0.637 ± 0.0090.728 ± 0.0220.736 ± 0.0120.714 ± 0.035
Operon, AUPRC0.3490.3750.3070.388
PPI, genome-macro Spearman0.2010.2580.0640.300
BGC detection*, AUPRC0.438 ± 0.0020.542 ± 0.0030.343 ± 0.0030.532 ± 0.006
BGC category*, macro F10.433 ± 0.0050.448 ± 0.0030.344 ± 0.0070.470 ± 0.006

*BGC rows report results on the updated cluster-family-disjoint (GCF-disjoint) split so all four models are compared on a unified, stricter split; original main-table BGC scores for gLM2 and VenusGenome-1B are in Table 2.

Smaller-model control (Q2a). Using the same frozen ESM2-35M inputs and evaluation protocol, VenusGenome-85M outperforms BacFormer-base across all 11 metrics shown above (with BGC* evaluated on the GCF-disjoint split). This confirms that the transfer gains over BacFormer-base hold even at the smaller 85M scale rather than relying solely on 1B scaling.

gLM2 evaluation (Q2b). Model and tokenizer revisions for tattabio/gLM2_650M are pinned with the encoder frozen; contigs are encoded in 4,096-token windows (512-token overlap) preserving protein sequences, intergenic DNA, and strand markers, and gene states average final-layer amino-acid representations. Across all 11 main-table metrics (Tables 1 and 2), VenusGenome-1B leads gLM2 on 9 of 11 metrics, while gLM2 leads on essentiality and marginally on temperature.

Manuscript change: Capacity comparison between VenusGenome-85M and BacFormer-base added to Appendix A.5 (Table 8); full eight-model results on the main benchmarks and the GCF-disjoint BGC benchmark appear in Tables 1, 2, and Appendix A.2 (Table 5), with the gLM2 extraction protocol documented in Appendix A.4.

[Q3, W3] We added two independent 512M pretraining seeds and a 1B 15%-masking control, confirming the positive direction of the 75%-masking PPI gain across checkpoints.

Thank you for asking about repeated pretraining runs and a 15% masking control at the 1B scale. We added two 512M seed replicates and a 1B M15 control evaluated on the same 53-genome PPI benchmark:

Encoder / pretraining seedM15M75Difference
512M / seed 42 (original)0.28190.2930+0.0112
512M / seed 43 (additional)0.27930.2874+0.0081
512M / seed 44 (additional)0.27730.2845+0.0073
1B / single run per setting0.27910.2996+0.0205

All three 512M seed pairs consistently reproduce a positive 75%-masking gain (+0.0073 to +0.0112; mean +0.0089), and M75 also leads M15 at the 1B scale from 10k through 40k steps. Alongside the 65.2% forward-backward speedup at 4,096 genes, this supports both the direction of the masking gain and its efficiency benefit.

Manuscript change: Three 512M seed comparisons and the 1B 15%-masking control added to Appendix B.6 (Table 14), and Figure 3b caption updated to note single-run curves.

[W5] We improved Figure 2 legibility and added numeric ablation tables.

Thank you for catching these presentation details. In the revised manuscript, we redesigned Figure 2 into a two-column layout with readable organism labels and annotated Pos./total gene counts, and added numeric tables for the architectural and masking ablations in Appendix B.5–B.6 (Tables 12 and 14). We will harmonize the remaining figure notation (LMAEL_{\mathrm{MAE}} vs. LMGM\mathcal{L}_{\mathrm{MGM}} in Figure 1a, and the symbol dd between Section 3.1 and Figure 4) in the final camera-ready figures.

Reviewer_Mz8EOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VenusGenome learns microbial gene and genome representations from frozen ESM2 protein embeddings. It combines visible-only masked autoencoding with contig-restricted attention, global tokens, relative strand biases, and positional encoding. The model is pretrained on more than 1.7 million assemblies. Frozen representations are evaluated on six genome-level tasks and five gene-level metrics. The reported means lead nine of eleven metrics, with an essential-gene tie at displayed precision (Tables 1-2, p. 6). An exploratory analysis relates representation sensitivity to average gene-disruption fitness.

Critical strengths
  1. The frozen encoder performs well across multiple task types. BGC detection improves from 0.536 to 0.667 AUPRC, and genome-macro STRING-score correlation improves from 0.201 to 0.300 relative to the strongest reported baselines. Species-disjoint supervised genome splits and shared probe protocols strengthen this evidence (Tables 1-2, p. 6).

  2. The design has informative internal controls and a practical efficiency benefit. At 512M and matched 15% masking, MAE reaches 0.282 correlation versus 0.192 for BERT-style reconstruction. Contig-attention removal reduces 0.293 to 0.203. Higher masking reduces measured forward/backward time by 65.2% at 4096 genes (Figure 3, p. 8; Table 6, p. 18). These results support the design package, within the stated single-run and timing limits.

Critical weaknesses
  1. The gene-level evaluation leaves important questions about label validity and confounding. BGC positives come from matches to known clusters, while unmatched genes are treated as negatives. An unmatched gene is not necessarily outside a BGC. The paper explicitly defines its PPI task as correlation with STRING association confidence, not direct physical-binding prediction. However, it does not identify the evidence channels or fully specify which protein pairs are included. Distance-matched controls would help distinguish functional association from genomic proximity. These details matter because STRING correlation is also the principal architecture-ablation metric (Section 5.1; Appendix A.1-A.2, pp. 14-15).

  2. The supplied artifacts do not support reproduction of the reported benchmark pipeline. The supplement omits training code, weights, benchmark data, and the full evaluation pipeline. Its inference code uses non-overlapping chunks, whereas the paper averages overlapping windows (Section 3.3, p. 5; Appendix A.4, p. 16; supplement README). The authors disclose this difference, but the exact benchmark extraction path remains unavailable. Species-disjoint probe splits are a useful safeguard. They do not establish whether evaluation species or close homologs appeared during unsupervised pretraining. Accession-level manifests and overlap-stratified results would clarify the scope of transfer (Appendix A.1, p. 14). Neither missing artifacts nor possible unsupervised exposure establishes invalid results or label leakage.

Questions for authors
  1. Which STRING score channels and pair-selection rules were used? Do the architecture gains persist with experimental-only associations and gene-distance/strand controls? Can BGC performance be assessed with curated negatives or cluster-family-disjoint splits?

  2. Can you provide anonymous checkpoints, the exact overlapping-window extraction and evaluation scripts, baseline model identifiers, and accession-level split/pretraining manifests? Please quantify exact and close-homolog overlap, and report results for non-overlapping subsets where available.

  3. Does the 0.011 masking gain persist across independent pretraining seeds, and do the main design gains transfer beyond STRING ranking? Paired test-set uncertainty and complete baseline extraction settings would also clarify small supervised margins (Figure 3, p. 8).

A response that resolves these evaluation concerns could change my recommendation. I would consider raising my rating if the benchmark definitions, overlap analysis, and robustness evidence support the reported gains.

7 comments

Thank you for your constructive feedback on evaluation rigor, label validity, and reproducibility. Below, we address each of your questions point by point with new analyses and manuscript revisions, while clearly noting the scope and remaining boundaries of each result.

[Q1a, W1] We specify the STRING pair-selection rules and show that architectural component advantages persist across evidence channels and within matched distance/strand strata.

Thank you for pointing out the need to specify the STRING evidence channels, pair-selection rules, and proximity controls.

The main PPI target is the benchmark-provided STRING combined score, integrating neighborhood, fusion, co-occurrence, coexpression, experiments, curated databases, and text mining, including transferred evidence where available. We reproduced these labels against STRING v12.0. The benchmark contains 8,525,221 unique undirected pairs of distinct proteins on the same contig across 53 genomes. We merge duplicate/reversed records, apply no additional distance or strand filter, and do not treat missing links as negatives. The metric is the unweighted mean of within-genome Spearman correlations with frozen-embedding cosine similarity. It measures functional-association ranking, not direct physical binding.

Evidence channels across architectural ablations. Experimental-only evaluation is not feasible here: only six original pairs have nonzero experimental scores (and coexpression has none), leaving no eligible genome. We therefore evaluate 40k-step, 512M-M75 (75% masking) architectural variants on the five remaining STRING channels (neighborhood, fusion, co-occurrence, database, and text mining; excluding transferred fields that rely on cross-species orthology). For each channel, we use its own nonzero score as the target on the same 8,525,221 within-contig pairs, requiring ≥20 supported pairs and nonconstant scores per genome (shared eligible sets within each column):

512M-M75 variantNeigh.FusionCooccur.DatabaseText
Full0.5090.2420.2890.2090.159
w/o contig attention0.4200.1600.2330.1370.122
w/o global tokens0.4850.2180.2810.1980.164
w/o strand bias0.4940.1960.2620.1910.155
Eligible genomes535253507
Included pairs664,84990,9124,106,45150,1993,693

These results confirm that our architectural gains are not confined to the genomic-neighborhood channel: the full model leads all component ablations across fusion, co-occurrence, and curated database as well (while text-mining coverage is sparse at 7 genomes).

Distance and strand controls. To control directly for genomic proximity, we stratified the same 8,525,221 STRING combined-score pairs by joint gene-index distance and strand orientation using the same 40k-step, 512M-M75 checkpoints (shared pairs and eligible genomes within each row):

DistanceStrandPairsGenomesFullw/o contigw/o globalw/o strand
1–5Same384,829530.5400.4450.5250.486
1–5Opposite128,367530.2640.2410.2280.280
6–20Same249,379530.2830.1830.2550.230
6–20Opposite133,225530.1880.1550.1680.156
21–100Same386,110530.1260.0670.0850.096
21–100Opposite301,599530.1100.0750.0840.083
101–500Same949,407520.0970.0470.0650.074
101–500Opposite839,068510.0900.0590.0750.067
501–2,000Same1,591,146350.1000.0460.0660.073
501–2,000Opposite1,630,954350.1080.0500.0800.087
>2,000Same925,967130.1050.0770.0930.080
>2,000Opposite1,005,150130.1060.0680.0880.080

The full model exceeds w/o contig and w/o global in all 12 strata and w/o strand in 11 of 12 (the sole exception being distance 1–5, opposite strand). While absolute correlations attenuate at larger separations and in short-range opposite-strand pairs (where operon-level co-transcription is rarer), the full model's relative gains over all three ablations persist across every long-distance stratum (>100 to >2,000 genes), where immediate neighborhood cues are absent. Thus, architectural gains persist within matched distance and strand strata rather than arising merely from pooling across heterogeneous groups.

Manuscript change: Main-model STRING evidence-channel and VenusGenome-1B distance/strand tables added to Appendix B.3; PPI target and pair-selection rules clarified in the gene-level evaluation section and appendix protocol.

[Q1b, W1] We completed the cluster-family-disjoint BGC evaluation using BiG-SCAPE 2.0.3, where VenusGenome-1B retains higher means than all baselines.

Thank you for suggesting this split control. Using BiG-SCAPE 2.0.3 (Pfam 37, glocal alignment, distance cutoff 0.3), we clustered 14,894 annotated regions while retaining the 555/61/153 genome partition. We removed validation regions sharing training families and test regions sharing training or retained validation families from supervised targets (omitted rather than relabeled as negatives, with genome inputs unchanged), leaving 7,591/644/1,699 non-overlapping families.

ModelGCF-disjoint detection AUPRCCategory macro F1
ESM2-35M0.264 ± 0.0020.314 ± 0.003
gLM20.343 ± 0.0030.344 ± 0.007
BacFormer0.438 ± 0.0020.433 ± 0.005
VenusGenome-1B0.532 ± 0.0060.470 ± 0.006

All models share identical retained targets across five probe seeds. VenusGenome-1B maintains a clear lead on both detection and category classification even when all test cluster families are unseen during probe training, confirming that the BGC gains do not rely solely on memorizing shared cluster families.

Manuscript change: GCF-disjoint BGC protocol, cohort statistics, and full eight-model results added to Appendix A.2 (Table 5), with a pointer added to the main BGC evaluation paragraph.

[Q2a, W2] We clarified the overlapping-window benchmark extraction protocol and released an anonymous reproducibility package with scripts, manifests, derived labels, and 1B-M75 weights.

Thank you for highlighting this discrepancy—we agree that the original inference-only supplement does not reproduce the full benchmark. The benchmark extraction uses 4,096-gene windows with 512-gene overlap, averages repeated gene states, and then forms genome readouts, whereas the released example used non-overlapping chunks.

To support full inspection and reproduction, we provide updated anonymized materials in a single anonymous Zenodo draft containing two archives (extract both into the same parent directory and start with VenusGenome/README.md):

  • VenusGenome_anonymous_reproducibility_v2.zip: model implementation, overlapping-window extraction and evaluation scripts, task configurations, downstream split manifests, and derived benchmark labels.
  • VenusGenome_1B_M75_weights.zip: the canonical 1B-M75, 40k-step model weights.

Raw sequence collections and all-model embedding caches are excluded due to size; completed checks and remaining provenance limits are documented in the package's validation reports.

Manuscript change: Appendix A.1 and A.4 updated to clarify how overlapping-window extraction differs from the simplified inference example, and Code Availability updated to point to the anonymized reproducibility package.

[Q2b, W2] We audited downstream accession overlap against our isolate-genome pretraining indexes and reported performance on the accession-absent test subsets.

Thank you for raising this point. Because species-disjoint probe splits do not rule out pretraining exposure, we checked test-genome accessions against our isolate-genome pretraining indexes (since MAGs lack matching assembly accessions) using the submitted 1B model and original fitted probes (BGC refers to the GCF-disjoint test set):

TaskTestMatchedAbsentFull-test scoreAbsent-subset score
Temperature79579500.862 ± 0.001N/A
Doubling time2661181480.336 ± 0.0110.438 ± 0.026
MIC3,519123,5070.270 ± 0.0160.274 ± 0.029
AMR3,519123,5070.682 ± 0.0340.676 ± 0.034
Binary phenotype3,6602,4931,1670.794 ± 0.0040.782 ± 0.007
Mixed phenotype3,6602,4931,1670.457 ± 0.0270.458 ± 0.028
BGC (GCF-disjoint, det.)15315300.532 ± 0.006N/A
Essential genes10820.714 ± 0.0350.577 ± 0.013
Operon11920.3880.318

On tasks with large accession-absent subsets (doubling time, MIC, AMR, and both phenotype tasks), performance holds up closely to the full test cohorts (supervised SDs are over five probe seeds; the two-genome essentiality/operon subsets are descriptive only; PPI lacks assembly accessions). Note that accession absence does not rule out species or sequence overlap from either isolate genomes (e.g., all MIC/AMR test species appear in the isolate-source taxonomy) or the 1.67M MAGs; because all-against-all sequence/homology clustering (e.g., ANI/MMseqs2) across >1.7M assemblies is computationally prohibitive within the rebuttal timeframe, exact sequence-level and close-homolog overlap remain unresolved.

Manuscript change: Added the accession-overlap audit table to Appendix A.1 (Table 4) and clarified that species-disjoint downstream splits do not imply pretraining exclusion.

[Q3a, W3] We added two independent 512M pretraining seeds and a 1B 15%-masking control, confirming the positive direction of the 75%-masking PPI gain.

Thank you for asking whether the masking gain holds across independent pretraining seeds. We added two 512M masking pairs at 40k steps and a 1B mask-15 control. All PPI results below use the same 53 genomes and 8,525,221 pairs.

Encoder / pretraining seedM15M75M75 minus M15
512M / seed 42 (original)0.28190.2930+0.0112
512M / seed 43 (additional)0.27930.2874+0.0081
512M / seed 44 (additional)0.27730.2845+0.0073
1B / single run per setting0.27910.2996+0.0205

All three 512M seed pairs consistently reproduce a positive 75%-masking gain (+0.0073 to +0.0112; mean +0.0089). At the 1B scale, M75 also outperforms M15 (+0.0205 at 40k steps, leading from 10k through 40k steps and slightly lower only at 5k), confirming that the masking benefit is reproducible across seeds and scales.

Manuscript change: Multi-seed 512M and 1B masking comparisons added to Appendix B.6 (Table 14), and Figure 3b caption updated to note single-run curves.

[Q3b, W3] We evaluated 512M architectural and masking ablations beyond STRING on BGC detection, binary phenotype, and operon ranking, showing that structural removal particularly harms BGC and operon performance.

Thank you for suggesting cross-task validation beyond STRING. We evaluated representative tasks beyond STRING at 512M / 40k steps:

512M / 40k configurationBGC detection AUPRCBinary phenotype AUROCOperon AUPRC
Full MAE-M750.6600 ± 0.00190.7893 ± 0.00230.3897
w/o contig attention0.5033 ± 0.00320.7903 ± 0.00400.3468
w/o global tokens0.6768 ± 0.00290.7890 ± 0.00540.3708
w/o strand bias0.6588 ± 0.00330.7944 ± 0.00280.3572
MAE-M150.6344 ± 0.00240.7919 ± 0.00380.3934
BERT-M150.6376 ± 0.00110.7896 ± 0.00350.3970

Overall, these results show that design gains are task-dependent rather than universal across every downstream benchmark: contig restriction and strand bias are critical for local gene-organization tasks (BGC detection and operon ranking), and 75% masking improves BGC over 15% masking, whereas coarse genome-level binary phenotype remains largely invariant across variants and MAE does not uniformly outperform BERT on every task.

Manuscript change: Cross-task ablation table added to Appendix B.5 (Table 12).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response to the Reviewers and Area Chair

Thank you for the constructive feedback. We strengthened the evaluation, clarified the contribution relative to prior work, and revised the manuscript to distinguish the effects of genome-aware attention, masked autoencoding, and higher masking. The main changes are:

  1. Scope and contribution. We changed the title to specify prokaryotic genomes and clarified that our evidence concerns bacteria and archaea. The Related Work now credits gLM, BacFormer, and MAE-LM and distinguishes VenusGenome's contig-restricted attention, relative-strand biases, and global-token routing. We improved Figure 2's legibility, added the 1B masking comparison to Figure 3b, and harmonized the loss and dimensionality notation in Figures 1 and 4.

  2. Stronger baselines and evaluation controls. We added gLM2 to the main comparisons and evaluated VenusGenome-85M against BacFormer base using the same frozen ESM2-35M inputs. On the gene-cluster-family-disjoint BGC split, VenusGenome-1B reaches 0.532 detection AUPRC and 0.470 category macro F1, compared with 0.438 and 0.433 for BacFormer. We also added paired test-genome bootstrap intervals for all 11 primary metrics. The marginal 95% intervals favor VenusGenome on seven metrics against the strongest corresponding baselines in the submitted tables (Tables 1-2, 5, 8, and 15).

  3. What the STRING benchmark measures. We documented pair selection and clarified that the benchmark measures functional-association ranking, not physical binding. We added evidence-channel and distance-by-strand analyses. VenusGenome-1B reaches 0.219 Spearman correlation on the curated database channel, versus 0.162 for the strongest baseline on that channel. Direct experimental support is too sparse for a standalone evaluation. We also added cross-operon and supplementary cross-contig analyses, including a single-window test of global tokens (Section 4.3; Appendix B.3).

  4. Task-dependent effects of pretraining design. At 512M, removing contig-aware attention reduces BGC, operon, and STRING scores. The advantage of MAE over BERT-style reconstruction is clearest on zero-shot STRING ranking. Two additional matched 512M seed pairs and a new 1B 15%-masking control show positive STRING gains for 75% masking. We describe higher masking primarily as an efficiency benefit, not a uniform downstream improvement (Section 5.1; Tables 12-14).

  5. Robustness and gene fitness. We added annotation-boundary and contig-fragmentation perturbations, assembly-quality stratification, and distance- and strand-matched operon controls. For zero-shot gene fitness, we compared against BacFormer under the same perturbation rule (median AUROC 0.673 versus 0.577) and checked both local-window and organism-mean input replacement (Section 4.4; Appendices A.3, B.7, and B.8).

  6. Pretraining overlap and reproducibility. We added an accession audit and distinguish downstream holdout from pretraining exclusion. The temperature and family-disjoint BGC test genomes match the available pretraining-source indices by accession; neither task's labels were used in masked reconstruction. We documented baseline extraction settings and updated the anonymous benchmark supplement. Replaying the original extraction order and rotary-position cache state with the released weights and original aligned ESM2-35M inputs reproduced temperature (Pearson 0.862110 across five probe seeds) and STRING ranking (Spearman 0.299600). These are the two results verified from re-extraction through scoring (Tables 4 and 6; Code Availability).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.