- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+2)
Read review
Paper summary
The paper proposes an unbiased stochastic algorithm for estimating prefix probabilities of transduced language models (TLMs), which compose a pretrained source LM with a finite-state transducer. The authors sample source prefixes without replacement and apply Horvitz–Thompson reweighting, and they prove unbiasedness provided the run halts almost surely. They further propose an adaptive-budget variant that shrinks the particle count as mass accumulates and uses Russian roulette to guarantee almost-sure halting. They evaluate the method in 5 experimental settings covering text and DNA-to-amino-acid conversion.
Critical strengths
The paper tackles this question by providing an algorithm with a solid proof, and evaluates the method in 5 experimental settings covering text and DNA-to-amino-acid conversion.
Critical weaknesses
The evaluation mainly measures how accurate the estimates are, but it does not test whether the transduced model itself is useful. For example, in the DNA-to-amino-acid setting, there is no comparison with a model trained directly on amino acid sequences, and no downstream task.
Questions for authors
- Could you compare against strong existing baselines, for example, a model of similar size that performs well on a related downstream task, to evaluate how useful the transduced model is?
- Minor formatting issue: on line 468, a space is missing after the period in "direction.To".
We thank the reviewer for the review and for catching the typo on line 468, which will be fixed in the revision.
W1 and Q1: usefulness of the transduced model, and comparison with a model trained on amino acids.
We would like to point out that the contribution of this paper is methodological: an estimator for the probabilities of transduced language models, with proofs that it is unbiased and that it halts. Our goal is not to present a scientific application, and we do not claim that a transduced model outperforms a model trained on the target units. The case for transduced models themselves is made in the two works we build on. Snæbjarnarson et al. (2026) introduce them as a way to obtain a model over the units a task needs, such as bytes, words or amino acids, from a pretrained model without retraining, and Kiegeland et al. (2026) apply them in psycholinguistics to predict human reading times, the analysis we repeat in §4.3. What our paper adds is the ability to compute these probabilities without bias, and at lengths where the existing methods fail (in Fig. 6 the capped baseline returns zero after 16 amino acids).
We agree that an example for the DNA-to-amino-acid setting helps to show what this makes possible, so we ran a small proof of concept. We took 198 randomly sampled ClinVar missense variants (single amino-acid changes in human proteins) and classified them as pathogenic or benign with the 7B-parameter Evo2 DNA model transduced to amino acids. It scores a variant by how much less probable the changed sequence (the new amino acid and the five after it) is than the original, given up to 1,000 bases of upstream DNA. With no training, it reaches an AUROC of 0.83, against 0.81 for the same DNA model scored directly on DNA and 0.86 for ESM-1v, an existing model trained directly on amino-acid sequences and a widely used baseline for this task (650M parameters). At this sample size none of these differences is distinguishable from zero. Note that this is an illustration of the kind of study the estimator enables and we would look forward to seeing it studied in dedicated future work with domain experts.
More generally, a transduced DNA model gives probabilities that neither a DNA model nor a protein model provides on its own: (i) the probability of a protein given its surrounding DNA, for example the sequence upstream of the gene, which a protein model cannot condition on; (ii) the probability of a protein summed over all of its synonymous DNA encodings, where a DNA model alone scores only one encoding; and (iii) by Bayes' rule, the probability of a particular DNA encoding given the protein it encodes, the quantity relevant to choosing codons in gene design. Again, we hope these open avenues for future studies; pursuing them is beyond the scope of this paper, which develops and evaluates the estimator itself.
Soundness.
The paper's contribution is a method with guarantees, and each of its claims is supported as follows.
We appreciate that the review describes the proof as solid and does not question any of these claims. Its concern is that the usefulness of the transduced model was not evaluated, which the prior work and the example above address. We hope this allows you to reconsider the soundness score.