- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
EMBER predicts TCR–pMHC binding from independently folded components without requiring a docked complex. It separately encodes their structural and sequence features, then uses cross-attention between CDR3β and peptide residues to predict binding. The authors introduce assembly-induced interface corruption (AIC) to describe how docking errors can distort the information passed to downstream models. Their approach removes dependence on the relative pose and allows component structures to be reused across candidate pairs.
Validation includes structural analysis of 49 crystal complexes and binding prediction on 45,909 pairs using five epitope-disjoint folds, six retrained baselines, and four structure-prediction tools. The authors report improved prediction performance, greater consistency across structure sources, and lower preprocessing costs. However, the experiments do not isolate whether avoiding docking error causes the improvement, particularly because the complex-input ablation also changes component conformations and graph processing. The binding evaluation also mixes assay-derived and generated negatives, while the exact-peptide split does not establish separation of similar peptides or related receptors. These limitations leave the proposed explanation for the gains less convincing.
Critical strengths
1.Reusing component structures offers a clear computational advantage. EMBER predicts each TCR and pMHC structure once and reuses them across candidate pairs, avoiding a separate complex prediction for every pairing. This matters for screening many receptors against many targets, where structure generation can become expensive. I consider this the paper’s strongest contribution because the practical benefit remains meaningful even if the explanation based on AIC needs further evidence.
2.The reported improvements persist across several structure sources. The authors evaluate four structure-prediction tools and retrain the baselines on the same data splits. EMBER improves the average binding-prediction results across these settings and produces more consistent rankings across structure sources. This gives some confidence that its performance is not dependent on one particular folding pipeline and supports the usefulness of the approach within the evaluated benchmark.
Critical weaknesses
1.The experiments do not show that avoiding AIC explains the improvement. The 49 crystal complexes demonstrate errors in reconstructing known binding interfaces, but do not establish how those errors affect binder–non-binder discrimination. The complex-input ablation also changes component conformations, node features, and cross-molecular message passing together. The missing control is to co-fold the complex, extract its components, and process them separately through the original EMBER architecture. A pose-only perturbation or correction would then test whether docking error actually causes prediction errors. This is central to my assessment because AIC is the paper’s main explanation for its advantage, yet the current experiments cannot distinguish that explanation from other architectural or input differences.
2.The binding benchmark leaves important questions about specificity and generalization unresolved. The negatives mix assay-derived non-binders with generated mismatches, without provenance allowing separate evaluation. Approximately 58% of negative rows come from receptors never observed in positive pairs, raising the possibility that receptor-background differences help distinguish the labels. Meanwhile, the split holds out exact peptide identities but reports neither sequence-similarity filtering nor an audit of receptor overlap across folds. These observations do not prove that EMBER exploits shortcuts, but the paper needs controls that exclude them. A receptor-only baseline, similarity-aware evaluation, and an independent assay-grounded test with positive and negative partners for the same receptors would be particularly informative. Without these, I cannot tell how much the reported gain reflects learning TCR–pMHC specificity that generalizes beyond the curated dataset.
Questions for authors
-
Do you have results from co-folding the complex, extracting its components, and processing them separately through the unchanged EMBER architecture? The reported complex ablation also changes features and graph connectivity. This missing comparison would help determine whether independent folding matters, while a pose-only intervention would establish whether docking error actually causes the downstream performance loss. Either result could materially change my assessment of the main contribution.
-
Were any peptide- or receptor-similarity filters applied before constructing the epitope-disjoint folds? Please quantify exact receptor overlap and the nearest-training-sequence similarities of test peptides and CDR3β sequences. This would clarify whether the reported results mainly reflect transfer among related examples or generalization to substantially different receptors and peptides.
-
How well does a TCR-only model perform under the same protocol? Given that approximately 58% of negative rows involve receptors never observed in positive pairs, this control would help determine how much performance is achievable without identifying the correct pMHC partner. If available, results comparing measured positive and negative partners within the same receptor would provide stronger evidence that EMBER learns pair specificity.
4 comments
Reviewer's Comment: the split holds out exact peptide identities but reports neither sequence-similarity filtering nor an audit of receptor overlap…. // Were any peptide- or receptor-similarity filters applied before constructing the epitope-disjoint folds ….
Response: Appendix F.1 now sets out the composition of STAG's negative samples and the filtering STAG applies to them before release. It passes the dextramer block through ITRAP, excludes the alleles it finds non-specific, and prunes on CDR3alpha and CDR3beta kernel similarity at 0.90 for non-binders and 0.95 for binders. We added no further similarity filter before the split, the folds hold out exact epitope identity, and both audits have been added. Table 15 counts receptor reuse, where a test receptor is reused when it also appears in one of the three training folds.
| Fold | Of test receptors | Of test rows | Of binder rows | Shared (receptor, epitope) pairs |
|---|---|---|---|---|
| 0 | 0.579 | 0.609 | 0.353 | 0 |
| 1 | 0.661 | 0.684 | 0.755 | 0 |
| 2 | 0.678 | 0.706 | 0.792 | 0 |
| 3 | 0.716 | 0.751 | 0.804 | 0 |
| 4 | 0.679 | 0.706 | 0.715 | 0 |
| Mean | 0.663 | 0.691 | 0.684 | 0 |
Receptors are reused and no (receptor, epitope) pair is shared with training in any fold. The split is on epitopes by design, and Sec. 5.1 now states the reason, that a screen ranks receptors against one epitope. The same receptor is then a candidate for the next epitope. Table 16 gives the Levenshtein distance from each distinct test sequence to the nearest sequence of the three training folds, pooled over folds, with the last five columns giving the share of test sequences at that distance.
| Sequence | Distinct | Median | Mean | 0 | 1 | 2 | 3 | >=4 |
|---|---|---|---|---|---|---|---|---|
| Test epitope | 343 | 4 | 3.77 | 0.000 | 0.259 | 0.111 | 0.061 | 0.569 |
| Test CDR3beta | 36,126 | 0 | 0.82 | 0.660 | 0.069 | 0.130 | 0.091 | 0.049 |
The two axes are not symmetric: no test epitope occurs in training and 0.569 sit at distance 4 or more, while 0.660 of the test CDR3beta sequences occur verbatim in training. For a similarity-aware evaluation, Appendix F.6 reruns the screen on the test epitopes at distance 4 or more. Under ESMFold2 EMBER reaches an enrichment of 2.264 at ten candidates against 2.158 for STAG-LLM and 2.100 for the strongest sequence baseline, the remaining baselines fall between 1.982 and 2.100, and EMBER's lead is maintained at distance.
Reviewer's Comment: How well does a TCR-only model perform under the same protocol….
Response: This control is now Appendix F.7, Table 28, on the rows of Table 6.
| Scorer | macro-AUROC | macro-AUPRC | micro-AUROC | micro-AUPRC | [email protected] | MCC* |
|---|---|---|---|---|---|---|
| EMBER | 0.759 | 0.584 | 0.670 | 0.372 | 0.273 | 0.304 |
| No peptide | 0.707 | 0.515 | 0.630 | 0.300 | 0.235 | 0.270 |
| Receptor identity only | 0.456 | 0.199 | 0.622 | 0.203 | 0.000 | 0.210 |
No peptide is EMBER retrained and tested with one fixed pMHC on every row, so it never sees the peptide's specificity. Receptor identity only is not trained at all. It assigns a row to class one when its TCR is a binder in some row of the three training folds, and scores the class by its positive rate on the validation fold. The receptor background does carry the pooled signal the 58% figure predicts, and the lookup reaches micro-AUROC 0.622 when all epitopes of a fold are pooled. Within an epitope its macro-AUROC is 0.456, below chance. Its [email protected] of 0.000 does not say the lookup cannot discriminate, only that a fixed cut at 0.5 does not suit a score that takes two values, and the reading we would point to is its MCC* of 0.210 against our 0.304. The shortcut therefore does not account for the whole of the model's discrimination. We cannot assemble measured positive and negative partners for the same receptor from this release, and the closest the paper comes is the within-epitope reading of Table 3. We agree that a second sampling strategy would strengthen the evidence, and regenerating its inputs is what we could not complete during the discussion period. The three baselines that take the predicted complex as their input would each need a new complex folded for every new pair. Table 7 puts the 45,909 pairs at 1,602 GPU-hours of inference for TCRmodel2, and the 38,538 non-binding rows are 84% of that. Generating the structures for a new negative set is close to 2,500 GPU-hours on an A100-SXM4-80GB, which is over three months and not likely be done during the rebuttal period.
Reviewer's Comment: an independent assay-grounded test with positive and negative partners for the same receptors would be particularly informative.
Response: We thank the reviewer for asking for an assay-grounded test. Our earlier replies said that regenerating the inputs for a second negative set over the curated collection would cost time beyond the discussion period. We therefore ran the comparison on a smaller collection, TCRvdb (Messemaker et al., 2026), with ESMFold2, the tool the paper's ablations use because it leads every interface and placement metric on the leakage-controlled targets of Table 12. TCRvdb holds 1,779 pairs over five epitopes, and its TCR-pMHC pairs were re-tested experimentally under a standardized readout, so both its binders and its non-binders are determined by experiments. Every row here carries one assay provenance, so the mixture of assay-derived and generated non-binders that Appendix F.1 could not separate does not arise. We evaluate EMBER and other baseline methods under the same protocol as in the paper to guarantee no leakage. A fold here is a single epitope, so a fold's score and the validation signal that selects the checkpoint both rest on one peptide, and the numbers carry more spread than the curated set gives. Each cell is the mean over the five folds, with the standard deviation over seeds in brackets.
| Method | AUROC | AUPRC | [email protected] | MCC* |
|---|---|---|---|---|
| STAG-LLM | 0.5891 (0.0072) | 0.5609 (0.0154) | 0.0618 (0.0512) | 0.2279 (0.0191) |
| SageTCR | 0.5726 (0.0189) | 0.5632 (0.0083) | 0.0924 (0.0160) | 0.2045 (0.0107) |
| Abeer | 0.4472 (0.0028) | 0.4665 (0.0066) | -0.0565 (0.0136) | 0.1211 (0.0216) |
| TRAP | 0.5435 (0.0123) | 0.5313 (0.0075) | 0.0192 (0.0196) | 0.1960 (0.0265) |
| NetTCR-2.2 | 0.5519 (0.0188) | 0.5335 (0.0108) | 0.0428 (0.0318) | 0.1921 (0.0338) |
| ERGO-II | 0.5074 (0.0336) | 0.5095 (0.0240) | 0.0403 (0.0365) | 0.1861 (0.0484) |
| EMBER | 0.5931 (0.0104) | 0.5708 (0.0077) | 0.0931 (0.0249) | 0.2341 (0.0256) |
From the results above, EMBER takes the lead compared with the other baselines. The drop in the absolute level of the metrics appears in all the baselines as well, and it is caused by the high variation that comes with a limited number of peptide classes. However, EMBER keeps its lead on a second dataset with different negative samples compared with the STAG dataset.
Reference
Messemaker, M., Kwee, B. P. Y., Moravec, Ž., de Paauw, S., Urbanus, J., Álvarez-Salmoral, D., Geerligs, J., Xie, R., Voogd, R., Morris, B., Guislain, A., Winkler, Y., Richterich, C. A., Mussmann, M., Bakker, M., Steinmetz, M., Iras, M., Marcus, E., Teuwen, J., Perrakis, A., Beijersbergen, R. L., Scheper, W., & Schumacher, T. N. (2026). Functional evaluation of TCR-pMHC pairs at scale allows in silico TCR reactivity prediction. Immunity, 59(10), 2775-2788.e9. https://doi.org/10.1016/j.immuni.2026.09.007
Follow-up on Rebuttal
Dear reviewers,
As the rebuttal deadline is approaching, I would like to kindly check whether our responses have addressed your concerns and questions. If there is anything that is still unclear or could benefit from further clarification, please let us know, and we will be happy to elaborate.
Thank you very much for your time and effort in reviewing our work.
We thank the reviewer for naming the control our experiments were missing and for saying why it is central to the assessment. Both concerns are actionable as written, and we take them in turn.
Response: We ran both variations and present the results in the revised paper. For the split control of Table 4, the ESMFold2 co-folded complex is cut into its TCR and pMHC halves, each half is featurized alone, and the two are encoded by the unchanged EMBER architecture, which is the arm described. It keeps the bound conformation and the complex-derived features, and removes only the joint graph and the cross-molecular message passing.
[email protected]MCC*The arm loses on all six metrics, by 0.020 of macro-AUROC and 0.026 of micro-AUPRC, so the drop is present once the halves carry the bound conformation. For the pose-only perturbation of Table 5, the docked arm reads the 49 crystal complexes at the crystal relative pose, and then with the receptor translated rigidly by 3 Å and by 8 Å in a fixed random direction. Nothing else changes and every target is a binder, so the correct response is no drop.
Table 11 puts the magnitudes in context. The median TCR docking RMSD is 5.87 to 8.09 Å for five of the six tools, and 22% to 51% of targets sit beyond 8 Å. Boltz-2 is the apparent exception at 1.50 Å, and Table 12 separates it, because 39 of the 49 targets precede its training cutoff and on the ten it cannot have memorized its median rises to 6.55 Å with no near-native prediction. The limit of this evidence is that all 49 targets are binders, so Table 5 reports a score shift and not a change in discrimination. What the two controls establish together is that a pose-reading score moves under a pose change alone, at displacements the tools produce, and that the loss is already present when the two halves are processed separately.