acceptodds
Under review as a conference paper at ICLR 2027

Does Register-Length Flexibility Help Peptide–MHC-II Classification? A Controlled Ablation on the Chicken Allele BLB2

Abstract

Discrete latent-alignment models are usually given a fixed-length candidate structure to select over. We ask what happens when that support is widened to match documented heterogeneity in the target structure itself, using peptide–MHC class II (MHC-II) ligand classification as a test bed. Most MHC-II predictors represent the binding core as a canonical nine-residue window and treat only its position as a latent register variable. The chicken allele Gaga-BLB2*002:01 is a useful test case because a structural study establishes that it can anchor a decamer core (Halabi et al., 2021), while a separate computational motif-deconvolution study reports its ligand pool splits nearly evenly between 9-residue and 10-residue annotated cores (Racle et al., 2023), a split our own compiled dataset reproduces (2,134 9-mers, 2,050 10-mers among 4,184 positives). We train a register variable, supervised by an auxiliary cross-entropy loss against the annotated core alongside the primary classification loss, to range jointly over both candidate lengths, and compare it under a parameter-matched ablation that masks the 10-mer candidates out. Holding architecture, data, and training procedure fixed across five outer-fold held-out evaluations (single seed, one allele), the two variants are statistically indistinguishable on ranking metrics (pooled ROC-AUC 0.920 for both), and the restricted model is, if anything, slightly higher on PR-AUC (0.597 versus 0.583; a paired bootstrap places the gap at 0.013, 95% CI [0.006, 0.021]). We cannot attribute that gap to candidate-length support alone, since masking also removes auxiliary supervision from the 10-mer-annotated half of positives. Only the flexible model can agree with a length-flexible annotation at all: it matches the annotated core length on 74.3% of held-out positives (a 51.0% majority-class floor) and reproduces the exact annotated core substring on 30.7%. We also find that a large share of what our dataset labels "BLB2-positive, 9-mer core" may in fact be a co-expressed allele's ligands misassigned by the motif-deconvolution pipeline, a hypothesis Halabi et al. themselves supply and that predicts the largest effect we observe: 9-mer-annotated positives score far lower (PR-AUC ≈0.34–0.35) than 10-mer-annotated positives (PR-AUC ≈0.58–0.59) at matched prevalence. Register flexibility, in this architecture and on this single allele, is therefore better read as a lever on whether a model's internal explanation can engage with heterogeneous annotations at all, not as a classification-accuracy improvement. Part of the negative result may itself be an artifact of imperfect training labels rather than of the inductive bias being tested.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
MHC-II ligand prediction, BLB2, chicken MHC-II, peptide classification, register-aware attention, latent alignment, ablation study, interpretability, immunoinformatics, inductive bias

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wXjzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether allowing variable binding-core lengths improves peptide–MHC-II classification for the chicken allele Gaga-BLB2*002:01. It compares an existing register-scoring model permitting 9- and 10-residue cores with a parameter-matched variant restricted to 9-residue cores. The dataset contains 62,590 records, including 4,184 BLB2 positives and 58,406 negatives. Validation includes five outer folds, subgroup analyses, supervised core-annotation recovery, and an external comparison with MixMHC2pred.

The flexible model does not improve the reported classification metrics: both models have pooled ROC-AUC 0.920, while pooled PR-AUC is 0.583 for the flexible model and 0.597 for the restricted model. The flexible model matches annotated core length on 74.3% of held-out positives and the exact annotated core on 30.7%. The intended contribution is an empirical distinction between classification performance and compatibility with heterogeneous core annotations, rather than a new scoring mechanism. The paper also investigates possible BLB1-related label contamination as a competing explanation.

Critical strengths

S1. A focused and biologically motivated question. Testing the fixed 9-residue assumption on an allele with structural support for a 10-residue core is a sensible experimental setting. Distinguishing classification from core recovery can reveal a limitation of relying on classification metrics alone. This supports acceptance in principle even if the method is not new and does not outperform existing predictors.

S2. Targeted diagnostics and transparent negative findings. The paper examines the 10-mer subgroup that should plausibly benefit, distinguishes length prediction from exact core recovery, reports negative controls, and acknowledges inferred annotations, synthetic negatives, and single-seed uncertainty. These analyses make the work more informative than a single aggregate performance comparison and provide a basis for a useful, bounded empirical conclusion.

Critical weaknesses

W1. The central experiment does not isolate length flexibility (Sections 3–6; Appendix Table 7). Masking 10-mer candidates also removes auxiliary core supervision for 10-mer-annotated positives. Moreover, the selected flexible model uses mildly weighted BCE while the restricted model uses ordinary BCE. Equal parameter counts do not control these differences. Consequently, the observed performance gap cannot be attributed specifically to candidate-length support. The missing 10-mer- only control also leaves open whether an allele-appropriate fixed length would suffice. This is central because the paper’s main contribution is a controlled empirical conclusion, not the introduction of a new mechanism.

W2. Evaluation independence and the interpretation of the findings remain insufficiently established (Sections 4, 6–10). If model choices were selected using the inner folds of outer round 1 and then reused across all five outer folds, later test folds may already have influenced those choices. For example, tuning on F2–F5 while testing F1 does not leave F2–F5 untouched for later evaluation. This is a conditional concern requiring clarification. Separately, agreement with supervised computational annotations does not establish faithful biological explanations, and low scores on double-labelled peptides do not establish contamination: a genuine BLB1/BLB2 dual binder missed by the model produces the same observation. These gaps limit both the reliability and the scope of the claimed insight.

Questions for authors

Q1. Required clarification and targeted control – Can you report a comparison using the same classification loss, class weights, splits, and selection protocol, with auxiliary core supervision disabled for all variants? Including 9-mer-only, 10-mer-only, and flexible variants would distinguish changing the preferred fixed length from requiring per-peptide flexibility. An additional comparison could supervise the same 9-mer examples in both the restricted and flexible models. Allowing 10-mer outputs does not require providing 10-mer auxiliary targets. The original fully supervised model can remain a reference. These controls would directly affect my assessment of W1.

Q2. Required clarification – Which data selected the architecture, loss, training duration, and threshold for each outer fold? Was there a development set independent of every reported test fold, or was selection nested separately within each outer training set? If choices from outer round 1 were reused after exposure to later test folds, please provide an evaluation that accounts for this selection. Protein-disjoint training and test sets alone do not establish independence of model selection.

Q3. Required clarification and missing comparison – Can you report exact core-match and start- position performance for both models, overall and by annotated length? In Table 4, 1257/2050 = 61.3% measures 10-mer length sensitivity, not exact recovery of a 10-mer core. Please distinguish this from the 30.7% overall exact-match result. The 74.3% length accuracy is a nontrivial supervised prediction result, but it validates agreement with the annotation procedure rather than independently establishing structural truth. If the claim concerns explanation of classification, what shows that the reported argmax core faithfully represents a classifier using soft pooling and additional whole-sequence branches? Otherwise, narrowing the claim to supervised annotation recovery would be appropriate.

Q4. Required clarification and sensitivity analysis – What evidence distinguishes incorrect BLB2 assignment from genuine dual binding or model failure? The reported enrichment of BLB1-positive labels in the 9-mer group is compatible with contamination but is not a contamination-rate estimate or lower bound. For example, a correctly labelled dual binder with a low BLB2 prediction would be counted in the same pattern. Please separate the biological hypothesis motivated by Halabi et al. from the present paper’s prediction about classification scores. Reporting results separately for double-positive and BLB2-only records, and repeating the subgroup and model comparisons after excluding double-positive records, would be useful sensitivity analyses without assuming those records are wrong. Also explain why contamination would differentially affect the two models, rather than merely make the 9-mer subgroup harder for both.

Q5. Required clarification and uncertainty analysis – How should the claim of statistically indistinguishable ranking performance be reconciled with a paired bootstrap PR-AUC difference of 0.013 and a 95% interval [0.006, 0.021]? Equal rounded ROC-AUC values are not an equivalence test. The mean fold-wise PR-AUC gap is about 0.003, whereas the pooled gap is about 0.014; please report paired fold-wise differences and discuss the role of cross-fold score scales. Protein-cluster resampling and a small number of independent seeds would better characterize the uncertainty relevant to this small effect. Please also distinguish the overall bootstrap result from subgroup-specific uncertainty and the tuning-stage negative controls from final outer-fold evidence.

Q6. Clarification of the comparison – Why was DeepMHCII, identified as closely related, not evaluated or discussed as an adaptable baseline under the same BLB2 splits? I am not requesting every recent predictor or assuming human-only tools support chicken alleles. Please explain applicability and adaptation constraints. MixMHC2pred v2.0.2 should be associated with the corresponding 2023 work rather than only the 2019 citation. A fixed 9-mer core alone does not make an evaluation zero-shot or out-of-distribution; this depends on training coverage, which also needs reconciliation with the stated possibility of overlapping chicken data. Scope of requests and minor corrections. Q1–Q5 concern the existing central claims, not a demand for a broad new research program. Multi-species validation and extensive new structural experiments are possible future work, not prerequisites here. Please also state the class ratio unambiguously as positive:negative approximately 1:14; check the boldface for pooled precision in Table 2 (0.694 versus 0.698); and provide the negative-sampling procedure, weighted-BCE weights, PR-AUC definition, and fold-specific selection details. These reporting corrections alone would not change my recommendation.

1 comment

Thanks for the very thorough review, we’ll go through this point by point.

On W1/Q1 (confound + requested control): we reran it with the exact grid you described — 9mer-only, 10mer-only, and flexible, all with auxiliary supervision disabled, same loss/weights/splits/selection across the board. Length alone isn’t significant between flexible and 9mer-only. It is significant for 10mer-only vs flexible though (flexible wins there). Separately, turning auxiliary supervision ON for the flexible model significantly hurts its PR-AUC (-0.025, 95% CI [-0.0395,-0.0110]). So the real effect in this whole thing is the auxiliary loss, not candidate-length support like we originally claimed. Rewriting that section.

On W2/Q2 (selection independence): you and yyLV both caught the same thing — selection was done once on round 1’s inner folds and reused for rounds 2-5, so later test folds could’ve influenced the choice. We reran selection independently per outer round. Result flips: flexible now slightly ahead (+0.0084 PR-AUC, CI [-0.0061,0.0229]) instead of behind like originally reported (-0.0133, CI also crossing zero). So this wasn’t just a mild leakage issue, the original gap direction itself wasn’t even stable.

On Q3 (metric clarification): fair catch, 61.3% is length-match not exact-core-match, we’ll fix the table/text to make that distinction explicit and clarify the 74.3%/30.7% numbers are agreement with the annotation procedure, not an independent ground-truth check.

On Q4 (contamination): excluded all dual BLB1/BLB2-labeled positives and reran the subgroup comparison, gap barely moves (0.236 to 0.239, flexible model), actually slightly wider. We also broke it out by dual-positive-only vs single-positive-only (9mer dual n=196, 10mer dual n=14, too small to report a separate PR-AUC on its own). So contamination doesn’t look like it’s driving the effect the way we suggested, we’re softening that claim.

On Q5 (uncertainty/bootstrap unit): you’re right, we were resampling at peptide level not protein level, which is the actual grouping unit for the splits. Redone with protein-clustered resampling, the original PR-AUC CI [0.006,0.021] becomes [-0.001,0.028], crosses zero now. Also reran at 2 more seeds (43,44), gap direction holds (+0.013/+0.014/+0.011) but given the CI above that’s “consistently not significant” not confirmation. And on fold-wise vs pooled: mean fold-wise diff is +0.004 (SD 0.025) vs pooled +0.013, difference comes from 3 of 5 folds favoring the ablation plus pooling being prevalence-weighted, not a uniform per-fold effect.

On Q6: DeepMHCII baseline is a fair ask but we can’t responsibly implement and tune a second published method before the deadline, we’ll flag it as the clearest future work item and note our current comparison should be read as an ablation study not a SOTA comparison. Also fixing the MixMHC2pred citation year, the class ratio wording (1:14), and the Table 2 boldface error you caught (0.698 should’ve been bold, not 0.694) — already corrected in our working draft.

Reviewer_yyLVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper examines whether explicitly allowing both 9- and 10-residue binding cores improves ligand classification for the chicken MHC-II allele Gaga-BLB2*002:01. The model takes a peptide sequence and combines CNN, BiLSTM, Transformer, and register-aware representations. The register branch scores candidate core lengths and positions, receives auxiliary supervision from computationally inferred core annotations, and contributes a pooled representation to the classifier. The main comparison masks the 10-residue candidates while retaining the same parameter count.

The authors use a compiled dataset of 62,590 peptide–protein examples: 4,184 BLB2 positives and 58,406 source-protein decoys. The positives comprise 2,134 annotated 9-residue cores and 2,050 annotated 10-residue cores. Evaluation uses five source-protein-grouped outer folds and one training seed. The flexible and fixed models both achieve pooled ROC-AUC of 0.920, while PR-AUC favors the fixed model, 0.597 versus 0.583. The flexible model recovers the annotated core length on 74.3% of held-out positives and the exact annotated core sequence on 30.7%.

Critical strengths
  1. The paper asks a specific, biologically motivated question and reports an unfavorable result for its proposed architectural choice. The existence of a decamer binding mode gives a concrete reason to investigate the conventional nine-residue assumption. Testing the 10-mer-annotated subgroup separately is particularly useful: the flexible model does not improve classification even among examples where additional candidate lengths might be expected to help. A reliable version of this experiment could inform how binding-core predictors are designed.
Critical weaknesses
  1. The central ablation does not isolate register-length flexibility. Masking the 10-mer candidates also removes auxiliary register supervision from 2,050 of the 4,184 positive examples. The final models additionally use different classification objectives: plain BCE for the fixed model and mildly weighted BCE for the flexible model. Identical parameter counts therefore do not make this a comparison in which only candidate length changes. The observed difference could arise from the candidate support, the auxiliary targets, the classification loss, or interactions among them.The authors explicitly acknowledge the auxiliary-supervision confound, which I appreciate. However, acknowledging it does not resolve the paper’s central question. Since the contribution rests on a controlled empirical finding, the corresponding control is essential.

  2. The evaluation procedure does not yet establish a reliable held-out negative result. Section 4 states that architecture, loss, and threshold selection use the inner-fold predictions from outer round 1, after which the selected configuration is evaluated across all five outer folds. Under ordinary five-fold partitioning, the development data for round 1 include the examples that later become test folds 2–5. Their labels would therefore have influenced model selection, even if they were excluded when fitting their eventual fold model. As written, this is not fully nested cross-validation. The authors should clarify whether a separate selection partition was used or repeat selection within each outer development partition.

Questions for authors
  1. How were all five outer test folds kept independent of model selection? Can you provide the relationship between the outer folds and the data used to select architecture, loss, epoch budget, and threshold. If selection used the development pool from outer round 1, please repeat selection within each outer development partition or evaluate the frozen choices on a separate untouched test set. Please also report cross-fold peptide and source-protein similarity, and repeat the decisive comparison across several seeds with uncertainty that respects source-protein grouping. Fold-wise paired differences should accompany the pooled results.

2.What are the exact sources and evidence behind the BLB2, BLB1, and core annotations? Can you identify the source releases, experimental samples, annotation procedure, and decoy-generation rules. Are the dual-positive labels independently supported experimental assignments, or outputs of related computational annotation procedures? A useful sensitivity analysis would repeat the fixed/flexible comparison after excluding dual-labeled examples and report results separately by annotation confidence or experimental source. This would test whether the conclusion depends on the suspected ambiguity, although removing dual-labeled examples should not be described as producing a verified clean dataset. Any claim about biological register recovery would require evaluation on independently supported core assignments.

1 comment

Thanks for the detailed review, these are fair points.

On the confound (W1): we reran it properly. Built a 3x2 grid that turns off candidate length and auxiliary supervision separately, same loss/epochs/threshold everywhere. Turns out length alone isn’t significant either way. The one real effect is that adding the auxiliary register loss actually hurts the flexible model’s PR-AUC (-0.025, 95% CI [-0.0395,-0.0110]). So the original story was wrong, it’s the supervision signal doing the work, not candidate length. We’re rewriting that part of the paper around this.

On the selection leakage (W2): you’re right, we picked architecture/loss/threshold once on round 1 and just reused it. We reran selection independently for all 5 rounds and the result actually flips — flexible model now slightly ahead of the 9-mer ablation (+0.0084, CI [-0.0061,0.0229]) instead of behind like we originally reported (-0.0133, CI also crossing zero). So the headline gap wasn’t real, not just inflated. This is probably the biggest fix from your review honestly.

On Q1 (fold overlap): reran the exact-duplicate check, zero duplicate peptides and zero protein overlap across folds. We haven’t done a homolog-level check though (near-identical but not exact sequences) — don’t have time to add a sequence alignment step before the deadline, will note as limitation.

On Q2 (BLB1/BLB2 contamination): excluded dual-labeled peptides and reran the 9mer/10mer comparison — gap doesn’t shrink (0.236 -> 0.239), if anything slightly bigger. So contamination isn’t really explaining it like we suggested, will tone that claim down.

Reviewer_9FfbOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This submission study whether a peptide–MHC-II classifier benefits from allowing its latent binding register to span both nine- and ten-residue cores. It compares a four-branch sequence model with a length-flexible register branch against a version that masks ten-residue candidates, using source-protein-grouped held-out folds. The two versions have similar classification performance, while the flexible version more often recovers the inferred core-length annotations. The paper also analyzes performance by annotated core length and examine possible overlap between BLB2 and BLB1 ligand labels.

Critical strengths
  1. The architectural question is focused, and the negative result is worth reporting. Both versions use the same held-out folds and main architecture.

  2. PR-AUC is the right main metric for 4,184 positives against 58,406 computational negatives. Reporting both per-fold and pooled results also helps interpret the comparison.

Critical weaknesses
  1. Obvious AI sloped submissions with super long, with citations, and very concrete numbers in the abstract. No need to mentioned tons of other issues in the main text.

  2. The result comes from one allele, one architecture, and one seed, using putative decoy negatives and some potentially misassigned positives. Source-protein-disjoint folds were verified, but sequence redundancy across homologous proteins was not checked.

  3. The bootstrap resamples peptides one by one, even though the split is grouped by source protein. If peptides from one protein are correlated, this may understate uncertainty; a protein-clustered analysis would align with the split unit.

  4. Masking 10-mer candidates also drops auxiliary register supervision for 2,050 positives. Equal parameter counts do not isolate candidate-length support, so the experiment cannot determine which change drives the small PR-AUC gap.

Questions for authors
  1. What happens when the auxiliary register loss is disabled in both models, or when both receive the same eligible supervised examples?

  2. Do the PR-AUC difference and subgroup pattern hold across training seeds?

  3. Could you resample source proteins for the paired PR-AUC interval and report uncertainty for the ROC-AUC difference as well?

1 comment

Thanks for your time and valuable feedback. We reran the central comparison with the confound removed: a 3×2 grid that separately switches off (a) which candidate lengths get scored and (b) the auxiliary register-supervision loss, holding loss function, epochs, and threshold fixed across every arm. The result showed restricting candidate length (9-mer-only vs. flexible) is not a significant effect, with or without auxiliary supervision. The one clean effect in the whole grid is that adding the auxiliary register loss to the flexible model significantly hurts its PR-AUC (Δ=-0.025, 95% CI [-0.0395,-0.0110], protein-clustered bootstrap, 2000 resamples). So the original framing: (flexible registers underperform because of fewer length options) doesn’t hold up; the real driver is the extra supervision signal, not candidate-length support. We are rewriting the central claim around this.

On Q2 (seed stability): reran both headline models at seeds 43/44. The point estimate is directionally consistent (+0.013/+0.014/+0.011 across seeds 42/43/44), but given the CI result above, that consistency is “stably not significant,” not confirmation of a real effect. We’ll also address the writing/citation points you raised in the camera-ready.

On point 3 (bootstrap unit): correct, we were resampling peptides, not the actual train/test grouping unit (protein). Redone with protein clustered resampling, the original ablation’s PR-AUC CI [0.006,0.021] becomes [-0.001,0.028] — it now includes zero.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.