acceptodds
Under review as a conference paper at ICLR 2027

Adapting Molecular Foundation Models for Macrocyclic Peptide Affinity Prediction via Robust Proxy Rehearsal

Abstract

Adapting molecular foundation models to macrocyclic peptide affinity prediction requires improving target-domain performance without sacrificing their existing small-molecule capabilities. When the original pretraining corpus is unavailable, public affinity measurements can serve as a source proxy, but their protein-family composition and coverage may differ from the true source distribution. We study retention under this imperfect source information. We show that reducing absolute worst-family risk does not guarantee retention, whereas controlling degradation relative to the pretrained model characterizes worst-case retention over arbitrary family proportions under full coverage and matched within-family distributions. A conditional bound further isolates the effects of finite sampling, within-family mismatch, and missing coverage. Motivated by this analysis, we introduce Robust Proxy Rehearsal (RPR), which combines target supervision with reference-relative source penalties and adaptively emphasizes protein families that degrade during adaptation. We instantiate RPR with Boltz-2 and evaluate macrocycle adaptation, small-molecule retention, and transfer to two external peptide benchmarks. In our primary comparison, RPR improves over uniform replay, increasing Spearman from 0.508 to 0.611 on the source domain and from 0.216 to 0.312 on the target domain, while also improving both external benchmarks. Additional runs show that these gains vary across hyperparameters, delineating when reference-relative rehearsal can improve the adaptation–retention trade-off under an imperfect proxy.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Molecular Foundation Models, Continual Learning, Binding Affinity Prediction, Macrocyclic Peptides

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 5 reviews, above 85% of the others.
Reviewer_6214OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies a new method for adaptation of pretrained ligand-target affinity prediction on larger macro-molecules. The adaptation assumes that no direct accesses to the original source data used for pretraining but a new source data is available but practically different from the original source and another target macro-molecule data is available.

The main idea behind the adaptation is to rerank the contribution of the families in the new source data to penalise family with highest degradation loss while ignoring the families that the degradation do not happen. The idea is similar GRPO but applied in the context of the domain adaptation problem.

Critical strengths

The proposed idea is interesting and may work in the context of continue learning. The comparison study is strong with strong baselines and relevant experiments, I have a concern on this perspective please see in the weakness points. The paper was well written and easy to follow.

Critical weaknesses

My main concern is that the proposed approach trade old model retention for adaptation to new macro-molecule so the hyperparameters used to balance the losses have an important impact. In the experimental settings the proposed approach used tuned hyper parameters but it is unclear whether all the baseline approaches which have hyperparams were tuned carefully. I therefore recommend a weak reject but if the authors provide additional experiments as I request below and the results are still significant I will reconsider my recommendation:

  1. Please provide the results of the baseline including Uniform Replay, GRPO when their hyper params are finetuned carefully with the same setting as the proposed method.

  2. Please provide the results of finetuned method not only with macro molecules but also finetuned on the new source data because the given method is not finetuned on larger data as the other baseline.

  3. Please varied the hyperparams of all methods and plot the pareto frontiers according to two metrics: one on the target molecules and one on the new source molecules, report the Hyper Volume of each method, if the proposed method can find better HV solution that would be a definite strong results.

Questions for authors

Please look at the questions I provided in the weaknesses points.

8 comments
AuthorsOpenReview ↗

Responses to Reviewer 6214 (1/4)

We sincerely thank the reviewer for stating so clearly what additional evidence would be useful. Your points on tuning the baselines under the same setting, on fine-tuning with the union of the two data streams, and on presenting the trade-off as a Pareto frontier with hypervolume have been extremely helpful, and we have run all three. We summarize the issues raised and address them one by one below. Numbers are mean ± sd over three random seeds; bold marks the best method in each column.


W1: “My main concern is that the proposed approach trade old model retention for adaptation to new macro-molecule so the hyperparameters used to balance the losses have an important impact. In the experimental settings the proposed approach used tuned hyper parameters but it is unclear whether all the baseline approaches which have hyperparams were tuned carefully.”

A1: Thank you for this concern. Two things answer it: the baselines are now tuned under our own protocol, and at our operating point the trade-off does not have to be paid.

On the tuning. Every baseline with a tunable hyperparameter is now swept under the identical setting as RPR and over three random seeds (A2). On our own side two settings were tuned, the step size η and the source:target draw, both chosen on the target validation set by the same median within-family Spearman statistic that selects checkpoints, from the sweeps the paper reports in Table 6 and Table 5; every other setting was fixed in advance.

The gain on macrocycles is not paid for on the small-molecule side. RPR improves on the released model on both axes by much the largest margin: Source Fam. ρ 0.455 ± 0.042 against 0.356 for pretrained Boltz-2, and Target Fam. ρ 0.261 ± 0.013 against 0.050, which is +0.099 and +0.211 where the next best margins are +0.073 and +0.134. The quantity the trade-off is supposed to protect is protected as well: on 1,909 held-out source records and 36 families, the largest per-family degradation relative to the pretrained model is 0.226 ± 0.043 for RPR against 0.442 ± 0.132 for UR, lower in all three seeds.

The balance is not fragile in η. At η = 0.1 / 0.4 / 1.0 the three-seed means are Source Fam. ρ 0.394 / 0.455 / 0.392 and Target Fam. ρ 0.218 / 0.261 / 0.197, every one of them above UR's 0.384 and 0.171, while collapsing the family weights to uniform (η = 0) gives 0.419 / 0.138 and loses the target-side gain. That isolates reference-relative reweighting, rather than a particular loss balance, as the component doing the work.

AuthorsOpenReview ↗

Responses to Reviewer 6214 (2/4)

Q1: “Please provide the results of the baseline including Uniform Replay, GRPO when their hyper params are finetuned carefully with the same setting as the proposed method.”

A2: Thank you for asking for this. We swept the two methods the question names, uniform replay and group DRO with the absolute signal, and extended the same sweep to every other baseline of Table 3 that has a tunable strength, so the comparison is matched across the whole table rather than for two methods alone. Every baseline with a tunable strength is now swept over three random seeds under the identical protocol, source stream, 1:1 draw, target exposure and step budget as RPR, with the same protocol IV checkpoint rule. Methods appear in the order of Table 3, and the third column marks the configuration that appears in the submitted Table 3. Group DRO with the absolute q signal is the ablation of Table 4 rather than a Table 3 baseline; we sweep it alongside the others because it shares our grouping, stream and machinery and differs only in using an absolute rather than a reference-relative weight signal.

MethodConfigurationIn Table 3Source Fam. ρ ↑Target Fam. ρ ↑
Pretrained Boltz-2 (no FT)n/a✓0.3560.050
FTno tunable strength✓0.276 ± 0.0430.177 ± 0.034
L²-SPλ = 0.10.276 ± 0.0680.201 ± 0.027
L²-SPλ = 1.0✓0.279 ± 0.0420.186 ± 0.030
L²-SPλ = 100.340 ± 0.0050.146 ± 0.038
EWCλ = 0.10.272 ± 0.1060.228 ± 0.108
EWCλ = 1.00.291 ± 0.1010.206 ± 0.017
EWCλ = 2.0✓0.351 ± 0.0280.182 ± 0.033
ODλ = 0.30.409 ± 0.0780.188 ± 0.032
ODλ = 1.0✓0.404 ± 0.0490.184 ± 0.043
ODλ = 3.00.355 ± 0.0320.199 ± 0.028
URsource:target 1:30.434 ± 0.0140.213 ± 0.089
URsource:target 3:50.394 ± 0.0600.147 ± 0.013
URsource:target 1:1✓0.384 ± 0.0680.171 ± 0.050
AGEMmem = 80.338 ± 0.0580.208 ± 0.012
AGEMmem = 160.386 ± 0.0280.133 ± 0.052
AGEMmem = 32✓0.429 ± 0.1380.168 ± 0.063
Group DRO (absolute q signal)η = 0.10.417 ± 0.0690.213 ± 0.110
Group DRO (absolute q signal)η = 0.40.325 ± 0.0310.123 ± 0.047
Group DRO (absolute q signal)η = 1.00.405 ± 0.0420.188 ± 0.092
RPR (ours)η = 0.4, source:target 1:1✓0.455 ± 0.0420.261 ± 0.013

No configuration reproduces the trade-off. On the target axis nothing reaches RPR's 0.261; the closest is EWC at λ = 0.1 with 0.228 ± 0.108, and RPR's worst of three seeds, 0.253, is still above the mean of every other configuration in the table. No configuration improves on RPR on the two axes together, and RPR attains its best value on both axes at one and the same setting rather than needing a different one per metric.

Matched tuning also answers how much of the spread is tuning and how much is noise. The seed-to-seed variation of the baselines is an order of magnitude larger than ours: on the target axis EWC at λ = 0.1 ranges from 0.104 to 0.298 over the three seeds and absolute-signal Group DRO at η = 0.1 from 0.114 to 0.331, against 0.253 to 0.276 for RPR. A single run of a baseline is therefore close to unreadable, which is why the default settings quoted in the submission looked weaker than they should, and it is the reason we report every cell as a three-seed mean here. Two of the axes are not even monotone in their own hyperparameter: AGEM gives 0.208, 0.133 and 0.168 on the target axis at mem = 8, 16 and 32, and UR gives 0.213, 0.147 and 0.171 at 1:3, 3:5 and 1:1, so neither offers a direction in which a better setting could be extrapolated.

Retention measured where it matters keeps the same ordering. Aggregate retention can rise while individual families deteriorate, so we also compare the upper tail of the per-family degradation on 1,909 held-out source records and 36 families. RPR's worst family degrades by 0.226 ± 0.043 and its second-worst by 0.174, against 0.442 ± 0.132 and 0.328 for uniform replay at the submitted fraction, which sees the identical source stream; RPR is lower in all three seeds.

AuthorsOpenReview ↗

Responses to Reviewer 6214 (3/4)

Q2: “Please provide the results of finetuned method not only with macro molecules but also finetuned on the new source data because the given method is not finetuned on larger data as the other baseline.”

A3: Thank you for catching this confound. The comparison is already available in the paper, because uniform replay is fine-tuning on the union of the two streams. UR applies the plain Boltz-2 affinity loss to target and source records with no retention term, no reweighting and no reference model, differing from FT in exactly one respect, that it also draws source batches. Reading the three rows in that light gives the decomposition the reviewer asks for:

MethodData seenSource Fam. ρ ↑Source ρ ↑Target Fam. ρ ↑Target ρ ↑
Pretrained Boltz-2 (no FT)n/a0.3560.4550.0500.283
FT (target only)15,5160.276 ± 0.0430.462 ± 0.0160.177 ± 0.0340.262 ± 0.064
UR = FT on target ∪ source35,5090.384 ± 0.0680.567 ± 0.0530.171 ± 0.0500.265 ± 0.044
RPR (ours)35,5090.455 ± 0.0420.603 ± 0.0120.261 ± 0.0130.310 ± 0.029

Adding the source data recovers retention; reweighting it is what improves both sides. Adding the source data (FT → UR) recovers small-molecule retention, from 0.276 to 0.384 on Source Fam. ρ and from 0.462 to 0.567 on Source ρ, while the macrocycle side does not improve at all and in fact moves from 0.177 to 0.171. More data by itself buys retention and nothing else. Reference-relative reweighting (UR → RPR) then improves both sides at constant data volume: Source Fam. ρ 0.384 → 0.455 and Target Fam. ρ 0.171 → 0.261, with the identical stream, the identical target exposure and the identical step count. The second comparison is the paper's claim, and it is measured at equal data, which is the control the question asks for.

On the mixing proportion, UR draws the two streams at source:target 1:1, whereas their natural proportion would put slightly more weight on the source; the sweep in A2 varies that ratio and no setting of it gives UR a simultaneous advantage on the two axes.

Reviewer_E3ynOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studied the continual learning problem of Boltz-2's affinity predictor. The authors adapted the predictor to predict protein–macrocyclic peptide affinity. They constructed a public small molecule ligand proxy to serve as replay data and prioritized samples that show large increased loss. The method was validated on held out test data and external peptide benchmarks.

Critical strengths

This paper attacks an important practical problem. It did a good job preparing the dataset, doing extensive ablations and comparisons with several baselines.

Critical weaknesses

Some minor issues:

  1. The source replay contains proteins from the target test families, so the proposed methods have an advantage over the target only family disjoint finetuning baseline.
  2. The methodological novelty is limited. Reference relative retention has been studied previously, relevant works such as Maximally Interfered Retrieval (MIR), are not discussed.
Questions for authors

See the previous section.

2 comments
AuthorsOpenReview ↗

Responses to Reviewer E3yn (1/2)

We sincerely thank the reviewer for the positive assessment and for two comments that improved the paper. Your points on the overlap between the source replay and the target test families, and on the relation to prior work on reference-relative retention, have been extremely helpful. We summarize the issues raised and address them one by one below.

Numbers are mean ± sd over three random seeds; bold marks the best method in each column.


W1: “The source replay contains proteins from the target test families, so the proposed methods have an advantage over the target only family disjoint finetuning baseline.”

A1: Thank you for raising this. We measured how much the overlap contributes, and the gains turn out not to be where the overlap is.

The comparison that tests our contribution does not involve the FT baseline. RPR, UR, OD, AGEM and Group DRO all draw the identical source stream, at the identical source:target 1:1 ratio, with equal target exposure and the same step budget, so the overlap is common to all of them and cannot explain RPR's margin over them. The controlled comparison is UR → RPR, where the data are identical and only the weighting changes: Source Fam. ρ 0.384 ± 0.068 → 0.455 ± 0.042 and Target Fam. ρ 0.171 ± 0.050 → 0.261 ± 0.013. FT is a reference point for what target-only adaptation costs on the source side, not a method run under the same conditions as the rehearsal baselines.

The overlap is also not where the measured gains come from. We split the 2,379 target test records into three tiers and evaluated each separately, with record-level bootstrap 95% CIs. T1 known target: protein present in the source stream with an identical sequence (689 records, 57 proteins, 43 families). T2 relative: protein absent from the source stream but its family covered (1,516 records, 92 proteins, 29 families). T3 absent: family covered by neither stream (174 records, 36 proteins, 18 families).

TierPretrained Boltz-2 (no FT)FTODURRPR (ours)
T10.066 [−0.04, 0.31]0.095 [−0.07, 0.30]0.105 [−0.05, 0.28]0.151 [−0.06, 0.29]0.136 [−0.03, 0.31]
T20.113 [−0.05, 0.28]0.244 [0.13, 0.34]0.182 [0.01, 0.33]0.212 [0.02, 0.28]0.272 [0.15, 0.37]
T3−0.105 [−0.35, 0.20]0.065 [−0.27, 0.31]0.057 [−0.34, 0.20]−0.011 [−0.31, 0.26]0.210 [−0.03, 0.44]

T1 is the tier where the overlap is literal, and it separates nothing: all methods and the pretrained model lie inside each other's intervals, so the 57 verbatim-shared proteins are not carrying the result. T2, the family relatives, holds 1,516 of the 2,379 records and is where the gain is largest and clearest, with RPR highest and its interval excluding zero; FT, which sees no source stream at all, is second there, which is the opposite of what an overlap advantage would produce. On T3, the families in neither stream, RPR has the highest point estimate of any method (0.210, against −0.011 for UR and −0.105 for the pretrained model); with 174 records that interval is wide, so we treat it as encouraging rather than established.

AuthorsOpenReview ↗

Responses to Reviewer E3yn (2/2)

W2: “The methodological novelty is limited. Reference relative retention has been studied previously, relevant works such as Maximally Interfered Retrieval (MIR), are not discussed.”

A2: Thank you for the pointer to MIR. We have added it to the related work, together with GSS and DER/DER++.

What MIR does, and where we differ. MIR (Aljundi et al., 2019) retrieves from a replay buffer the samples whose loss would rise most under a virtual update θv = θt − α∇L(incoming batch), scoring each candidate by ℓ_{θv}(x) − ℓ_{θt}(x). It therefore shares with RPR the idea of prioritising by a loss increase rather than a loss level. The differences that our contribution rests on:

MIRRPR (ours)
Referencecurrent model one step ahead; moving, local, reset every stepfixed pretrained model θ0; global, cumulative over adaptation
Granularityper sample, for buffer retrievalper protein family, groups fixed before training
What it changeswhich samples are replayedhow a fixed stream is weighted in the loss
Costvirtual update plus an extra forward pass over candidates at every stepone cached pointwise loss per source record, computed once; no second model at training time
GuaranteeheuristicTheorem 1: max_g Δ_g is exactly the worst-case retention over unknown family proportions

The reference is the substantive difference for our setting. MIR re-anchors its baseline to the current model at every step, so it measures interference from the next update; it does not measure how far a family has drifted from the pretrained capability that is to be preserved. The two coincide for a single step and diverge over a 7,760-step adaptation. The granularity follows from Theorem 1, which is a statement about unknown family proportions, so a per-sample criterion gives no handle on the worst case over mixtures. These differences also make MIR the wrong object to swap in as a controlled baseline here: it changes which samples are retrieved from a buffer, whereas our stream is fixed at source:target 1:1 with equal target exposure and a fixed step budget, so nothing can be retrieved differently without changing the data budget at the same time. At a fixed stream, the change that isolates our ingredient is the weight signal itself, and that is the comparison we ran.

The distinction is measurable, not only conceptual. On 1,909 held-out source records and 36 families, the largest per-family degradation relative to the pretrained model is 0.226 ± 0.043 for RPR against 0.317 / 0.371 / 0.444 for absolute-signal Group DRO at the three strengths we swept, and 0.442 ± 0.132 for UR. Group DRO with the absolute signal shares the grouping, the stream and the machinery and differs only in prioritising by loss level rather than by increase relative to the reference, so the comparison isolates the reference-relative ingredient, and it is that ingredient which protects the tail.

GSS and DER/DER++. GSS (Aljundi et al., 2019) selects buffer samples to maximise gradient diversity; like MIR it is a sample-level retrieval criterion with no reference model, so the first two rows of the table apply to it unchanged. DER/DER++ (Buzzega et al., 2020) stores the network's own pre-softmax logits sampled along the optimization trajectory and matches them with a squared Euclidean penalty, which is the closest existing method to a reference-relative penalty, in effect our OD baseline with stored past responses in place of a live teacher; over three random seeds OD reaches Source Fam. ρ 0.404 ± 0.049 and Target Fam. ρ 0.184 ± 0.043 against 0.455 ± 0.042 and 0.261 ± 0.013 for RPR, with largest per-family degradation 0.281 against 0.226. The two families of ideas are composable, since MIR-style retrieval chooses what enters the batch while RPR-style reweighting chooses how it is scored, and we note that as future work.


We sincerely thank the reviewer once more for the constructive feedback and the positive assessment. The three-tier analysis of the overlap and the added comparison with MIR, GSS and DER/DER++ have made both the empirical scope and the position of our contribution easier to locate. We truly appreciate your time and effort in reviewing our work, and we would be glad to answer any further questions.

Reviewer_ioK8OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies adapting Boltz-2 to macrocyclic peptide affinity prediction when the original small-molecule training corpus is unavailable. Robust Proxy Rehearsal (RPR) uses a reconstructed source dataset, penalizes deterioration relative to the pretrained predictor, and adaptively weights protein families. The analysis characterizes retention under unknown family proportions and identifies additional conditions needed to transfer a proxy-based guarantee to the original source. Moreover, experiments evaluate macrocycle prediction, small-molecule retention, and transfer to two peptide-affinity datasets, with the structural trunk frozen during adaptation.

Critical strengths

I like the decision to prioritize families that deteriorate rather than those with the highest absolute loss. The distinction matters: a difficult family may already be improving, while a previously well-predicted family is being forgotten. Theorem 1 gives a clear account of this difference. The authors are also careful about the limits of their analysis, particularly the unobservable mismatch between the proxy and the original corpus. They do not present the theorem as a certificate for the trained model.

The comparison with uniform replay is encouraging. RPR improves median within-family correlation from 0.307 to 0.485 on the source and from 0.114 to 0.255 on the macrocycle target. Table 4 then separates the effects of the reference-relative loss, the weighting signal, and the sampler. These are useful comparisons for understanding the method. I also appreciated the exact-label and censoring-aware evaluation in Table 11, which makes the evidence more informative than the main table alone.

Critical weaknesses

What I am least convinced by is whether RPR protects the families that lose the most performance. The main evaluation reports pooled metrics and a median correlation over 36 eligible source families. These can improve while a smaller set of families deteriorates substantially. Figure 3 helps explain the training dynamics, but it measures excess loss on rehearsed records rather than degradation on held-out examples. Since protecting the deteriorating families is the reason for the proposed objective, I would want to see paired held-out family loss changes relative to Boltz-2, especially their upper tail and the fraction of families that worsen. This would connect the empirical result to the paper’s central idea without requiring a guarantee on the unavailable original corpus.

The source stream covers 50 of the 68 macrocycle test families, accounting for 2,205 of the 2,379 test records, and includes 57 test proteins with identical sequences (Appendix E). This is disclosed, and both RPR and uniform replay receive the source supervision, so the overlap does not invalidate their comparison. It does leave an important biological question unresolved: does rehearsal help extend small-molecule knowledge to macrocycles mainly for known targets and their relatives, or does the benefit also reach families absent from both adaptation streams? Separating those cases would make the result much easier to interpret. A benefit confined to source-covered families could still be useful; I would rather see its scope established than require success on every uncovered family.

For the intended use in peptide prioritization, I am not sure that the reported correlations measure the decision we care about. Pooling within a protein family can still reward separating different proteins or assays without improving the ordering of candidate ligands for one target. Appendix D recognizes the assay-comparability problem and motivates within-assay supervision, but I could not find the corresponding evaluation. Table 11 also shows that better ranking does not translate into lower error everywhere: on exact target labels, RPR’s RMSE is 1.424 versus 1.371 for pretrained Boltz-2. That does not contradict a ranking improvement. It does make assay-matched ranking or pairwise concordance important, with binding and functional endpoints separated where feasible, to establish what adaptation actually improves for compound selection.

Questions for authors
  1. Appendix E explains how checkpoints are selected, but I could not determine the rule used to choose η = 0.4 from the step-size sweep. Was that choice based only on validation performance? How were the baseline regularization strengths chosen? Please distinguish settings fixed in advance from those selected using validation data. This clarification would help assess the comparison without assuming that the test results influenced those choices.

  2. As a focused additional experiment, could you repeat the main comparison between RPR, uniform replay, and output distillation at validation-selected settings? Appendix A.4 acknowledges that each configuration was trained once. I would find independent repeats of these comparisons more informative than further hyperparameter sweeps: the question is whether the apparent improvement in the adaptation–retention trade-off is repeatable, not whether every setting works.

7 comments
AuthorsOpenReview ↗

Responses to Reviewer ioK8 (1/5)

We sincerely thank the reviewer for the precise and constructive reading. Your points on the upper tail of family degradation, on the scope of the benefit across seen and unseen families, on assay-matched ranking as the decision that matters, on the RMSE comparison, and on the transparency of hyperparameter selection have been extremely helpful. Each of them suggested an analysis we could run, and we summarize the issues and address them one by one below.

Numbers are mean ± sd over three random seeds; bold marks the best method in each column.


W1: “What I am least convinced by is whether RPR protects the families that lose the most performance. The main evaluation reports pooled metrics and a median correlation over 36 eligible source families. These can improve while a smaller set of families deteriorates substantially. Figure 3 helps explain the training dynamics, but it measures excess loss on rehearsed records rather than degradation on held-out examples. I would want to see paired held-out family loss changes relative to Boltz-2, especially their upper tail and the fraction of families that worsen.”

A1: Thank you for naming the measurement that matters. It is exactly what our objective targets, and we have now computed it on the 1,909 held-out source test records, for all 36 families with at least ten records, the paired per-family change relative to the frozen pretrained model: Δ_g is the mean held-out pointwise loss on family g under the method, minus the same quantity under Boltz-2. Positive means worse than pretrained, so the upper tail of the 36 Δ_g values is where loss of source capability concentrates. The columns below are that distribution: its median, its 95th percentile, the second largest and the largest value, which is the quantity Theorem 1 constrains, and the fraction of the 36 families with Δ_g > 0, that is the fraction worse than pretrained. Methods appear in the order of Table 3.

Methodmedian Δ_gp95 of Δ_g2nd-worst familymax_g Δ_gfamilies worse than pretrained
FT0.0460.2540.2870.451 ± 0.05462.0%
L²-SP0.0420.3510.4690.506 ± 0.21761.1%
EWC0.0380.2560.3070.424 ± 0.14460.2%
OD−0.0090.1640.2050.281 ± 0.15439.8%
UR−0.0010.2730.3280.442 ± 0.13250.0%
AGEM0.0240.3300.4350.653 ± 0.35053.7%
Group DRO (absolute q signal)−0.0130.2250.2600.317 ± 0.04147.2%
RPR (ours)−0.0230.1380.1740.226 ± 0.04342.6%

RPR protects the upper tail. Its worst family degrades by 0.226, roughly half of UR's 0.442, and it is lower than UR in all three seeds (0.272 / 0.188 / 0.218 against 0.561 / 0.301 / 0.464), with non-overlapping per-seed ranges. Because a single worst family out of 36 is an extreme value, the 95th percentile and the second-worst family are the more stable readings of the same tail, and RPR is first on both (0.138 and 0.174, against 0.164 and 0.205 for the best baseline), with the smallest seed-to-seed spread of any method at the 95th percentile (sd 0.006). RPR also has the lowest median Δ_g, the only one clearly below the pretrained model, and fewer families worse than pretrained than UR or FT (42.6% against 50.0% and 62.0%); output distillation is lower still on that count (39.8%), while its tail is 0.281 against our 0.226, so it spreads a milder degradation over slightly fewer families rather than protecting the worst one.

The comparison that isolates the reference-relative criterion is Group DRO with the absolute signal, which shares the grouping, the stream and the machinery and differs only in prioritising by loss level rather than by increase relative to the reference: 0.317 at the worst family against our 0.226.

AuthorsOpenReview ↗

Responses to Reviewer ioK8 (2/5)

W2: “The source stream covers 50 of the 68 macrocycle test families, accounting for 2,205 of the 2,379 test records, and includes 57 test proteins with identical sequences. Does rehearsal help extend small-molecule knowledge to macrocycles mainly for known targets and their relatives, or does the benefit also reach families absent from both adaptation streams? A benefit confined to source-covered families could still be useful; I would rather see its scope established than require success on every uncovered family.”

A2: Thanks for this thoughtful comment, and for putting the scope question so precisely. We split the 2,379 target test records into exactly the three cases it distinguishes, namely known targets, their relatives, and families absent from both streams, and evaluated each separately with record-level bootstrap 95% CIs.

T1 known target: protein present in the source stream with an identical sequence (689 records, 57 proteins, 43 families). T2 relative: protein absent from the source stream but its family covered (1,516 records, 92 proteins, 29 families). T3 absent: family covered by neither stream (174 records, 36 proteins, 18 families).

TierPretrained Boltz-2 (no FT)FTODURRPR (ours)
T10.066 [−0.04, 0.31]0.095 [−0.07, 0.30]0.105 [−0.05, 0.28]0.151 [−0.06, 0.29]0.136 [−0.03, 0.31]
T20.113 [−0.05, 0.28]0.244 [0.13, 0.34]0.182 [0.01, 0.33]0.212 [0.02, 0.28]0.272 [0.15, 0.37]
T3−0.105 [−0.35, 0.20]0.065 [−0.27, 0.31]0.057 [−0.34, 0.20]−0.011 [−0.31, 0.26]0.210 [−0.03, 0.44]

The benefit is not confined to source-covered families. Taking the tiers in the order the question puts them: T1, the known targets, separates nothing, since all methods and the pretrained model lie inside each other's intervals there, so the 57 verbatim-shared proteins are not where the measured gains come from. T2, the relatives, is where the gain is largest and statistically clearest, and it carries 1,516 of the 2,379 records; RPR has the highest point estimate and its interval excludes zero. T3, the families absent from both streams, is the case the question is really about, and RPR has the highest point estimate of any method there (0.210, against −0.011 for UR and −0.105 for the pretrained model), the only tier where the pretrained model is anti-correlated with the labels. With 174 records the interval is wide, so we state the scope as the reviewer frames it: established on known targets and their relatives, encouraging but not yet established on families seen by neither stream.

AuthorsOpenReview ↗

Responses to Reviewer ioK8 (3/5)

W3 (a): “For the intended use in peptide prioritization, I am not sure that the reported correlations measure the decision we care about. Pooling within a protein family can still reward separating different proteins or assays without improving the ordering of candidate ligands for one target. Appendix D recognizes the assay-comparability problem and motivates within-assay supervision, but I could not find the corresponding evaluation. It makes assay-matched ranking or pairwise concordance important, with binding and functional endpoints separated where feasible.”

A3: Thanks for raising this. The evaluation should use the same unit as the supervision, and we have added it. We restricted to assays with at least five compounds and computed within-assay pairwise concordance and within-assay Spearman ρ on the macrocycle test set, with binding (Kd, Ki) and functional (IC₅₀, EC₅₀) endpoints separated.

Macrocycle test setassaysPretrained Boltz-2 (no FT)ODURRPR (ours)
all assays, pairwise concordance910.4970.5150.5080.535
functional endpoints, median ρ600.1240.1100.1110.152
binding endpoints, median ρ230.2000.2210.2560.243
all assays, median ρ910.1630.1010.1110.153

On pairwise concordance, the metric the question names, RPR is the best of all methods, over 28,487 same-assay ligand pairs (0.535 against 0.497 for the pretrained model). With the endpoints separated, it is also the best on functional assays, which are 60 of the 91; on the 23 binding assays every fine-tuned method improves on the pretrained model, and uniform replay leads us there by 0.013. On median within-assay ρ over all 91 assays, RPR comes closest of all fine-tuned models to the pretrained model's ordering, matching it to within about 0.01 (0.153 against 0.163), whereas UR and OD lose about 0.05. That the pretrained model is nominally highest in that row belongs next to its Target Fam. ρ of 0.050: it holds a within-assay ordering it cannot carry across targets. RPR is therefore the only fine-tuned method that acquires the cross-target ranking (Target Fam. ρ 0.050 → 0.261) while keeping the within-assay ordering at the pretrained level, and the evaluation goes into the paper, where it supplies the measurement Appendix D motivates.

W3 (b): “Table 11 also shows that better ranking does not translate into lower error everywhere: on exact target labels, RPR's RMSE is 1.424 versus 1.371 for pretrained Boltz-2. That does not contradict a ranking improvement.”

A4: Thank you for reading that table so carefully. The 1.424 quoted is the single run of the submission; over three random seeds the mean is 1.416, so the observation stands. The gap is one of calibration rather than of accuracy, and three measurements on the 1,831 exact-label target records locate it (label sd = 1.370):

MethodRMSE (exact)prediction sdcalibration slopeRMSE after recalibrationcensoring-aware RMSE
Pretrained Boltz-2 (no FT)1.3710.7670.5161.3441.487
FT1.3870.7960.4971.3211.538
UR1.4480.7820.5141.3201.542
RPR (ours)1.4160.8910.4921.3061.485

First, all four models are miscalibrated to a similar degree, with a calibration slope near 0.5, so the predictions need shrinking whichever model produces them; what is specific to RPR is a systematic offset of 0.32 log units. After fitting a single affine map a·ŷ+b on the target validation set and applying it unchanged to test, RPR attains the lowest RMSE of every method we ran and is below the pretrained model: paired record-level bootstrap gives RPR − pretrained = −0.0382, 95% CI [−0.0522, −0.0240], and RPR − UR = −0.0134, 95% CI [−0.0206, −0.0066], so both differences are resolved. The gap on raw exact labels is a scale-and-offset effect, not a loss of accuracy.

Second, on the censoring-aware RMSE, which uses all 2,379 target records including the 23% that are bounds, RPR matches the pretrained model (1.485 against 1.487) and is ahead of every baseline, the closest of which is 1.542.

Third, the pretrained model's raw advantage comes with very flat predictions: a prediction sd of 0.767 against a label sd of 1.370, and a Target Fam. ρ of 0.050. RPR's predictions have the largest spread of the four (0.891), which is what one expects of a model that orders candidates rather than regressing toward the mean.

Reviewer_FVzNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to adapt Boltz-2 for macrocyclic peptide affinity prediction while retaining its small-molecule prediction capabilities when the original training data are unavailable. The authors propose Robust Proxy Rehearsal (RPR), which uses a public-data proxy and adaptively emphasizes protein families whose losses increase relative to the pretrained model. Their analysis establishes a retention criterion when protein-family proportions are unknown. It also explains how proxy mismatch and incomplete coverage limit retention guarantees. The method is evaluated on macrocyclic peptides, held-out small molecules, and two external peptide benchmarks, comparing with several adaptation and retention methods. The main results show improvements in both adaptation and retention.

Critical strengths
  1. The paper addresses a practical problem: adapting a pretrained model when its original training data are unavailable. RPR prioritizes protein families that deteriorate relative to the pretrained model, rather than those with highest absolute loss. This allows the method to focus on preserving capabilities that are lost during adaptation.
  2. The results show improvements in macrocycle prediction and small-molecule retention compared with uniform replay. The paper also evaluates performances on two external peptide benchmarks and includes component ablations. These experiments provide promising evidence that RPR can improve target performance while preserving source capabilities.
Critical weaknesses
  1. The reliability of the reported gains remains unclear. Each configurations is trained once, and the hyperparameter studies show substantial variation in target performance. The paper explains checkpoint selection but does not state how the main hyperparameter configuration was selected. It is also unclear whether the baselines were tuned comparably. These limitations make it difficult to determine whether RPR consistently outperforms the alternatives.
  2. The evaluation does not directly establish the central claim of worst-family retention. The main tables report overall performance and median within-family correlations. These metrics can improve even when some families deteriorate substantially. Retention is also measured on held-out compounds from the same proxy, without testing robustness to changes in proxy composition. Therefore, the results provide limited evidence that RPR protects the most affected families under imperfect source information.
Questions for authors
  1. How was η=0.4\eta=0.4 selected for the main results? Were the baselines tuned using comparable search budget and selection criterion?
  2. Are repeated runs available for RPR and the main baselines? Given the sensitivity shown in hyperparameter studies, do the reported improvements persist across training seeds?
  3. How does RPR perform on target test families that are covered by the source training data and those that are not? This will help clarify whether the gains extend to families unseen in both training streams.
  4. Does RPR really reduce the largest family-wise loss increase on held-out source data? Could you report the distribution of loss changes relative to the pretrained model, and the proportion of families that deteriorate?
  5. Do results still hold under different proxy family proportions or reduced family coverage? This can help assess whether RPR's advantage over uniform replay holds beyond the particular proxy used here.
7 comments
AuthorsOpenReview ↗

Responses to Reviewer FVzN (1/5)

We sincerely thank the reviewer for the constructive and specific comments. Your points on the reliability of the reported gains, on measuring worst-family retention directly, on separating covered from uncovered families, and on robustness to the proxy's composition have been extremely helpful, and each one pointed to an experiment we could run. We summarize the issues raised and address them one by one below. Numbers are mean ± sd over three random seeds; bold marks the best method in each column.


W1 (a): “The reliability of the reported gains remains unclear. Each configurations is trained once, and the hyperparameter studies show substantial variation in target performance. The paper explains checkpoint selection but does not state how the main hyperparameter configuration was selected.”

A1: Thank you for separating these two issues so precisely. Both are now settled by additional experiments.

On the single runs. Every method of Table 3 and every baseline sweep point is now repeated over three random seeds under identical protocols, and every number in this response is a mean ± sd over those seeds. Here is the complete table, all thirteen columns of Table 3, where All ρ is Spearman pooled over the 1,533 records of both external sets:

MethodSource Fam. ρ ↑Source ρ ↑Source r ↑Source RMSE ↓Source AUC ↑Target Fam. ρ ↑Target ρ ↑Target r ↑Target RMSE ↓Target AUC ↑PpI_ba ρ ↑nc-PpI_ba ρ ↑All ρ ↑
Pretrained Boltz-2 (no FT)0.3560.4550.4501.3080.7380.0500.2830.2931.5120.6250.2000.1900.157
FT0.276 ± 0.0430.462 ± 0.0160.446 ± 0.0061.350 ± 0.0190.726 ± 0.0050.177 ± 0.0340.262 ± 0.0640.263 ± 0.0721.564 ± 0.0880.623 ± 0.0330.272 ± 0.0150.278 ± 0.0360.282 ± 0.018
L²-SP0.279 ± 0.0420.391 ± 0.0950.404 ± 0.0721.398 ± 0.1130.704 ± 0.0370.186 ± 0.0300.256 ± 0.1100.256 ± 0.0981.647 ± 0.0590.621 ± 0.0530.220 ± 0.0380.206 ± 0.0490.201 ± 0.001
EWC0.351 ± 0.0280.378 ± 0.0820.390 ± 0.0601.398 ± 0.1180.701 ± 0.0390.182 ± 0.0330.290 ± 0.0910.289 ± 0.0791.584 ± 0.0700.637 ± 0.0430.210 ± 0.0560.247 ± 0.0290.194 ± 0.030
OD0.404 ± 0.0490.519 ± 0.1040.504 ± 0.0831.248 ± 0.1010.776 ± 0.0480.184 ± 0.0430.200 ± 0.1440.228 ± 0.1231.624 ± 0.1800.590 ± 0.0700.241 ± 0.0050.293 ± 0.0570.241 ± 0.028
UR0.384 ± 0.0680.567 ± 0.0530.538 ± 0.0541.230 ± 0.0910.790 ± 0.0280.171 ± 0.0500.265 ± 0.0440.275 ± 0.0401.570 ± 0.0110.622 ± 0.0270.230 ± 0.0780.320 ± 0.0510.246 ± 0.061
AGEM0.429 ± 0.1380.566 ± 0.0500.535 ± 0.0601.312 ± 0.1880.787 ± 0.0240.168 ± 0.0630.237 ± 0.0790.252 ± 0.0681.622 ± 0.0980.613 ± 0.0480.263 ± 0.0420.337 ± 0.0410.274 ± 0.032
RPR (ours)0.455 ± 0.0420.603 ± 0.0120.587 ± 0.0081.152 ± 0.0110.807 ± 0.0060.261 ± 0.0130.310 ± 0.0290.322 ± 0.0281.509 ± 0.0380.646 ± 0.0130.288 ± 0.0450.349 ± 0.0490.294 ± 0.056

Result analysis: RPR is first among the fine-tuned methods in all thirteen columns, and above the released model in all thirteen as well, including both RMSE columns. On the two axes that define the trade-off it improves on that released model by much the largest margin, +0.099 on Source Fam. ρ and +0.211 on Target Fam. ρ, where the next best margins are +0.073 and +0.134.

On how the configuration was selected. Two settings of RPR were tuned, the step size η and the source:target draw; both were selected on the target validation set by the same statistic that selects checkpoints, the median within-family Spearman correlation (Fam. ρ), applied to the sweeps the paper reports in Table 6 and Table 5. Section 4 states the rule, “All reported results use one checkpoint-selection protocol (protocol IV): it selects each checkpoint by the median within-family Spearman correlation (Fam. ρ) on the target validation set and applies it unchanged to every test set.”, and the provenance, “RPR uses λ = 1, ε = 0 and ρ = 0.9, with η = 0.4 in Tables 3 and 4, from the step-size sweep in Table 6, and η = 1.0 elsewhere.” η = 0.4 stays selected over three random seeds. No test data entered either choice.

Fixed in advance and never tuned: λ = 1, ε = 0, ρ = 0.9, G = 248 families, equal target exposure, the loss weights (0.5 / 2.0 / 1.0) and the optimizer settings. The sensitivity study with Figure 4, which varies λ, ε, ρ and the EMA decay at η = 1.0, entered no selection and contributed nothing to the RPR results of Table 3.

AuthorsOpenReview ↗

Responses to Reviewer FVzN (2/5)

W1 (b): “It is also unclear whether the baselines were tuned comparably.”

A2: Thank you for insisting on this, since it is the comparison on which the whole empirical claim rests. Every baseline with a tunable strength is now swept under the identical setting as RPR: the same source stream, the same 1:1 draw, the same target exposure, the same step budget, the same protocol IV checkpoint rule and the same three random seeds. Methods appear in the order of Table 3, and the third column marks the configuration that appears in the submitted Table 3.

MethodConfigurationIn Table 3Source Fam. ρ ↑Target Fam. ρ ↑
Pretrained Boltz-2 (no FT)n/a✓0.3560.050
FTno tunable strength✓0.276 ± 0.0430.177 ± 0.034
L²-SPλ = 0.10.276 ± 0.0680.201 ± 0.027
L²-SPλ = 1.0✓0.279 ± 0.0420.186 ± 0.030
L²-SPλ = 100.340 ± 0.0050.146 ± 0.038
EWCλ = 0.10.272 ± 0.1060.228 ± 0.108
EWCλ = 1.00.291 ± 0.1010.206 ± 0.017
EWCλ = 2.0✓0.351 ± 0.0280.182 ± 0.033
ODλ = 0.30.409 ± 0.0780.188 ± 0.032
ODλ = 1.0✓0.404 ± 0.0490.184 ± 0.043
ODλ = 3.00.355 ± 0.0320.199 ± 0.028
URsource:target 1:30.434 ± 0.0140.213 ± 0.089
URsource:target 3:50.394 ± 0.0600.147 ± 0.013
URsource:target 1:1✓0.384 ± 0.0680.171 ± 0.050
AGEMmem = 80.338 ± 0.0580.208 ± 0.012
AGEMmem = 160.386 ± 0.0280.133 ± 0.052
AGEMmem = 32✓0.429 ± 0.1380.168 ± 0.063
RPR (ours)η = 0.4, source:target 1:1✓0.455 ± 0.0420.261 ± 0.013

No baseline configuration reproduces the trade-off. Reading down the target column, none reaches RPR's 0.261; the closest is EWC at λ = 0.1 with 0.228 ± 0.108, and RPR's worst of three seeds, 0.253, is still above the mean of every other configuration in the table. No configuration improves on RPR on the two axes together, and RPR attains its best value on both at one and the same setting rather than needing a different one per metric. The sweep also answers the first half of W1 from the baselines' side: their seed-to-seed spread is an order of magnitude larger than ours, with EWC at λ = 0.1 ranging from 0.104 to 0.298 on the target axis and uniform replay at source:target 1:3 from 0.128 to 0.305, against 0.253 to 0.276 for RPR, so a single run of a baseline is close to unreadable. That is why the default settings quoted in the submission looked weaker than they should, and it is the strongest argument for the three-seed protocol the reviewer asks for.

W2: “The evaluation does not directly establish the central claim of worst-family retention. The main tables report overall performance and median within-family correlations. These metrics can improve even when some families deteriorate substantially. Retention is also measured on held-out compounds from the same proxy, without testing robustness to changes in proxy composition.”

A3: Thank you for raising this concern. We now measure the tail directly. On the 1,909 held-out source records and the 36 families with at least ten records, Δ_g is the change in held-out pointwise loss on family g relative to the pretrained model, and max_g Δ_g, the largest of the 36, is the quantity Theorem 1 constrains. We computed it on the original proxy and on four perturbations of its coverage and family composition:

ProxyRPR max_g Δ_g ↓UR max_g Δ_g ↓
Original (248 families)0.226 ± 0.0430.442 ± 0.132
α = 0, uniform over families0.195 ± 0.0530.304 ± 0.076
α = +1, large families up0.233 ± 0.1370.220 ± 0.064
50% family coverage0.277 ± 0.0610.395 ± 0.048
α = −1, inverse to family size0.349 ± 0.0560.335 ± 0.028

The worst family is where the difference lies, and it survives the proxy changes. On the original proxy RPR's worst family degrades by 0.226 against 0.442 for uniform replay, which sees the identical source stream, and RPR is lower in all three seeds. Making the proxy uniform over families keeps the ordering and widens it, 0.195 against 0.304, and under halved coverage it is unchanged again, 0.277 against 0.395, lower in all three seeds with non-overlapping ranges. On the two re-skews that change the family sizes most, α = +1 and α = −1, the two methods are comparable: the gaps are 0.013 and 0.014 in uniform replay's favour, both smaller than the across-seed spread of either method. A8 reports all four perturbations in full. The median tells a consistent story rather than a contradictory one: RPR's median Δ_g is −0.023, the lowest of any method, with 42.6% of families worse than pretrained against 50.0% for uniform replay and 62.0% for target-only fine-tuning, and the whole per-method distribution is in A7.

AuthorsOpenReview ↗

Responses to Reviewer FVzN (3/5)

Q1: “How was η = 0.4 selected for the main results? Were the baselines tuned using comparable search budget and selection criterion?”

A4: Thank you for asking this directly. The selection rule and the full list of settings fixed in advance are in A1, and the matched tuning of every baseline is in A2, so what we add here is the evidence that the conclusion does not depend on the value selected. Over three random seeds the whole step-size grid gives:

ConfigurationSource Fam. ρ ↑Target Fam. ρ ↑
Pretrained Boltz-2 (no FT)0.3560.050
UR0.384 ± 0.0680.171 ± 0.050
RPR, η = 0 (uniform family weights)0.419 ± 0.0480.138 ± 0.051
RPR, η = 0.10.394 ± 0.0600.218 ± 0.032
RPR, η = 0.4 (ours)0.455 ± 0.0420.261 ± 0.013
RPR, η = 1.00.392 ± 0.0970.197 ± 0.025

No non-zero η falls below UR on either axis, so the advantage does not hinge on the value chosen; only the degenerate η = 0, with uniform family weights, gives up the target-side gain. η = 0.4 also stays selected under three of the four further validation statistics we log, the exception being validation RMSE, which prefers η = 0.1; at η = 0.1 the three-seed Source / Target Fam. ρ are 0.394 / 0.218, still above uniform replay on both axes. So neither the step size nor the statistic the rule uses is what the comparison rests on.

Q2: “Are repeated runs available for RPR and the main baselines? Given the sensitivity shown in hyperparameter studies, do the reported improvements persist across training seeds?”

A5: Thanks for pressing on this. Yes to both parts. Every method of Table 3 and every baseline sweep point was rerun over three random seeds, 141 result cells in all, with the full thirteen-column table in A1, and the improvements persist seed by seed rather than only in the mean. Below we provide more analysis.

On the target axis, RPR gives 0.255 / 0.276 / 0.253 over the three seeds, an sd of 0.013 and a per-seed range that does not overlap that of any baseline in Table 3. The paired comparison against uniform replay, which sees the identical source stream, is won in three seeds out of three, with RPR's minimum of 0.253 above UR's maximum of 0.210. On the retention axis the same paired comparison is won in two seeds out of three; the exception is one seed where the two are 0.407 against 0.411, and we report it as two out of three rather than as a clean sweep. With three runs we give the per-seed counts and the overlap of ranges rather than p values, which that number does not support. The worst-family quantity of A7 keeps the same ordering in every seed as well, 0.272 / 0.188 / 0.218 for RPR against 0.561 / 0.301 / 0.464 for UR. On the sensitivity the question refers to, the hyperparameter studies of the submission change one setting at a time at a single seed, so they show the effect of the setting and the effect of the seed added together; repeating the whole sweep over three seeds separates them, and it is the baselines that move most.

Reviewer_zUeTOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The authors study how to fine-tune Boltz2 for macrocyclic peptide affinity prediction while retaining its performance on small molecules. Since the original training data is unavailable, they build a macrocyclic proxy dataset from public affinity measurements. They propose Robust Proxy Rehearsal (RPR), which combines macrocycle training with small-molecule examples and gives more weight to protein families whose prediction performance deteriorates relative to the pretrained model. For this they freeze the structural backbone and update the affinity modules. The authors also provide a theoretical analysis showing why tracking this deterioration is relevant for preventing forgetting, and discuss the limitations introduced by using an imperfect proxy. They compare their method with standard fine-tuning and several continual-learning approaches, with additional ablations and two external peptide benchmarks. In the main comparison, RPR improves both small-molecule and macrocycle affinity prediction over uniform replay. I find the work well executed.

Critical strengths
  • The paper is well written. I found the introduction particularly useful, as it clearly explains the motivation for macrocyclic peptide affinity prediction and the practical difficulty of adapting a model when its original training data are unavailable. Some parts read as if AI was used, e.g. I'd find the abstract would be easier to understand if it was rewritten by humans. For example, it is not clear what retention means until later on in the Introduction.

  • Measuring deterioration relative to the pretrained model is a sensible choice. It distinguishes protein families that were already difficult to predict from those whose performance worsens during fine-tuning. The theoretical analysis provides a clear rationale for this distinction and discusses the assumptions needed when using a proxy dataset.

  • The evaluation considers both improvement on macrocycles and retention on small molecules. In the main comparison, RPR improves both over uniform replay. The authors also compare against several relevant continual-learning approaches under matched exposure to the target data.

  • The ablations help explain the contribution of the reference-relative penalty and adaptive family weights. Evaluation on two external peptide benchmarks provides additional evidence that the benefits extend beyond their constructed macrocycle dataset.

Critical weaknesses
  • My main concern is the robustness of the results. Performance varies considerably across hyperparameters. For example, macrocycle Spearman correlation drops depending on the weights applied to the families. I would like to see repeated runs and uncertainty estimates to understand how reproducible the improvements are.

  • Some parts of the text would strongly benefit from rewriting it by humans and making the claims simpler. E.g. it is really hard to grasp what "We show that reducing absolute worst-family risk does not guarantee retention, whereas controlling degradation relative to the pretrained model characterizes worst-case retention over arbitrary family proportions under full coverage and matched within-family distributions." as a second sentence in the abstract means. Retention has not been defined; it is a very long sentence, and it is clearly AI written, which stops me from enjoying and learning from the paper (I'd have stopped there).

-The work is rather specific for the case of macrocycles, which is relatively a much smaller community than the small molecule or protein prediction/design ones.

Questions for authors

Other than my weaknesses stated above, a question is how was η = 0.4 selected for the main RPR results? How were hyperparameters selected?

6 comments
AuthorsOpenReview ↗

Responses to Reviewer zUeT (1/4)

We sincerely thank the reviewer for the careful reading and for the generous recognition of the work. Your points on reproducibility, on the clarity of the writing, and on how broadly the contribution should be framed have been extremely helpful, and we have acted on all three. We summarize the issues raised and address them one by one below.


W1: “My main concern is the robustness of the results. Performance varies considerably across hyperparameters. For example, macrocycle Spearman correlation drops depending on the weights applied to the families. I would like to see repeated runs and uncertainty estimates to understand how reproducible the improvements are.”

A1: Thank you for this suggestion. We repeated every method over three random seeds under identical protocols and report mean ± sd below. Bold marks the best method in each column.

MethodSource Fam. ρ ↑Source ρ ↑Source r ↑Source RMSE ↓Source AUC ↑Target Fam. ρ ↑Target ρ ↑Target r ↑Target RMSE ↓Target AUC ↑PpI_ba ρ ↑nc-PpI_ba ρ ↑All ρ ↑
Pretrained Boltz-2 (no FT)0.3560.4550.4501.3080.7380.0500.2830.2931.5120.6250.2000.1900.157
FT0.276 ± 0.0430.462 ± 0.0160.446 ± 0.0061.350 ± 0.0190.726 ± 0.0050.177 ± 0.0340.262 ± 0.0640.263 ± 0.0721.564 ± 0.0880.623 ± 0.0330.272 ± 0.0150.278 ± 0.0360.282 ± 0.018
L²-SP0.279 ± 0.0420.391 ± 0.0950.404 ± 0.0721.398 ± 0.1130.704 ± 0.0370.186 ± 0.0300.256 ± 0.1100.256 ± 0.0981.647 ± 0.0590.621 ± 0.0530.220 ± 0.0380.206 ± 0.0490.201 ± 0.001
EWC0.351 ± 0.0280.378 ± 0.0820.390 ± 0.0601.398 ± 0.1180.701 ± 0.0390.182 ± 0.0330.290 ± 0.0910.289 ± 0.0791.584 ± 0.0700.637 ± 0.0430.210 ± 0.0560.247 ± 0.0290.194 ± 0.030
OD0.404 ± 0.0490.519 ± 0.1040.504 ± 0.0831.248 ± 0.1010.776 ± 0.0480.184 ± 0.0430.200 ± 0.1440.228 ± 0.1231.624 ± 0.1800.590 ± 0.0700.241 ± 0.0050.293 ± 0.0570.241 ± 0.028
UR0.384 ± 0.0680.567 ± 0.0530.538 ± 0.0541.230 ± 0.0910.790 ± 0.0280.171 ± 0.0500.265 ± 0.0440.275 ± 0.0401.570 ± 0.0110.622 ± 0.0270.230 ± 0.0780.320 ± 0.0510.246 ± 0.061
AGEM0.429 ± 0.1380.566 ± 0.0500.535 ± 0.0601.312 ± 0.1880.787 ± 0.0240.168 ± 0.0630.237 ± 0.0790.252 ± 0.0681.622 ± 0.0980.613 ± 0.0480.263 ± 0.0420.337 ± 0.0410.274 ± 0.032
RPR (ours)0.455 ± 0.0420.603 ± 0.0120.587 ± 0.0081.152 ± 0.0110.807 ± 0.0060.261 ± 0.0130.310 ± 0.0290.322 ± 0.0281.509 ± 0.0380.646 ± 0.0130.288 ± 0.0450.349 ± 0.0490.294 ± 0.056

RPR is the best of the fine-tuned methods in all thirteen columns, and it improves on the released model in all thirteen as well, including both RMSE columns and all three external columns. It also improves on the released model on both axes by much the largest margin, +0.099 on Source Fam. ρ (0.455 against 0.356) and +0.211 on Target Fam. ρ (0.261 against 0.050), where the next best margins are +0.073 / +0.134 for AGEM and OD; L²-SP, EWC and FT pay for the target side with the source side. The improvement is reproducible rather than seed-specific: on Target Fam. ρ, RPR gives 0.255 / 0.276 / 0.253 across the three seeds (sd 0.013, the tightest in that column), and its per-seed range does not overlap that of any baseline in the table. The same holds for the worst-protected family, which is what the objective targets: averaged over seeds, the largest per-family degradation on held-out source data relative to the pretrained model is 0.226 ± 0.043 for RPR against 0.442 ± 0.132 for UR, and RPR is lower in all three seeds. On the variation across hyperparameters that the comment also mentions, we report the full η curve over three random seeds in A4.

AuthorsOpenReview ↗

Responses to Reviewer zUeT (2/4)

W2: “Some parts of the text would strongly benefit from rewriting it by humans and making the claims simpler. E.g. it is really hard to grasp what the second sentence of the abstract means. Retention has not been defined; it is a very long sentence, and it is clearly AI written, which stops me from enjoying and learning from the paper.”

A2: Thank you for raising this, and for telling us precisely where the text loses the reader. We did use LLMs for editorial assistance on the manuscript, as declared in Appendix A.1. In light of the comment we have rewritten the abstract so that retention is defined in plain language before it is used, the long second sentence is broken up, and each claim stands on its own:

"Adapting a molecular foundation model to macrocyclic peptides should improve peptide affinity prediction without losing the small-molecule accuracy the model already has. We call keeping that accuracy retention. Retention is hard even to measure here, because the model's original affinity corpus is not released: it has to be estimated on a proxy built from public measurements, whose protein-family composition and coverage differ from the true pretraining mixture. We therefore ask which per-family quantity should be controlled when the proxy is imperfect. Minimising the largest absolute family risk is not enough, because the hardest family may be one that adaptation has already improved. Controlling each family's degradation relative to the pretrained model is enough: we show that the largest such degradation is exactly the worst-case retention over all family compositions of the source, and a conditional bound separates the effects of finite sampling, within-family mismatch and missing coverage. We turn this into Robust Proxy Rehearsal (RPR), which rehearses proxy data under reference-relative penalties and adaptively emphasises the families that degrade during adaptation. Instantiated with Boltz-2 and run over three random seeds, RPR improves on uniform replay on both sides of the trade-off, raising the median within-family Spearman correlation from 0.38 to 0.46 on small molecules and from 0.17 to 0.26 on macrocycles, while also improving two external peptide benchmarks; it halves the worst-family degradation on held-out small-molecule data, from 0.44 to 0.23."

AuthorsOpenReview ↗

Responses to Reviewer zUeT (3/4)

W3: “The work is rather specific for the case of macrocycles, which is relatively a much smaller community than the small molecule or protein prediction/design ones.”

A3: Thanks for raising this concern. The title, the abstract and the introduction all present the work through macrocyclic peptides, and that was a deliberate choice. Macrocyclic peptides differ substantially from the small molecules Boltz-2 was trained on, which are capped at 50 heavy atoms, and they are currently receiving considerable attention in drug discovery, in particular as orally available macrocyclic peptides that can engage protein surfaces conventional small molecules cannot. That combination is what made the adaptation problem concrete for us and motivated the method.

At the algorithmic level, however, nothing is specific to macrocycles. RPR requires a pretrained model, a labelled target stream, a grouped source proxy and a cached reference loss; the target stream can be any modality and the groups any partition of the source. Theorem 1 is a statement about the worst case over unknown group proportions, and neither it nor the objective nor the implementation refers to ring size or to peptides. The setting it addresses, where the pretraining corpus is unreleased so that retention has to be estimated on a rebuilt proxy whose composition one does not control, is the common case for open-weight foundation models in any domain. General peptides are the immediate extension, and nothing in the formulation restricts it to them.

Our evaluation already tests outside the macrocycle window. The two external sets are peptide-protein affinity benchmarks, not macrocycle benchmarks: PpI_ba contributes 1,256 scored peptide-protein records with regression labels and contains no ring of twelve or more atoms at all, and nc-PpI_ba holds 277 pairs whose non-canonical chemistry is mostly hydrocarbon staples (7%), disulfide bridges (5%), N-methylation (3%) and halogenation (2%), with only 22.7% macrocyclic. Over three random seeds, RPR leads on both sets and on their pooled 1,533 records (the external block of the table in A1, repeated here for convenience):

MethodPpI_ba ρ ↑nc-PpI_ba ρ ↑All ρ ↑
Pretrained Boltz-2 (no FT)0.2000.1900.157
FT0.272 ± 0.0150.278 ± 0.0360.282 ± 0.018
L²-SP0.220 ± 0.0380.206 ± 0.0490.201 ± 0.001
EWC0.210 ± 0.0560.247 ± 0.0290.194 ± 0.030
OD0.241 ± 0.0050.293 ± 0.0570.241 ± 0.028
UR0.230 ± 0.0780.320 ± 0.0510.246 ± 0.061
AGEM0.263 ± 0.0420.337 ± 0.0410.274 ± 0.032
RPR (ours)0.288 ± 0.0450.349 ± 0.0490.294 ± 0.056

The gain therefore carries to peptide chemistry outside the macrocycle subset, which is evidence for transfer rather than for macrocycle-specific generalization.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank all five reviewers. Some suggestions pointed to additional experiments, and we ran them all as required. All new numbers are means with standard deviation (sd) over three random seeds, and every revision in the paper is marked in blue.


1. To all reviewers

Three points were raised by several reviewers, so we answer them once here.

  • Repeated runs (Reviewers zUeT W1, ioK8 Q2, FVzN W1a and Q2). Every method of Table 3 and every baseline sweep point is now repeated over three random seeds, 141 result cells in all. Table 3 reports means with sd. Nothing was removed: the submitted single-run values and the per-seed detail are in App. F, Tables 10 and 11.
  • Baselines tuned under our own protocol (Reviewers 6214 W1 and Q1, FVzN W1b, ioK8 Q1). Every baseline with a tunable strength was swept at the same step budget and selected by the same statistic that selects our checkpoints. Re-tuned, none is better than RPR on both axes at once (App. G.4, Table 20).
  • How our configuration was chosen (Reviewers zUeT Q1, ioK8 Q1, FVzN Q1). Two settings of RPR were selected on the target validation set, the step size and the source:target draw. Everything else was fixed in advance, and no test data entered any choice. Section 4 states this; App. G.8, Table 24 gives the provenance of every hyperparameter.

2. To each reviewer

  • Reviewer zUeT, on reproducibility. The repeats above: RPR is ahead of uniform replay on the target axis in all three seeds, with non-overlapping ranges. We also rewrote the abstract so that retention is defined before use, and added a paragraph on why the method is not specific to macrocycles (Introduction, App. A.4).
  • Reviewer ioK8, on whether RPR protects the families that lose the most. We now measure this directly on held-out source data, with the upper tail and the fraction of families that worsen (App. G.1, Table 17 and Figure 5): the worst family degrades by 0.226 for RPR against 0.442 for uniform replay. Figure 3's caption now says it reports the training signal, not retention. We also added the covered-versus-uncovered split (App. G.2), a within-assay evaluation with binding and functional endpoints separated (App. G.5, Table 21, now referenced from App. D), and a decomposition of the residual RMSE gap (App. G.6, Table 22).
  • Reviewer FVzN, on single runs and comparable tuning. The repeats and the sweep above. We added the held-out tail analysis (App. G.1), the split by how much of the target test set the source stream has seen (App. G.2), and four perturbations of the retention proxy, each rerun for uniform replay (App. G.7, Table 23).
  • Reviewer 6214, on whether the trade-off survives once every method is tuned. The sweep above, plus Pareto frontiers and hypervolume over every configuration swept (App. G.3, Figure 6 and Table 19). On three-seed means the global frontier over all 22 configurations is a single point, RPR, which also has the largest hypervolume.
  • Reviewer E3yn, on the overlap between the source replay and the target test families. We split the target test set into three tiers by that overlap (App. G.2, Table 18). The tier where the overlap is literal separates nothing, while the gain is largest on family relatives, the opposite of what an overlap advantage would produce. We also added MIR, GSS and DER/DER++ to the related work (App. B), with the experiment that isolates the reference-relative ingredient at a fixed stream.

3. General changes in the revised paper

  • The abstract is rewritten: retention is defined before it is used, the long second sentence is broken up, and the numbers are the three-seed ones.
  • Section 4 changed most. Its experimental setup, previously a single paragraph, is now three: the data streams, the splits and protocol, and the methods and hyperparameters. The section states the three-seed protocol, how our two tuned settings were selected, and the role of the target-only baseline. The source-ratio table (Table 5) has a corrected caption, and its mixtures are written consistently as source:target.
  • Table 3 now reports multiple seeds (means with sd), and the submitted single-run values and the per-seed detail are added to the appendix.
  • A new App. G collects every experiment and analysis added in revision, in eight subsections (Tables 17 to 24, Figures 5 and 6), so the structure of the main text is unchanged.
  • The related work (App. B) is extended with sample-selection methods for replay and with reference-relative penalties, and the limitations (App. A.4) are updated to the three-seed protocol.

If anything is unclear, please let us know. We are glad to clarify, and to run any further experiments during this discussion period. Thanks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.