acceptodds
Under review as a conference paper at ICLR 2027

AMP-Modify: Chemical Modification Design of Antimicrobial Peptides with a Closed-Loop Framework

Abstract

Antimicrobial peptides (AMPs) offer a promising route to combating drug-resistant infections, but their therapeutic development requires addressing limitations in stability, selectivity, and activity. Chemical modifications, such as lipidation, cyclization, and terminal capping, provide a critical way to tune these properties beyond amino acid sequence optimization. However, computational AMP design has largely focused on sequence generation, leaving chemical modification underexplored. We formulate the modification problem as a joint decision over whether to modify a parent peptide, where, and what chemical structure to introduce. A central challenge is the mismatch between abundant general chemical knowledge and scarce experimental data linking parent peptides, modifications, and outcomes. Here, we develop AMP-Modify to address it by separating chemically explicit construction and validation from AMP-specific effect prediction. An LLM-assisted search uses general chemical knowledge to propose edit strategies within a modification space defined by a rule registry, while a molecular compiler instantiates and validates candidates. An AMP-specific modification evaluator trained on parent-modified relations scores each candidate relative to its parent, and its predictions are fed back to guide subsequent search and final selection. The evaluator outperforms existing methods in ranking modification effects, while case studies show that modifications proposed by our framework align with experimentally observed trends. AMP-Modify extends computational AMP design beyond sequence generation to chemically explicit modification.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Antimicrobial peptides, Chemical modification, Modification effect prediction, Closed-loop design

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_nooZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper formulates antimicrobial peptide (AMP) modification as a parent-relative decision: whether to modify a peptide, where to edit it, and which chemical structure to introduce. AMP-Modify combines an LLM-assisted search controller, a registry of 20 chemical edit types, a deterministic molecular compiler, and an AMP-specific evaluator that jointly represents peptide sequence, molecular structure, and edit events. The evaluator is trained using a curated corpus of 8,145 parent–modified relations, of which 980 high-confidence pairs provide direct pair-effect supervision. Reported evaluations include held-out pair ranking, ten literature cases, and seven retrospective search tasks. The framework improves construction efficiency and evaluator-qualified search yield over several controls; however, the experimental MIC support for final search recommendations remains indirect.

Critical strengths
  1. Well-motivated parent-relative formulation. Predicting whether an explicit edit benefits its parent is more decision-relevant than merely predicting whether the modified peptide remains antimicrobial. Treating retention of the parent as a possible outcome also makes the problem formulation practically meaningful.

  2. Executable, auditable chemical design. Separating LLM proposals from rule-governed construction and downstream evaluation reduces structurally invalid and duplicate candidates. Connected-component data splitting and the unusually detailed appendix provide a useful starting point for rigorous verification of this modular workflow.

Critical weaknesses
  1. Uncertain validity of the supervision and comparative evidence. Organism-group MIC aggregation does not, by itself, guarantee assay-comparable parent–variant effects. The C7 literature classification appears inconsistent with the manuscript’s molar two-fold threshold, and the stated Termini baseline architecture differs from its source. These issues affect the reliability of both targets and reported gains.

  2. The central closed-loop claim lacks an independent outcome test. The evaluator both guides search and determines which generated candidates count as improvements. Consequently, 57.3% evaluator-qualified yield does not establish a corresponding gain in measured antimicrobial potency. The actual final, post-filtered top-K recommendations—and whether retaining the parent is the right choice—are not directly validated against measured outcomes.

Questions for authors

Q1. Pair-label provenance and the C7 discrepancy. How are parent and modified-peptide MIC values matched before organism-group medians are formed—at the level of strain, study, and assay conditions—and how do results change on strictly matched pairs? In C7, the reported IDR1018 / PEG6-IDR1018 values (16 → 32 μg/mL; molecular weights 1535.8 / 1872.3 g/mol) convert to approximately 10.42 → 17.09 μM, a 1.64-fold increase rather than the two-fold molar increase required for “worsen” (Tables 2 and 8; Appendix C.1). What exact records justify that label? If these are the evaluated endpoints, please correct C7 and the nine-case summary. Please also quantify the effect of censored MIC substitutions on labels near the threshold.

Q2. Fidelity and fairness of baseline comparisons. Was Termini implemented as its original Chemprop graph model with Morgan-count features, or as the Morgan- count logistic classifier described in Appendix D.2? Please provide the exact model revisions, configuration, and output-to-pair-score mapping. If the latter is a simplified substitute, distinguish it from the original Termini architecture. Please also clarify whether organism-group information is supplied comparably to pairwise controls and what score is used in the “without pair loss” ablation, for which a mostly unsupervised pair head would not be a task-matched control.

Q3. Independent validation of the feedback loop. Can the claimed benefit of evaluator feedback be demonstrated on experimentally measured effects that are independent of the evaluator and unavailable through runtime retrieval? The 57.3% versus 48.4% strict-improve yields in Table 3A are defined by the same frozen evaluator used to steer the search, so they measure agreement with that evaluator rather than measured MIC improvement. A blinded retrospective benchmark on independently measured parent–variant series—with unmeasured proposed candidates explicitly separated— would directly assess whether feedback improves true enrichment.

Q4. Quality of deployed recommendations and abstention. What are the actual post-gate top-K outputs, and how often do experimentally beneficial edits enter those final lists? Appendix F.8 places exact C5, C8, and C10 literature edits at ranks 55, 55, and 84 in the full candidate pool, not in the improve-qualified shortlist; these ranks alone cannot establish whether the edits are recommended. Please report qualified ranks and compare the current MIC-activity-probability ordering with parent-relative alternatives. Please also evaluate cases where the appropriate decision is to retain the parent, including false-positive modification recommendations.

Q5. Exactness of chemical structures and claimed coverage. How sensitive are the literature-case conclusions to missing or approximate chemical representations? Appendix E.2 states that C4 and C6 lack complete molecular graphs and that C7, C8, and C10 use proxies; in particular, the C8 head-to-tail-cycle proxy differs from the reported side-chain cyclization. Can the authors use exact, stereochemically specified structures wherever available, rerun the relevant scores, and provide matched SMILES/event records? Independently checked examples should also demonstrate that the compiler constructs the intended edit, beyond passing graph-syntax checks.

Q6. Added value and repeatability of the LLM search policy. How much improvement is attributable specifically to LLM-guided strategy selection rather than shared registryconstraints, compilation, deduplication, allocation, and feedback? Please compare against a non-LLM policy with the same registry, compiler, prior evidence, candidate budget, and stopping rules, and distinguish usable candidates from actual measured improvements. Because the present search settings are run once per case with unspecified provider decoding defaults (Appendix F), repeated runs and per-case variability are important for interpreting the reported differences.

12 comments
AuthorsOpenReview ↗

Response to Reviewer nooZ, Q1

Q1. Question: Pair-label provenance and the C7 discrepancy. How are parent and modified-peptide MIC values matched before organism-group medians are formed—at the level of strain, study, and assay conditions—and how do results change on strictly matched pairs? In C7, the reported IDR1018 / PEG6-IDR1018 values (16 → 32 μg/mL; molecular weights 1535.8 / 1872.3 g/mol) convert to approximately 10.42 → 17.09 μM, a 1.64-fold increase rather than the two-fold molar increase required for “worsen” (Tables 2 and 8; Appendix C.1). What exact records justify that label? If these are the evaluated endpoints, please correct C7 and the nine-case summary. Please also quantify the effect of censored MIC substitutions on labels near the threshold.

Answer:

Thank you for pointing this out. Pair-effect labels are derived from matched experimental measurements, not differences between organism-group medians. For each parent--modified relation, we first match MIC records by species, strain, and activity-measurement group, requiring positive molar values from exact measurements or reported mean$\pm$SD. Among compatible records, the highest-scoring match is selected. Tier-A additionally requires the same publication and no conflicts in reported assay conditions (medium, CFU, pH, ionic strength, and salt type). The pair effect is then calculated directly from the matched μ\muM values; organism-group medians are used only for the absolute MIC task. Our audit confirms that all 980 Tier-A pairs satisfy the strain/study matching criteria and have no recorded assay conflicts. Missing metadata are not treated as conflicts. 562 pairs have at least one unreported assay field. We therefore additionally evaluated the 418 pairs with all five context fields available. On this stricter subset, family-balanced Spearman is 0.3310.331 versus 0.3780.378 and Pair Spearman is 0.4610.461 versus 0.4030.403 (Table 1), using the same out-of-fold predictions without retraining. Pair-ranking therefore remains in a comparable range; we do not assume that unreported assay conditions are identical. Section 4.2 reports the 0.3310.331 subset result, and the matching protocol is in the revised appendix.

We agree that the original worsen label is inconsistent with our molar MIC criterion. For the reported MRSA endpoint, IDR1018 and PEG6-IDR1018 have MICs of 16→32 μ16\rightarrow32\,\mug/mL and molecular weights of 1535.81535.8 and 1872.31872.3 g/mol, respectively. These correspond to 10.42→17.09 μ10.42\rightarrow17.09\,\muM, a 1.641.64-fold increase, which must be classified as neutral under our twofold threshold. We therefore relabel the reported C7 MRSA endpoint as neutral. The same publication reports 16→64 μ16\rightarrow64\,\mug/mL for PA14, E. coli, and A. baumannii, corresponding to a 3.283.28-fold molar increase and hence worsen; however, those distinct endpoints do not justify the original MRSA annotation. With the existing predictions unchanged, correcting C7 alone reduces our originally reported nine-case agreement from 7/9 to 6/9, while constant-improve remains at 7/9. Revised Table 2 and Section 4.3 use this corrected summary. The additional case-level corrections discussed in R1-Q8 are in the literature appendix.

None of the 980 Tier-A labels relies on censored MIC substitution. Both parent and modified measurements are exact or reported mean$\pm$SD values. The substitutions for censored measurements (<u↦u/2<u\mapsto u/2 and >u↦2u>u\mapsto2u) are applied only when constructing the separate organism-group absolute-MIC data, where 16,789 of 60,551 records (27.7%) contain censored measurements. Consequently, censoring substitutions change 0/980 pair labels, including those near the decision boundary. Separately, the choice of direction threshold does affect label assignments. Changing the default twofold cutoff to 1.5-fold or fourfold changes 101 or 267 labels, respectively (Table 2). This sensitivity reflects the classification threshold, not censored-value imputation. The revised appendix now makes this distinction and the matching protocol explicit.

Table 1. Pair-ranking performance under stricter assay-context matching. Both subsets use the same single-seed out-of-fold predictions without retraining.

Evaluation subsetnnImprove/Neutral/WorsenFamily-bal. Sp.Pair Sp.
Tier-A (same strain/study; no conflicts)980413/364/2030.3780.403
Complete assay context (five fields)418210/124/840.3310.461

Table 2. Sensitivity of Tier-A direction labels to the fold-change threshold. Changed labels are counted relative to the default twofold cutoff; no Tier-A labels use censored MIC substitutions.

ThresholdImproveNeutralWorsenChanged labels
1.5-fold455263262101
2-fold (default)4133642030
4-fold248631101267
AuthorsOpenReview ↗

Response to Reviewer nooZ, Q2(1/2)

Q2. Question: Fidelity and fairness of baseline comparisons. Was Termini implemented as its original Chemprop graph model with Morgan-count features, or as the Morgan-count logistic classifier described in Appendix D.2? Please provide the exact model revisions, configuration, and output-to-pair-score mapping. If the latter is a simplified substitute, distinguish it from the original Termini architecture. Please also clarify whether organism-group information is supplied comparably to pairwise controls and what score is used in the ``without pair loss'' ablation, for which a mostly unsupervised pair head would not be a task-matched control.

Answer:

We thank the reviewer for raising this point. We used a simplified adaptation, not the original Chemprop architecture, for the Termini result in the original Table 1A. Specifically, it uses 2048-dimensional Morgan-count fingerprints (radius 2), organism-group one-hot features, StandardScaler, and class-balanced logistic regression (maxiter=1000\mathrm{max_iter}=1000), retrained on our connected-component folds. To test whether this simplification accounts for the observed performance gap, we additionally retrained a Chemprop D-MPNN with Morgan-count features following Termini's published architecture. This implementation uses Chemprop 1.5.2, seven organism-group classification outputs, and 10 training epochs under the same five-fold protocol. We explicitly distinguish it from the original species-specific implementation (Chemprop 1.7.1) rather than claiming an identical reproduction. The D-MPNN obtains a five-seed family-balanced Spearman of 0.245±0.0230.245 \pm 0.023 on 951 scorable Tier-A pairs, compared with 0.211±0.0000.211 \pm 0.000 for the logistic adaptation (Table 1). Revised Table 1A reports this D-MPNN as Termini. The logistic substitute is appendix-only. The 29 gaps are pairs that a SMILES graph model cannot ingest, typically PEG, Fmoc, or metallocene edits, not pairs dropped from the benchmark. These additional results indicate that the main performance difference is not simply explained by replacing the graph architecture with logistic regression.

We provide organism-group information to the applicable activity-based controls. Termini-LogReg uses one-hot features, Termini-Chemprop uses group-specific output tasks, and AMPGAN/AMPGAN-e use organism-group indices. Our evaluator incorporates the group identity directly into its supervised pair head. For both Termini implementations, the relative score is spair=logit⁡(Pactive,parent(g))−logit⁡(Pactive,modified(g))s_{\mathrm{pair}}=\operatorname{logit}(P_{\mathrm{active,parent}}^{(g)})-\operatorname{logit}(P_{\mathrm{active,modified}}^{(g)}), where gg denotes the organism group. This difference is aligned with Δlog⁡2MIC\Delta\log_2\mathrm{MIC} (modified minus parent). AMPGAN similarly derives pair scores from differences in absolute activity predictions. These methods do not train a dedicated pair-effect head. DeepDelta directly predicts molecular property differences but does not receive organism-group information in our implementation. The revised appendix records these distinctions.

We agree that the original ``w/o pair loss'' evaluation was not a task-matched absolute-MIC control. The reported family-balanced Spearman of 0.0530.053 (Pair Spearman −0.167-0.167) was computed from a pair head whose supervision was disabled. We therefore additionally score the same ablated model using the parent--modified difference from its trained absolute MIC head, obtaining an MIC-Δ\Delta Spearman of 0.3190.319, compared with a pair-head Pair Spearman of 0.4030.403 for the full model (Table 2). Revised Table 1B reports both rows. The MIC-Δ\Delta score is the appropriate absolute-prediction control. The unsupervised pair-head result is retained only as a diagnostic, rather than being used to argue that absolute activity prediction cannot recover relative effects. Checkpoint selection also switches from validation pair performance to validation MIC F1 when pair supervision is removed. These corrections provide a more faithful assessment of the contribution of explicit pair supervision.

AuthorsOpenReview ↗

Response to Reviewer nooZ, Q2(2/2)

Table 1. Comparison of the two Termini implementations (five-seed mean ±\pm standard deviation). Revised Table 1A reports the D-MPNN. The 29 Chemprop gaps are pairs that a SMILES graph model cannot ingest, not pairs dropped from the benchmark.

ImplementationCoverageFamily-bal. Sp.Pair Sp.Top-20%
Termini-LogReg (appendix)980/9800.211±0.0000.211 \pm 0.0000.302±0.0000.302 \pm 0.0000.561±0.0000.561 \pm 0.000
Termini-Chemprop (D-MPNN)951/9800.245±0.0230.245 \pm 0.0230.294±0.0550.294 \pm 0.0550.630±0.0350.630 \pm 0.035
AMP-Modify-Eval980/9800.350±0.032\mathbf{0.350 \pm 0.032}0.393±0.026\mathbf{0.393 \pm 0.026}0.653±0.0810.653 \pm 0.081

Table 2. Task-matched comparison of pair-effect scores (single seed). The MIC-derived difference is the Spearman correlation between predicted and measured Δlog⁡2MIC\Delta\log_2\mathrm{MIC}; it is not a supervised pair-head score. NR means family-balanced Spearman was not computed for that score.

Configuration/scoring methodPair Sp.Family-bal. Sp.MIC-Δ\Delta Sp.
Full model / supervised pair head0.4030.4030.3780.3780.2620.262
w/o pair loss / unsupervised pair head−0.167-0.1670.0530.053—
w/o pair loss / MIC-derived difference——0.319\mathbf{0.319}
Reviewer_yPkoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents AMP-Modify, a closed-loop framework for computational antimicrobial peptide (AMP) chemical modification.

Critical strengths
  1. The paper considers an important problem
  2. The framework is evaluated on a curated corpus of 8,145 parent-modified relations (with a 980-pair Tier-A benchmark), a set of 10 literature case studies, and end-to-end search budget experiments.
Critical weaknesses
  1. Technically, the paper is not novel
  2. The literature case studies are limited.
  3. The way the baselines have been selected is not acceptable. For example, DeepDelta was not considered as a primary baseline
  4. Lot of heuristics have been used. It is not clear what their contributions are
Questions for authors

Please discuss how the framework handles molecules for which it is not possible to map 3D conformation, stereochemistry, and complex macrocycles through simple event-vector linearizations.

Reviewer_TWNgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets chemical modification of existing AMPs rather than sequence generation, framed as a joint decision over whether, where and what to modify, judged relative to the parent and with retaining the parent a valid output. The system separates the task by the kind of supervision each part needs. A versioned rule registry and a deterministic compiler define and validate an executable space of construction types, AMP-Modify-Eval scores each compiled candidate relative to its parent from matched parent-variant MIC contrasts, and an LLM-assisted controller proposes bounded edit strategies rather than molecules, with evaluator scores written back to guide later proposals and final selection. Three contributions are claimed, namely the parent-relative formulation, the closed-loop framework, and the evaluator, and the validation follows that structure. An ablation removes the parent-relative objective, a comparison against modification-aware predictors on held-out pairs runs alongside a panel of withheld literature modifications, and end-to-end search is tested against an ungrounded LLM, a no-feedback run and a registry-only random baseline, measured by proposal yield, evaluator-labelled improvement rate, and recovery of published edits withheld from the prompts.

Critical strengths

The authors pick a problem the AMP design literature has mostly stepped around, and chemical modification is how a lot of real AMP engineering happens, so the gap is worth the machinery they build for it. The paper defends three contributions, and the two strengths below reach the first one fully and only the construction half of the second. I treat the loop and the evaluator’s margin in the weaknesses.

1. The parent-relative formulation, the first contribution, is established cleanly, and I consider it the paper’s real result.

Removing the pair objective drops family-balanced Spearman from 0.378 to 0.053 while MIC F1 rises to 0.748 (Table 1B), and SeqOnly reaches the same place from another direction (MIC F1 0.711, family-balanced 0.068). Two independent routes, with a drop an order of magnitude past the seed std, so this holds under the seed concerns I raise later. It shows that the absolute-activity predictors the field uses to screen modified peptides carry almost no signal about whether an edit helped its parent, and the authors demonstrate that rather than assert it. The released registry and curated relations give the subfield something it did not have.

2. The construction half of the second contribution holds up, and the authors show where it matters rather than only that it works.

AMP-Modify returns 628 unique valid candidates against 266 for the Raw LLM over 700 slots, accounted per case and granular enough to check (Tables 3A and 11). Table 3B is the more convincing evidence, because it separates two regimes instead of averaging them. Both systems handle canonical N-terminal lipidation, which language-model priors already cover. Only the compiled pipeline reaches L13-Ahx and the glutamate side-chain cyclization, and neither reaches the C4 stapling site. That is a specific, falsifiable statement about when registered construction earns its cost.

Critical weaknesses

My concerns are not about the quality of the engineering but about which of the three contributions the evidence reaches. What is well supported concerns representation and supervision; what is unsupported concerns the loop and the selection step it feeds. The two words doing the most work in the title, "closed-loop", are precisely the part with no evidence external to the evaluator's own scores, and the one test of the framework's actual output against known chemistry shows that output failing. Reframing around what is demonstrated would leave a stronger paper than defending the loop with evaluator-internal metrics.

1. The LLM and the feedback loop, which carry the second and third contributions, have no evidence external to the evaluator's own scores.

On the LLM: the base model is named only in Appendix F.2, never in the main text, with temperature, top-p and decoding seed unset, provider defaults unrecorded, snapshots mixed across two versions, one run per case, and no ablation over model family or size anywhere. For the component that decides which chemistry gets explored, this is not enough to reproduce or to attribute. On the loop: strict-improve yield counts candidates the frozen evaluator labels improve, and that same evaluator supplies the feedback the search optimizes, so 57.3% versus 48.4% cannot separate better design from better exploitation of the evaluator's failure modes. The non-circular measure points the other way. Table 10 shows Registry-Random, with no LLM and no feedback, also constructs exact L13-Ahx and the exact glutamate cycle, and the authors themselves conclude those recoveries are a property of the registered space.

2. The selection step, which produces the framework's actual output, fails on the chemistry the search correctly reached.

The stated deliverable is a ranked shortlist truncated to K = 20, and Table 12 is the only test of that shortlist against experimentally characterized edits. Of the four exactly recovered increments, three rank 55, 55 and 84 in the unique-valid pool, so none reaches the returned list. The one that ranks 1 is C6, whose reported effect is neutral and which is therefore not improve-qualified. I read this as a consequence of ordering qualified candidates by delta-P_mic, a difference of two absolute activity predictions, rather than by the pair head the paper argues is the signal that matters. The result sits in Appendix F.8 and is not reflected in the claims, but it bears more directly on whether the framework works as a design tool than any number in the main text.

Questions for authors

The first four questions concern the two weaknesses above; the rest are clarifications I was unable to resolve from the text.

1. Please name the base model and its decoding configuration in the main text, pin a single snapshot, and report multiple runs per case with dispersion for Table 3A. Does the 57.3% versus 48.4% gap survive run-to-run variance?

2. Does the search result hold with an open-weights model, and how does it vary with model size?

3. Given Table 12, what is shortlist recall at K = 20 when qualified candidates are ordered by P(improve) instead of delta-P_mic? Why is the pair head used as a gate but not as the ranking signal, when the paper argues it is the quantity that matters?

4. C7 is excluded from the search panel because PEGylation worsens MIC, but negative cases seem the most informative test of whether the system declines to modify. What does the system return on C7, and how often is the empty shortlist actually produced?

5. What do the roughly 7,165 non-Tier-A relations contribute? If only connected-component topology for the fold split, please state that plainly and revise the abstract's "trained on 8,145 parent-modified relations".

6. Appendix C.2 says the default run tracks the pair task for early stopping and checkpoint selection. On which data? If the held-out fold, the Tier-A numbers are optimistically biased relative to the fold-retrained baselines.

7. Please report Table 1A as five-seed mean and standard deviation for the method and for every retrained baseline, with per-organism-group pair counts, and add Pair Spearman to the ablation tables. Which dashes are capability limits and which are reporting choices?

8. The literature panel contains seven improvements out of nine, so a constant "improve" predictor also scores 7/9. What evidence separates the evaluator from that baseline on this panel? Relatedly, C4 and C6 are scored with a zeroed molecular branch and C7, C8 and C10 with structural proxies, so how should direction agreement on non-exact inputs be read?

7 comments
AuthorsOpenReview ↗

Response to Reviewer TWNg, Q1-Q2

Q1. Question: Please name the base model and its decoding configuration in the main text, pin a single snapshot, and report multiple runs per case with dispersion for Table 3A. Does the 57.3% versus 48.4% gap survive run-to-run variance?

Answer:

We thank the reviewer for highlighting the importance of reproducibility and run-to-run variability. The planner uses gpt-5.6-sol (snapshot 2026-07-09), with temperature 1.01.0 and top-p=1.0p=1.0. The revised appendix (End-to-End Search Details) records this configuration.

To evaluate the stability of the reported improvement, we conducted three independent runs (seeds 42, 43, and 44) for both AMP-Modify and w/o feedback across seven search cases, using the same budget of 100 proposal slots per case (700 per run). Table 1 reports mean ±\pm sample standard deviation. These three-run means are the numbers in revised Table 3A.

Run-to-run repeatability across three independent runs (seeds 42, 43, and 44) on seven search cases. Each run uses 700 proposal slots. Standard deviations for percentage metrics are expressed in percentage points (pp).

MethodStrict-improve yield (%)Unique-valid yield (%)Improve / valid (%)
AMP-Modify56.33±0.84\mathbf{56.33 \pm 0.84}86.48±2.82\mathbf{86.48 \pm 2.82}65.17±1.14\mathbf{65.17 \pm 1.14}
w/o feedback48.76±2.7748.76 \pm 2.7783.43±2.3583.43 \pm 2.3558.42±1.7158.42 \pm 1.71

The repeated experiments retain a 7.57-percentage-point average advantage in strict-improve yield over w/o feedback, compared with the 8.9-point gap in the original single-run results. AMP-Modify also exhibits lower run-to-run variability in this metric. These results support the repeatability of the feedback benefit in terms of evaluator-predicted improvements under the tested search settings (revised Section 4.4).

Q2. Question: Does the search result hold with an open-weights model, and how does it vary with model size?

Answer:

We thank the reviewer for this suggestion. To examine the dependence on the LLM backbone, we additionally evaluated AMP-Modify using two open-weight models of different sizes: DeepSeek-V4-Pro (1.6T total / 49B active parameters) and DeepSeek-V4-Flash (284B total / 13B active parameters).

Both models were evaluated under the same AMP-Modify search framework, with a budget of 700 proposal slots. The results are summarized below.

Search performance with different open-weight LLM backbones.

ModelActive paramsUnique-valid yield (%)Predicted-improve yield (%)Improve / valid (%)
DeepSeek-V4-Pro49B82.054.366.2
DeepSeek-V4-Flash13B83.055.667.0

Generalization across LLM backbones. Both open-weight models achieve high unique-valid yields (82.0--83.0%) and predicted-improve yields (54.3--55.6%), comparable to the results obtained with our original LLM backbone. This demonstrates that the search framework can operate effectively with different open-weight LLMs rather than relying on a single proprietary model.

Effect of model size. Despite its substantially smaller parameter count, DeepSeek-V4-Flash achieves performance comparable to DeepSeek-V4-Pro, with slightly higher yields in this experiment. These results suggest that increasing model size does not necessarily improve search performance within the tested model configurations.

AuthorsOpenReview ↗

Response to Reviewer TWNg, Q3-Q4

Q3. Question: Given Table 12, what is shortlist recall at K = 20 when qualified candidates are ordered by P(improve) instead of delta-P_mic? Why is the pair head used as a gate but not as the ranking signal, when the paper argues it is the quantity that matters?

Answer:

We thank the reviewer for this valuable suggestion. We agree that using P(improve)P(\mathrm{improve}) for final ranking is more consistent with our parent-relative objective. The revised manuscript now orders qualified candidates by s=P(improve)s=P(\mathrm{improve}) and keeps ΔPmic\Delta P_{\mathrm{mic}} as an auxiliary activity gap (Section 3.4).

We compared the two ranking strategies while keeping the generated candidate pools and improvement gate unchanged. Here, the rank refers specifically to the position of the exact experimentally reported literature modification among the candidates predicted as improved for each case, rather than the ranking of the case itself. Post-gate ranks are also in the revised appendix.

Post-gate ranks of exact literature modifications under two ranking strategies.

CaseRank by ΔPmic\Delta P_{\mathrm{mic}}Rank by P(improve)P(\mathrm{improve})
C5554
C84850
C10822

C3 and C6 are not included in the ranking table because their exact literature modifications are absent from the improvement-qualified candidate pool, for different reasons. For C3, the framework generated other candidates predicted as improved, including modifications similar to the reported edit, but not the exact literature modification. For C6, the exact literature modification was generated but predicted as neutral and therefore excluded by the improvement gate, consistent with its experimentally measured neutral effect.

Across the four experimentally beneficial literature modifications (C3, C5, C8, and C10), Recall@20 increases from 0/4 (0%) to 2/4 (50%) when ranking by P(improve)P(\mathrm{improve}). C3 is counted as a miss because the exact edit was not generated, while C6 is excluded from the denominator because its measured effect is neutral.

These results support adopting P(improve)P(\mathrm{improve}) as the final ranking criterion, which the revision implements. This adjustment only changes the ordering of qualified candidates, without affecting candidate generation, chemical validation, or the improvement gate.

Q4. Question: C7 is excluded from the search panel because PEGylation worsens MIC, but negative cases seem the most informative test of whether the system declines to modify. What does the system return on C7, and how often is the empty shortlist actually produced?

Answer:

We thank the reviewer for raising this important question. As also pointed out by Reviewer nooZ, the original classification of C7 as worse requires correction. After re-examining the reported MIC values and molecular weights, we confirm that the molar MIC increases from 10.42 to 17.09 μ\muM (1.64-fold), which does not reach our twofold worsening threshold. Therefore, the PEG6 modification is classified as neutral rather than worsening. Revised Table 2 uses this label, and Section 4.3 reports 6/9 agreement on the nine MIC-labeled cases.

We additionally evaluated C7 using the same improvement-seeking search objective, with three independent runs (seeds 42, 43, and 44). The results are summarized below and in the revised appendix.

C7 search results across three independent runs.

SeedValid candidatesImprove-qualified candidatesPEG6 passes improve gate
428668No
436147No
446148No

The literature-reported PEG6 modification was generated in all three runs but did not pass the improvement gate. However, other candidates were predicted as improved, resulting in nonempty shortlists in all three runs. This is consistent with our design: a neutral modification does not necessarily imply that no other modification of the same parent could be beneficial.

Regarding abstention, across the original seven-case search panel repeated three times (21 case-runs), empty shortlists occurred in 6/21 runs (28.6%), corresponding to C3 and C6 in all three repetitions. The framework returns an empty shortlist when no generated valid candidate passes the improvement gate. This is a model-based abstention decision rather than a guarantee that no experimentally beneficial modification exists. The C7 runs and the 6/21 count are in the revised appendix.

AuthorsOpenReview ↗

Response to Reviewer TWNg, Q5-Q6

Q5. Question: What do the roughly 7,165 non-Tier-A relations contribute? If only connected-component topology for the fold split, please state that plainly and revise the abstract's "trained on 8,145 parent-modified relations".

Answer:

Thank you for pointing out this ambiguity. The 8,145 curated parent--modified relations serve two distinct purposes. First, all 8,145 relations are used to construct the parent--variant graph and determine connected components for leakage-resistant five-fold data splitting. The remaining 7,165 non-Tier-A relations contribute to this graph topology but are not used as pair-effect supervision. Second, only the 980 high-confidence Tier-A relations, with matched parent--modified MIC measurements, are eligible for the pair-effect loss in our main evaluator. Separately, peptides appearing in the graph may contribute their individual MIC, HC50, and IC50 measurements to the absolute-property prediction tasks, where such measurements are available.

The revised abstract states that the evaluator uses pair supervision on 980980 of 8,1458{,}145 parent-modified relations. Section 4.1 and the method keep the same distinction. The 8,1458{,}145 relations define the connected-component graph, and only the 980980 Tier-A pairs enter the pair loss.

Q6. Question: Appendix C.2 says the default run tracks the pair task for early stopping and checkpoint selection. On which data? If the held-out fold, the Tier-A numbers are optimistically biased relative to the fold-retrained baselines.

Answer:

Thank you for pointing this out. Using the held-out fold for checkpoint selection would indeed bias the reported Tier-A numbers relative to the fold-retrained baselines. We do not do this. Our evaluation follows a nested validation protocol within the five-fold connected-component split.

Specifically, for each outer fold, we reserve the held-out components exclusively for testing and further split the remaining training components into training and inner-validation subsets. Early stopping and checkpoint selection are performed solely on the inner-validation set, using family-balanced Pair Spearman as the selection metric (patience = 8). For the "w/o pair loss" ablation, the selection metric is switched to validation MIC F1. The reported Tier-A results are computed from out-of-fold predictions on the five held-out test folds, which are never used for model selection. Therefore, the reported performance is not subject to the suggested test-fold selection bias. The revised appendix (Evaluator Data and Training) now states this nested protocol explicitly.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to All Reviewers

We sincerely thank all reviewers for their constructive feedback, thoughtful suggestions, and recognition of the motivation and value of our work.

We greatly appreciate the opportunity to clarify and strengthen AMP-Modify. We have carefully addressed the technical and experimental questions raised by the reviewers in our detailed point-by-point responses, including dataset construction, evaluation protocols, baseline fairness, search reproducibility, chemical structure fidelity, and final recommendation quality. We have also conducted extensive additional experiments, including repeated search evaluations, open-weight LLM comparisons, and more comprehensive baseline analyses. Importantly, we expanded our literature validation with 11 additional case-endpoints covering improving, neutral, and worsening modifications. On this additional panel, AMP-Modify-Eval achieves 8/11 direction agreement, compared with 5/11 for the constant-improve baseline, and also demonstrates favorable performance relative to the evaluated competing methods.

These additional results further support the effectiveness of our parent-relative modification evaluation framework. We have updated the main manuscript and supplementary materials accordingly, incorporating the new experimental results, methodological clarifications, and corrected annotations. We believe these revisions substantially strengthen the clarity, reproducibility, and experimental evidence of our work. We sincerely thank the reviewers again for their valuable feedback and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.