- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces PepFM-TanRL, which uses a reward-based finetuning of a flow matching model. The method introduces time-decaying noise into ODE sampling and extends GRPO objectives to different modalities. The main result independently fine-tunes models for four peptide-target complexes and shows improved reward and clash-free rates. The paper also shows a separate 128-target evaluation of pretrained models; however, the 128 target evaluation does not evaluate the proposed RL methods.
Critical strengths
I think combining noise injection during sampling with policy regularization is interesting. The noise gives the sampler some kind of stochastic transition property, and the KL term explicitly penalizes the pretrained reference policy. I think it's a potentially useful way to adapt pretrained generative models.
Critical weaknesses
- The presentation makes the method and its contribution hard to read and follow. The paper needs to give a clearer explanation of the method, rather than showing too many implementation details. I would also like to see what has changed and why we need these changes for PepFlow. The case studies are in the Appendix, which makes it hard to understand the results. Also, there are some weird names, such as pepflowww_repro; I suggest the authors do some proofreading of the paper.
- The experimental results use very small datasets, and it's unclear how to find the relationships between different experimental sections.
Questions for authors
- Could you provide a self-contained explanation of how the method modifies PepFlow, and clearly explain each part of the method?
- Can the current method show consistently better results on larger and more diverse datasets?
- How does the final fine-tuned model compare with PepFlow pretrained weights? I also think we need more ablation studies on different parts of the method.
4 comments
A short note to flag that the run we mentioned in our reply to Q4 has finished.
The factored importance ratio is now ablated against a single joint-likelihood ratio, so Appendix C ablates three things rather than two: the sequence parameterization, the advantage/reward rescaling strategy, and the construction of the importance ratio itself. The new material is a table of both runs at 3,000 and 6,000 steps, in the format of the per-checkpoint tables and with the equivalent best-of-N added, and a figure of the clip fraction over training.
The numbers and the caveats are in the general follow-up comment on this submission. If anything there raises further questions, we would be glad to answer them in this thread.
I appreciate the authors’ response and their careful answers to my questions. I have increased the presentation and soundness scores. However, I still believe my initial overall rating reflects my assessment of the paper, and therefore I would like to keep it.
Thank you for the careful reading in both rounds, and for revisiting the presentation and soundness scores after our response. We appreciate the reconsideration, and we respect your decision.
One addition since your comment: the joint-likelihood-ratio ablation, which our earlier follow-up reported on 7YC2 alone, finished in time on a second target (1SSH, a PepFlow-split target evaluated with the other reward version), so Appendix C now covers both. We added it so that the ablation does not rest on a single target; the numbers are in our general follow-up comment on this submission.
Thank you again for the questions you raised — they sharpened several parts of the paper.
Response to Reviewer eKgM
Thank you - the presentation criticisms were fair and we acted on all of them. Our general response summarizes the revision; below we answer your points.
Q1 (self-contained explanation of how the method modifies PepFlow). Added as Appendix B, which can be read without the PepFlow paper. It opens with what we keep (the flow-matching formulation, dataset, pocket definition, loss terms and weights, IPA backbone), and Table 2 then lists all five changes - torsion state, side-chain feature aggregation, sequence target per step, side-chain feature content, time conditioning - each with its reason and a pointer to the subsection that details it (App. B.1-B.4). The first three exist for one reason, which we now state plainly: an importance ratio needs the probability of each sampled transition, and PepFlow draws a discrete amino acid type inside the generation loop, which makes that density unavailable in closed form. The last two are independent architectural choices.
Weird names. Removed.
pepflowww_orig/_oh/pepflowww_repro/_ohare now "PepFlow (released)" and "PepFlow (retrained)", with a separate column for whether the sequence target is continuous or one-hot - the two axes the suffixes encoded. The dataset file name and the code-style typesetting of MSE and CE are gone too; the only monospaced text left is peptide sequences.Q2 (larger and more diverse datasets). Added nine Vilya Riptides targets - the entries our model can represent (linear unmodified peptides of 3-25 standard residues, standard-residue pocket, absent from the PepFlow training split). Deposited June 2023 to January 2024, after the PepFlow dataset, they act as a temporal hold-out, and all nine were fine-tuned under one shared configuration with no per-target tuning. Six of nine improve on both clash-free rate and the Vinardo score of the clash-free samples, e.g. 7YC2 10.6 -> 68.5% and 8GW4 16.5 -> 56.9% (Sec. 5.3, Table 1). The other three fail outright: no pretrained sample clears the clash-penalty threshold there, so the advantage only ranks degrees of failure; we trace it to how enclosed the initialization is on those receptors (App. E.5).
Q3 (comparison with PepFlow's pretrained weights). Table 13 now includes the released checkpoint alongside our retrained PepFlow and our own pretrained model, under the clash-free definition used elsewhere, so it reads directly against Table 1. It is strongly target-dependent: 78.1% clash-free on 1SSH, 21.9% on 5UZW, 6.3% on 2R02, 0.0% on 6V63. The fine-tuned checkpoints exceed it on all four:
R_95%(same order)Caveats are in App. E.7: the architectures differ, the fine-tuned numbers come from 1,024 noise-injected samples against 64 deterministic ones, and our method cannot be applied to the released weights at all. Not a controlled comparison, then; what it shows is that the optimization passes the published model's reward statistics from a weaker start.
Q4 (more ablations). Partly. Appendix C still ablates the sequence parameterization and the advantage/reward rescaling strategy. We added the reasoning for the factored importance ratio (Sec. 4.2) and for the probability-weighted torsion ratio, now stated to be an approximation rather than an exact density ratio (App. A.2.1). A run that replaces the four per-modality ratios with a single joint-likelihood ratio is in progress; if it finishes during the discussion period we will report it.
Relationships between the experimental sections. Section 5 now reads as one argument: 5.1 setup and definitions, 5.2 the four individually configured targets, 5.3 nine held-out targets under one configuration, 5.4 what that is worth against oversampling the pretrained model, 5.5 what it costs in similarity to the reference. The two analyses carrying the claim are in the main text - the Vinardo score of the clash-free samples, now a column of Table 1 with the distributions in App. Fig. 4, and the best-of-N comparison - while the per-structure case studies stay in the appendix as the qualitative check they are. The abstract was also shortened and no longer claims improved affinity.