acceptodds
Under review as a conference paper at ICLR 2027

Making Every Experiment Count: Analogical Molecular Property Prediction

26 views

Abstract

Evidence for therapeutically relevant molecular properties is diverse, spanning cell-based experiments, animal studies, and human trials. How can machine learning models utilize such indirect evidence—measurements on related molecules, often from different assay contexts—when predicting a property of a molecule that has never been tested? We introduce an analogical-reasoning approach that, rather than predicting solely from structure, aggregates such evidence from the literature and reasons about its relevance. The core component is a property-transfer model, trained on literature evidence, that scores how likely a measurement on one molecule is to carry over to another. A context optimization procedure then selects evidence maximizing diversity, relevance, and transfer likelihood, which frontier LLMs can use to predict by analogy. We assemble a dataset of direct and indirect safety and ADME evidence mined from a 25M-paper corpus and use the direct measurements to build literature-derived benchmarks for six tasks. With Gemini 3.8 Flash, our system reaches a mean macro-F1 of 0.736 on the literature set and 0.815 on the de facto TDC benchmark, exceeding the best ML baseline on each benchmark—including MiniMol trained multi-task on the same indirect evidence—by 11.9 and 11.2 percentage points, respectively. Against naive retrieval with the same backbone—the comparison that isolates our evidence selection—the gains are 2.4 and 4.5 points with Gemini, and 3.4 and 8.8 points averaged across the three backbones, the smaller Gemini margin reflecting its stronger naive baseline. Gains over the ML baselines hold for all three models. Adding indirect evidence improves mean performance in every backbone-benchmark combination, while supplying the same evidence to baselines as auxiliary multi-task targets lowers mean performance on both benchmarks. Ablations support the contributions of learned transfer scoring, informativeness, and diversity to evidence selection.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
LLMs, Analogy, Reasoning, molecule, therapeutics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JrXoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Ampere, a framework for molecular property prediction using direct and indirect experimental evidence from scientific literature. It combines a learned property-transfer model with submodular context optimization to select evidence for LLM-based analogical reasoning. Experiments on six ADME and safety tasks across two benchmarks show improvements over conventional molecular predictors and similarity-based retrieval baselines.

Critical strengths
  1. The paper addresses the practical challenge of using indirect experimental evidence from related molecules. I find this direction interesting, especially for properties where direct measurements are limited.
  2. The evaluation covers six molecular properties, two benchmarks, and three LLM backbones. The improvements over conventional predictors and the component ablations suggest that the proposed evidence selection is useful.
Critical weaknesses
  1. The paper does not experimentally compare with related methods such as MolRAG. Its main matched retrieval baseline uses fingerprint similarity, which is relatively simple. With the strongest backbone, the improvements over this baseline are only 2.4 and 4.5 macro-F1 points on the two benchmarks. While these gains are promising, I find the current comparisons insufficient to establish a convincing advantage over stronger retrieval-based methods.
  2. The literature-derived benchmark and retrieval corpus are constructed using the same automated extraction pipeline, making independent validation of data quality important. More importantly, Appendix E.2 states that the test cohorts were evaluated in earlier experiments. It is unclear whether these evaluations influenced method development. The relatively small TDC test sets and the lack of uncertainty estimates for single-rollout LLM predictions further weaken confidence in the reported gains.
Questions for authors
  1. How does Ampere compare with stronger retrieval-based methods such as MolRAG, or a simpler approach that ranks evidence directly using the learned transfer scores, under matched experimental settings?
  2. Were the previously evaluated test cohorts used to guide method development or hyperparameter choices?
  3. Can the authors provide repeated-run results or confidence intervals to establish the reliability of the reported improvements?
2 comments

We thank the reviewer for the thoughtful review. We've performed new experiments to incorporate your feedback and address some of the concerns you raised:

  1. How does Ampere compare with stronger retrieval-based methods such as MolRAG, or a simpler approach that ranks evidence directly using the learned transfer scores, under matched experimental settings?
    Our naive LLM baseline follows MolRAG's core RAG workflow: retrieving structurally similar labeled molecules and prompting an LLM. However, it uses a different prompt and larger retrieval budget. For a more direct comparison, we reproduced MolRAG's best-reported 4-shot Sim-CoT configuration. As no official code is available, we reconstructed it from the paper, preserving Morgan/Dice retrieval, four distinct references, and Sim-CoT reasoning. All other settings, including LLM backbones, are matched:

    MethodTDCLiterature derived benchmark
    MolRAG / GPT-6 Luna73.5659.82
    MolRAG / DeepSeek70.5260.9
    MolRAG / Gemini 3.877.870.38
    Ours / GPT-6 Luna78.7168.97
    Ours / DeepSeek79.9772.09
    Ours / Gemini 3.881.5473.6

    On all three LLMs, Ampere performs significantly better than MolRAG on both benchmarks.
    Beyond the MolRAG baseline, we also have ablations in Table 1. The transfer tool is the largest individual contributor to our system, resulting in a 1.7 F1 reduction in performance across literature sourced datasets when removed.

  2. Were the previously evaluated test cohorts used to guide method development or hyperparameter choices?
    We apologize for the lack of clarity and poor writing in this appendix section. In all cases, we optimized hyperparameters exclusively on development data. Across all data settings, test sets were used to sanity check generalization before submission fewer than 5 times, and no model optimization or development took place on it. We will revise the description in the Appendix section accordingly to clarify.

  3. Can the authors provide repeated-run results or confidence intervals to establish the reliability of the reported improvements? We thank the reviewer for this valuable suggestion. To assess the reproducibility and reliability of our reported improvements, we conducted three independent runs for all stochastic ML baselines and DeepSeek-based methods, reporting the mean ± standard deviation (SD) across runs. In general, the SD of performance of our models across both literature and TDC evaluation sets was less than 1 point, indicating our models reliably outperform the baselines we have considered.

    The repeated-run results on both benchmarks are summarized below:

    MethodLiterature-derived BenchmarkTDC
    Morgan KNN57.68 ± —69.17 ± —
    MiniMol KNN59.66 ± —69.08 ± —
    RASAR57.17 ± 0.6566.87 ± 0.62
    MiniMol60.57 ± 0.7370.98 ± 1.86
    DeepSeek V4 Flash / Zero-shot60.41 ± 0.1964.49 ± 0.43
    DeepSeek V4 Flash / Naive Direct63.70 ± 0.8767.70 ± 1.00
    DeepSeek V4 Flash / Naive Direct+Indirect67.53 ± 0.3470.69 ± 0.99
    DeepSeek V4 Flash / MolRAG60.97 ± 0.2871.91 ± 0.95
    DeepSeek V4 Flash / Ours Direct64.50 ± 0.5677.98 ± 0.56
    DeepSeek V4 Flash / Ours Direct+Indirect71.46 ± 0.5280.15 ± 0.79

Dear Reviewer,

Thank you again for your thoughtful feedback. We’ve included new experimental results in our rebuttal to address the questions you raised. When you have a chance, we’d welcome your thoughts and would be happy to discuss any remaining concerns.

Reviewer_iyasOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Ampere, a framework for predicting molecular ADME and properties using experimental evidence from related molecules. It extracts literature records with assay conditions, trains a property transfer model to predict whether a query molecule would yield a similar experimental outcome, and selects evidence using transfer scores, structural similarity, endpoint informativeness, and diversity.The core prediction is based on large language model analysis on analogy. Evaluation show their method performs well on those data.

Critical strengths
  1. The paper provides a useful formulation of heterogeneous evidence integration. And the prediction results show that those evidence could be used to predict the properties of untested molecules.
  2. The ablation study clearly showed that the optimized evidence selection improves the retrieval performance over the naive retrieval, and the components ablations support the contribution of the learned transfer scores.
Critical weaknesses

The experiment settings were solid, but the paper lacks some baselines which are relevant. Morgan and MiniMol KNN, MiniMol with and without auxiliary measurements, RASAR, RASAR plus indirect evidence, and naive retrieval with matched LLM backbones. My concern is therefore not an absence of molecular-analogy baselines. Rather, the transfer scorer is mainly tested against structural similarity, leaving its advantage over established molecular-pair learning unclear. DeepDelta directly learns property differences from molecule pairs on ADMET tasks, while matched molecular pair analysis has been used to select suitable analogs for read-across. These are relevant to the specific question of whether a measurement should transfer, although they require adaptation to this paper’s heterogeneous endpoints and conditions. MolRAG is acknowledged in related work but has no separately reported reproduction.

Questions for authors
  1. What advantages does the transfer scorer offer over a molecular pair predictor trained on the same data, especially when structural similarity is controlled for? It would be helpful to report both direct transfer-ranking performance and downstream results when only the scorer is replaced.
  2. How much of the MolRAG pipeline does the naive LLM baseline actually cover, and what differs between them? Since the RASAR adaptation may also affect the comparison, a matched reproduction or a clear explanation of the differences would help establish whether the comparison is fair.
  3. Do the transfer scorer and the final reasoning LLM agree on which structural changes allow or prevent knowledge transfer? A few representative examples of successful and failed transfers, including cases affected by experimental conditions, would help support the proposed mechanism. A comprehensive fragment library is not necessary at this stage.
  4. Why does adding indirect evidence lower Gemini's F1 score on TDC carcinogenicity from 0.8303 to 0.7737, even though the average performance across six tasks improves? It would be useful to determine whether this drop comes from inappropriate analog selection, differences in experimental conditions, or how the LLM integrates the retrieved evidence.
2 comments

We thank the reviewer for the thoughtful and constructive feedback. We respond to the questions below:

  1. Q1
    We agree that DeepDelta and other molecular-pair predictors are viable alternatives. Unlike DeepDelta's graph representations, our scorer uses pretrained Intern-S1 representations, which flexibly capture experimental context, such as species and measurement conditions, without architectural adaptation. Our contribution is not the scoring architecture, but a framework integrating heterogeneous direct and indirect evidence. Incorporating graph-based predictors is promising future work. We do not claim Intern-S1-mini is superior to DeepDelta; rather, our ablation shows learned transferability adds value beyond structural similarity: removing the scorer reduces mean macro-F1 from 0.721 to 0.704 (Table 1).

  2. Q2
    Our naive LLM baseline follows MolRAG's core RAG workflow: retrieving structurally similar labeled molecules and prompting an LLM. However, it uses a different prompt and larger retrieval budget. For a more direct comparison, we reproduced MolRAG's best-reported 4-shot Sim-CoT configuration. As no official code is available, we reconstructed it from the paper, preserving Morgan/Dice retrieval, four distinct references, and Sim-CoT reasoning. All other settings, including LLM backbones, are matched:

    MethodTDCLiterature derived benchmark
    MolRAG / DeepSeek70.5260.9
    MolRAG / GPT-6 Luna73.5659.82
    MolRAG / Gemini 3.877.870.38
    Ours / DeepSeek79.9772.09
    Ours / GPT-6 Luna78.7168.97
    Ours / Gemini 3.881.5473.6

    For RASAR, our direct adaptation uses a random forest with labels and similarities from the three nearest molecules (matching KNN), while Simple RASAR uses the closest positive and negative examples. The indirect variant adds values and similarities from the nearest eligible molecule for each evidence type, following RASAR's data-fusion principle. Unlike the original RASAR, it excludes known properties of the query molecule, which are unavailable in our setting.

  3. Q3
    In the prompt, the language model was encouraged to use the transfer tool’s judgement. In the excerpts below, we show what this looks like, but in general the reasoning traces are very long and technical. In any future version of the paper, we will try to compress and summarize some of these within the appendix to highlight a few representative cases.

    The following cases illustrate agreement and disagreement. These examples predict absolute oral bioavailability, with a positive label defined as F ≥20%. Quoted passages are verbatim excerpts from the saved model answers.

    1. Agree with transfer scorer

    …NXY-059, a small disulfonated molecule, had poor brain uptake even in ischemia; however it is not structurally similar to the large albumin-binding bis-azo dye, and transfer likelihood is low, so it carries little weight for the query….

    2. Disagree with transfer scorer

    …Coenzyme Q10 (transfer score=0.9810) has low oral bioavailability attributed to its very high molecular weight, extreme lipophilicity, and P-gp efflux; the query molecule is much smaller and contains a naphthoquinone rather than the large decaprenyl benzoquinone, making this negative evidence only weakly transferable...

  4. Q4
    In general, individual TDC datasets are small and have high variance, which is, in part, why we collected larger evaluation sets via literature. This particular swing on carcinogenicity represents only two new errors relative to TDC labels. We examined the paired original traces and found the model’s predictions fairly convincing and so conducted a literature review of the specific data points in TDC. Unfortunately we found significant evidence that these labels were wrong across several papers, consistent with prior identifying high error rates in some TDC datasets (https://arxiv.org/abs/2605.07022). While individual performance variances on TDC datasets can be high, evaluating aggregates across tasks can be more stable. In such settings, we improve reliably, and in general, across larger and more stable literature derived evaluations we reliably improve.

Thank you for the response. I remain supportive of the paper and maintain my rating of Weak Accept.

Reviewer_DLANOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose a framework for molecular propety prediction that uses experiemntal evidence from related molecules to help llms with the prediction. The authors mine literature for relevant experimental measures and they fine tune an llm to predict how well a molecule's measurement would transfer to another. They combine this score with other standard scores to give as context to a pretrained llm to predict properties.

Critical strengths

I think incorporating indirect experimental evidence into molecular property prediction is an interesting direction. I think this is something that should be done, but perhaps in a different manner than the proposed method.

Critical weaknesses

The central contribution of the paper is the learned transferability model but the experiments fail to demonstrate why it is actually needed. The gains rely on incomplete baselines and the improvements over naive rag seem to be much smaller. My concern is that the real driver of performance is simply giving the llm more access to relevant experimental evidence, which is useful but hardly surprising. I dont see why we need to train a whole separate model for this when there are potentially much simpler ways to incorporate the same information into existing LLM workflows (that are more scalable as well as more experimental evidence comes in). I think we should be careful about jumping to train new models without first asking whether we actually need them. In its current form I dont think the paper warrants acceptance at ICLR and see it more suitable for a workshop.

Questions for authors

Unclear why the proposed framework is necessary and limited novelty overall.

Correct me if I am wrong, but the "analogy framing" seems a bit overdressed-- i see this method as essentially RAG for molecular property prediction. The main contribution appears to be fine tuning an llm to predict "transferability," or essentially pairwise experimental outcome similarity between molecules, and then using these scores alongside standard molecular (tanimoto) similarity and diversity measures to select context for another LLM to use for property prediction. While the application to this indirect experimental evidence is interesting, it is unclear what is novel about this framework beyond existing retrieval and context selection approaches. I do think the approach may work, but it is not clear why this additional complexity is necessary. For example, why train a separate LLM for transferability prediction rather than allowing the downstream reasoning model to assess and select relevant experimental evidence from a retrieved candidate pool? Comparisons against simpler LLM evidence selection/summarization approaches are missing, making it difficult to understand what the proposed framework actually contributes over a more straightforward pipeline.

I think more broadly, as a field, we should be careful about immediately introducing new trained model artifacts without first understanding what is actually necessary. In this case, the main benefit seems to come from access to additional experimental evidence, and I would have liked to see more careful ablations of how that information could be incorporated into existing LLM workflows in a simpler and more scalable way, rather than immediately training a separate model for it. Overall, i do not think the methodological contributions or insights understood from this work warrant an acceptance, I feel the the paper would perhaps be more fit for a workshop in its current form.

Benchmarks seem misleading and indicate only marginal improvements

The authors emphasize 11-12 percentage point improvements over ml baselines, but these are misleading as the authors do not compare against the actual sota on these TDC benchmarks. More importantly, when compared against naive rag using the same llm, the improvements are noticably much smaller and the transfer model only has marginal contribution in their reported ablation (goes to my point above).

4 comments

We thank the reviewer for their feedback and appreciate the opportunity to clarify our paper.

Motivation and Novelty:

Our contributions go beyond just training a property transfer tool and, broadly, include: integrating indirect, heterogeneous experimental evidence through comparative reasoning. Such reasoning, generally, has been referred to as analogies (https://plato.stanford.edu/entries/reasoning-analogy/), and while we are not committed to this framing, the reasoning our models do and the evidence they use represents a significant departure from previous work.

Existing methods either learn structure-to-property mappings or use RAG. RASAR incorporates indirect evidence through engineered features, while MolRAG and MolecularGPT retrieve structurally similar training molecules. MolE-RAG retrieves literature but only for the target molecule and property, limiting its use for untested molecules. In contrast, our framework leverages indirect evidence from experiments on other molecules and properties, recognizing that direct measurements capture only a fraction of the evidence relevant to property prediction. As the reviewer notes, this extra data is a fundamental advantage, but deriving its benefit is non-trivial. Without our context optimization, GPT-6 Luna with MolRAG fails to outperform MiniMol on TDC; with our approach, it exceeds MiniMol by 10.8 F1 (Figure 3).

Baselines and performance relative to SOTA:

Our choice of baselines was informed by current leaderboards for molecular property prediction ( https://tdcommons.ai/benchmark/admet_group/overview/ ). However, given limited resources, we focused on systems that were best in aggregate across tasks (Minimol is the top performing system across the vast majority of benchmark tasks, and hence our main point of comparison). For this rebuttal, in an effort to evaluate a broader range of systems from more recent papers and older but well-known baselines, we also evaluated methods such as ChemBERTa, CheMeleon, and others (see table below). However, these more modern methods do not reliably outperform Minimol, and the general trends in our experiments hold despite this larger evaluation: on TDC, Ampere outperforms existing approaches by more than 10 F1.

MethodLiterature bench: directLiterature bench: with indirectTDC: directTDC: with indirect
MiniMol (2024 [1])60.1159.3168.7567.23
CheMeleon (2026 [2]])59.7458.6670.1870.03
Uni-Mol 2 84M (2024 [3])57.9356.0968.2367.29
GROVER (2020 [4])55.1253.6568.0368.34
ChemBERTa 2 (2022 [5])58.6658.2372.0471.43
MoLFormer (2022 [6])57.6056.6067.8266.46
SMI-TED (2025 [7])59.2458.0371.5169.87
NV-KERMT v2 (2026 [8])57.5155.9169.4868.24
MolRAG / GPT-6 Luna (2025 [9])59.82—73.56—
MolRAG / DeepSeek V4 Flash60.90—70.52—
MolRAG / Gemini 3.8 Flash70.38—77.80—
Ours / GPT-6 Luna62.7368.9777.8178.71
Ours / DeepSeek V4 Flash65.3672.0977.7679.97
Ours / Gemini 3.8 Flash71.2173.6080.7981.54

Need for Training Models:
Whether the training complexity is justified is a fair concern. Structural-similarity retrieval, however, is the straightforward way to incorporate the indirect evidence, and we already evaluate it as a baseline. Our improvements thus demonstrate the value of learned evidence selection beyond access to the evidence itself (Figure 3). To further address this concern, we evaluate two training-free LLM baselines using a pool of 100 direct and 1,000 indirect records from the most structurally similar molecules. In Select, an LLM chooses the N most relevant records, matching our selection budget, and in Score, an LLM scores records for our context optimization procedure, replacing the learned property transfer model, and hyperparameters are re-optimized. Both use DeepSeek-V4-0731. On TDC, these alternatives underperform our full system by 3.64 F1, supporting the value of learned evidence selection.

MethodLiterature-derived BenchmarkTDC
LLM Select - Direct + Indirect67.79 ± 0.3576.19 ± 1.00
LLM Score + Optimizer - Direct + Indirect67.82 ± 0.7575.59 ± 0.33
Ours - Direct+Indirect71.46 ± 0.5280.15 ± 0.79

[1] https://arxiv.org/abs/2404.14986
[2] https://pubs.acs.org/jcisd8/article-abstract/66/17/10692/5250516/Deep-Learning-Foundation-Models-for-Low-Data
[3] https://arxiv.org/abs/2406.14969
[4] https://arxiv.org/abs/2007.02835
[5] https://arxiv.org/abs/2209.01712
[6] https://arxiv.org/abs/2106.09553
[7] https://www.nature.com/articles/s42004-025-01585-0
[8] https://huggingface.co/nvidia/NV-KERMT-70M-v2
[9] https://arxiv.org/abs/2606.05693v2

Thank you for the additional experiments.

  1. I still have some concerns about the benchmark comparisons. Some of the reported results do not line up with published TDC leaderboard performance for some models (bioavailibilty score on TDC for example). This is a small point though and instead I am more concerned about the exclusive use of macro F1 as the metric of comparison. It is quite difficult to assess the claimed improvements against existing methods, given that AUROC/AUPRC are the standard evaluation metrics for these benchmarks (TDC). While I understand the justification in Section 3.2, AUROC does not require calibrated probabilities, only meaningful ranking scores, and there are established ways to obtain these.

  2. Upon closer look I am also realizing that the evaluation is restricted to six binary classification tasks, leaving continuous property prediction largely unexplored. Taking all of this together, I find the empirical evaluation rather limited and insufficient to fully characterize the method's performance relative to existing approaches across tasks and metrics. I would need much stronger empirical evidence to be convinced that the work warrants acceptance.

  3. I appreciate the additional no training llm baselines, which address some of my original concerns. However, these only test two specific ways of using llms to select evidence, rather than establishing that a separately trained model is necessary. In the original Table 1 ablation, removing the learned transfer score only reduced performance marginally suggesting that most of the improvement comes from simpler selection methods. I remain unconvinced that more scalable ways of letting llms reason over experimental evidence have been sufficiently explored.

We thank the reviewer for their response and appreciate the opportunity to clarify these points. We’d like to address each concern below:

  1. We reproduced baseline results on both AUC and F1 (Appendix Table 6), finding AUC agreement within 1–2 points of TDC leaderboard results except for oral bioavailability. Differences may reflect TDC rules permitting training on validation data, which we reserve for hyperparameter selection. For oral bioavailability, a reported issue suggests leaderboard entries may duplicate results from another task. While unconfirmed, our results agree with MiniMol’s official repository.

    We agree that AUROC requires meaningful ranking scores, not calibrated probabilities, but deriving these from LLMs is non-trivial. Each approach has limitations:

    • Answer-token probabilities: Conditioned on reasoning that has already reached a conclusion, this approach produces severe saturation. Empirically, probabilities are almost exclusively above 0.9998 or below 0.0002, providing little discrimination within predicted classes.
    • Self-consistency: Multiple rollouts are computationally expensive and measure generation consistency rather than class affinity (e.g., consistently predicting a slight negative yields 0.0).
    • Verbalized probabilities: Directly elicited confidence is known to be poorly calibrated (https://arxiv.org/abs/2405.16908).
    • Pairwise tournaments: Quadratic scaling makes ranking computationally prohibitive.

    Given these factors, we have ensured a fair comparison by tuning decision thresholds on the validation set to maximize macro F1 for every baseline reported in the paper and rebuttal.

    New AUC Evaluations with LLMs:

    However, in an effort to provide comparable AUC metrics, we’ve re-run our main experiment with the Verbalized Probability approach using DeepSeek V4 Flash on all six TDC tasks, and our reported improvements generalize under this metric as well.

    MethodTDC (AUROC)
    Ours - Direct + Indirect (Verbalized Probability)0.8662
    MiniMol fine-tuned0.7845
    MiniMol fine-tuned + indirect0.7851

    Despite this being an evaluation that disadvantages language model based approaches, Ampere substantially outperforms the baseline for the work. We can include this in any future version of the paper.

  2. The scope of our evaluation follows precedent from prior LLM-based molecular prediction, such as MolRAG, which similarly evaluates on seven classification benchmarks due to the large inference cost of LLM reasoning. While extending to continuous regression is important for future work, binary endpoints (e.g., toxicity, permeability, bioavailability) constitute a foundational part of ADMET screening and early-stage drug discovery. Demonstrating strong gains across six diverse tasks provides validation of our method within this core regime.

  3. In Table 1, ablating the learned transfer score yields the single largest performance drop (-1.7 Macro F1) of any individual component, roughly double the impact of the other terms (-0.7 to -1.0). That access to indirect evidence and the remaining selection terms collectively drive the bulk of the overall gain is expected, and does not diminish the distinct contribution of the learned transfer score. In general, this transfer function is not our only source of novelty: leveraging indirect assays and reasoning, alongside the transfer function, forms a broader system that works well in coordination.

    Given the broader system we have proposed, alongside the training-free baselines in our original paper, we added two new LLM-based selection baselines directly in response to the initial feedback. Proving that there does not exist a training-free alternative that outperforms our system is impossible, and it is unclear to us where the bar is for you. We respectfully ask: is there a specific training-free formulation that would address your concern?

I appreciate the new AUROC results although comparisons against the expanded ML baselines under this metric remain limited. I also agree that it is unreasonable to expect you to rule out every possible training free alternative, that was not my intention to ask. Rather, I wanted to emphasize that I still have some reservations about the necessity of a separately trained model, given its relatively modest contribution in the original ablation.

while I remain somewhat unconvinced by the methodological novelty and the limited evaluation beyond binary classification, the authors have addressed several of my concerns and I will raise my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.