acceptodds
Under review as a conference paper at ICLR 2027

Principia: Test-Time Hypotheses to Principle Construction for Scientific Discovery

Abstract

Scientific discovery requires identifying the scientific principles underlying observed phenomena to guide further investigation. As these empirical principles are often unknown in complex discovery processes, they are constructed via a hypotheses-to-principle pipeline to obtain recurring causalities from progressively accumulated observations during test-time discovery. However, constructing this knowledge requires substantial domain expertise, time, and effort, making this translation stage a bottleneck for scientific discovery. To address this limitation, we propose Principia, an LLM-based framework that constructs and continually refines scientific principles at test-time. Drawing on how human researchers develop principles, Principia forms comparative causal hypotheses from related inputs and their observed outcomes, then synthesizes recurring relationships across these hypotheses into inductive principles. By applying the constructed hypotheses and inductive principles to the task, it obtains new observations, revises the used hypotheses based on these new observations, and reconstructs principles from the updated hypotheses, allowing scientific knowledge to continuously evolve throughout test-time discovery. Across benchmarks spanning molecular optimization, protein engineering, gene perturbation, and molecular property prediction, Principia outperforms baselines relying on internal knowledge, external sources, or experience summaries, while generating principles that are plausible and consistent to scientific literature, demonstrating its effectiveness in constructing and refining task-specific hypotheses and principles at test-time for scientific discovery.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Test-time Hypotheses to Principle Construction, Large Language Models, Scientific Discovery

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VJLzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes PRINCIPIA, an LLM-based framework for constructing and refining scientific knowledge during test-time discovery. It generates causal hypotheses by comparing related inputs and their observed outcomes, then generalizes recurring patterns into principles through clustering. These hypotheses and principles guide subsequent optimization or prediction, and are updated based on new observations. Experiments on molecular optimization, protein engineering, gene perturbation, and molecular property prediction show improvements over several LLM-based baselines.

Critical strengths
  1. The framework provides a structured way to organize and update knowledge from experimental observations, without requiring additional model training.
  2. The experiments cover a diverse range of scientific tasks, with promising improvements over relevant baselines such as MolLEO, ExLLM, and DeepEvolve.
  3. The paper includes useful ablation studies on comparative hypotheses, principle construction, revision, and retrieval, along with detailed implementation descriptions and prompts.
Critical weaknesses
  1. The generated hypotheses and principles are not systematically validated for scientific correctness. The qualitative examples are encouraging, but they provide limited evidence that the inferred relationships are genuinely causal or scientifically reliable.
  2. Although the paper includes an ablation without principle construction, it does not compare against directly retrieving historical input-output comparisons without LLM-generated hypotheses in the optimization tasks. This would help clarify whether the hypothesis generation step provides additional benefits beyond access to relevant past observations.
  3. The methodological contribution appears somewhat incremental. Prior work such as LLM4SD and DESRO has explored deriving scientific rules from molecular observations, while PiFlow and PiEvo have investigated principle-guided scientific discovery and iterative refinement. PRINCIPIA organizes these ideas into a structured test-time memory framework, but its additional technical contributions appear relatively limited.
Questions for authors
  1. Appendix C.1 uses PiFlow's 24-iteration protocol, while other methods are evaluated with fixed oracle budgets. Could the authors clarify how many oracle evaluations PiFlow actually performs and how Avg@100 is computed in this setting?
  2. The efficiency analysis focuses on oracle evaluations, but PRINCIPIA introduces additional LLM calls for hypothesis generation, principle construction, retrieval, and revision. Could the authors report the total token usage or runtime compared with the baselines, to clarify whether the reduced oracle budget translates into lower overall computational cost?
6 comments
AuthorsOpenReview ↗

Initial Response by Authors (1/3)

Thank you for your constructive comments. We appreciate your recognition of the following strengths:

  • Structured knowledge construction and updating without additional model training.
  • Promising performance improvements across diverse scientific tasks.
  • Useful ablation studies and detailed implementation descriptions and prompts.

We address your concerns below.



Comment 1: The generated hypotheses and principles are not systematically validated for scientific correctness. The qualitative examples are encouraging, but they provide limited evidence that the inferred relationships are genuinely causal or scientifically reliable.

Response 1: Unfortunately, scientific problems are still open ended and causal relationships underlying scientific problems may not be fully established, limiting to systematically validating scientific correctness of generated hypotheses and principles. Literature search can identify supporting or conflicting evidence where relevant studies exist. However, due to a lack of literature, their validation could not be rigorous for specific inputs or settings. Instead, we adopt qualitative analysis to see the plausibility of generated hypotheses and principles and show the procedure of PRINCIPIA including the revision.

To mitigate the limitations of literature-based validation, we conduct additional experiments to assess the scientific correctness of the generated principles. Specifically, we evaluate whether principles constructed in one discovery run improve predictions of fitness-change directions on held-out comparisons from separate runs. As shown in Table C.1, providing these principles consistently improves prediction, providing quantitative evidence supporting the correctness of the input-outcome relationships captured by the generated principles.

MethodJNK3Median1Median2
w/o principles69.6781.6774.33
w/ principles74.6784.3384.00

Table C.1: Fitness direction prediction accuracy (%) with and without PRINCIPIA principles

On the other hand, in real-world drug discovery, limited observations can leave hypotheses and principles incomplete, even when developed by experts. To address this practical constraint, we propose a revision algorithm that reassesses previously used hypotheses as new observations become available. Therefore, our contribution within Applications to Sciences focuses on an algorithmic framework for continually constructing and updating scientific knowledge under real-world scenarios where observations do not fully cover the input space and empirical hypotheses and principles are incomplete.


Comment 2: Although the paper includes an ablation without principle construction, it does not compare against directly retrieving historical input-output comparisons without LLM-generated hypotheses in the optimization tasks. This would help clarify whether the hypothesis generation step provides additional benefits beyond access to relevant past observations.

Response 2: We appreciate your insightful suggestion. We further compare our comparative causal hypotheses with historical input-output pairs. As shown in Table C.2, using comparative causal hypotheses outperforms using historical comparisons, suggesting that explaining which input characteristics may contribute to observed outcomes provides useful guidance beyond access to past observations alone.

MethodJNK3GSK3BValsartan-SMARTSAvg.
Ours w/ Input-output pairs0.3370.3800.0950.271
Ours w/ Comparative casual hypotheses0.3500.4520.1060.303

Table C.2: Top-10 AUC comparison between using simple input-outcome pairs and causal hypotheses. We report the mean over three independent trials.


Due to the character limit, the response continues below.

AuthorsOpenReview ↗

Initial Response by Authors (2/3)

Comment 3: The methodological contribution appears somewhat incremental. Prior work such as LLM4SD and DESRO has explored deriving scientific rules from molecular observations, while PiFlow and PiEvo have investigated principle-guided scientific discovery and iterative refinement. PRINCIPIA organizes these ideas into a structured test-time memory framework, but its additional technical contributions appear relatively limited.

Response 3: Our main contribution is test-time hypotheses-to-principle construction, which couples principle formation with the continual revision of its underlying explanatory hypotheses. We highlight two distinctions from prior work:

  1. Revision under incomplete evidence. LLM4SD derives rules for predictive features, while DESRO derives reasoning supervision for training. During discovery, limited observations can yield explanations that overlook relevant factors or overgeneralize beyond their supporting evidence. PRINCIPIA retains these explanations as revisable knowledge units. When new observations become available, it revisits the hypotheses used in subsequent decisions, updates their explanations while retaining the original comparative evidence, and reconstructs principles from the updated hypotheses.

  2. Explanatory hypotheses as the basis of principles. In PiFlow and PiEvo, hypotheses primarily refer to experimental candidates proposed under guiding principles. PiEvo also evolves these principles using experimental feedback. PRINCIPIA maintains comparative explanatory hypotheses about the relationships between input differences and observed outcomes. These explanations remain individually usable and revisable after their recurring relationships are generalized into principles. Revising them therefore updates the explanatory basis from which principles are reconstructed.

Therefore, our contribution is this coupled process of comparative explanation, generalization, and evidence-driven revision throughout scientific discovery.


Comment 4: Appendix C.1 uses PiFlow's 24-iteration protocol, while other methods are evaluated with fixed oracle budgets. Could the authors clarify how many oracle evaluations PiFlow actually performs and how Avg@100 is computed in this setting?

Response 4: We would like to clarify that PiFlow’s generation procedure is different from MolLEO-based models: it generates and evaluates candidates sequentially, while MolLEO-based models leverage population-based genetic algorithms. We adopted the original PiFlow setting of 24 iterations for molecular optimization and protein engineering. The average numbers of unique oracle evaluations were 15.73 and 20.67, respectively. Therefore, Avg@100 and Avg@50 average all available candidates. For gene perturbation, Hit Ratio uses the same number of distinct genes across methods.

We further examined PiFlow under the same maximum oracle-evaluation budgets as PRINCIPIA, conducting three runs per task. Because PiFlow frequently generated duplicate sequences in the GB1 task of protein engineering, we supplemented its prompts with previously proposed sequences and instructions to avoid proposing them again. This provides a potentially unfair advantage to PiFlow, since PRINCIPIA was not given the same explicit exclusion list. Even with this additional support, as shown in Table C.3 and C.4, PRINCIPIA achieves higher performance on average, indicating that PRINCIPIA still outperforms PiFlow under the same oracle-evaluation budgets. We have updated Tables 1 and 2 in the revised paper to report these results under matched maximum oracle-evaluation budgets.

MethodJNK3GSK3BMedian1Median2Valsartan SmartsAvg.
Avg@100AUC@10Avg@100AUC@10Avg@100AUC@10Avg@100AUC@10Avg@100AUC@10AUC@10
PiFlow0.1580.2540.1840.2750.2510.2830.1570.1640.0000.0000.195
Ours0.4480.3500.6170.4520.2980.2840.2200.2030.1170.1060.279

Table C.3: Molecule optimization under matched maximum oracle budgets for three runs.

MethodSyn-3bfoAAVGB1Scharenberg22IFNGAvg.
Avg@50AUC@10Avg@50AUC@10Avg@50AUC@10Hit RatioAUC@10Hit RatioAUC@10AUC@10
PiFlow-29.8130.1640.1400.1010.4680.4620.2890.5570.1110.6820.393
Ours0.0840.7420.6810.5270.3980.3830.3080.6300.1140.7420.605

Table C.4: Protein engineering results under matched maximum oracle budgets. Gene perturbation results are adapted from the original evaluation, which already used matched maximum oracle budgets.


Due to the character limit, the response continues below.

AuthorsOpenReview ↗

Initial Response by Authors (3/3)

Comment 5: The efficiency analysis focuses on oracle evaluations, but PRINCIPIA introduces additional LLM calls for hypothesis generation, principle construction, retrieval, and revision. Could the authors report the total token usage or runtime compared with the baselines, to clarify whether the reduced oracle budget translates into lower overall computational cost?

Response 5: Our primary focus on oracle efficiency is mainly due to real-world scientific discovery settings where wet-lab experiments are substantially more expensive and time-consuming than LLM inference. In other words, while PRINCIPIA leverages additional LLM computation in constructing and updating scientific knowledge, it enables to reduce the number of experimental evaluations required for discovery.

To quantify this trade-off, we report total LLM calls, token usage, API cost, and LLM wall-clock time at matched optimization performance on Syn-3bfo, using the performance targets in Fig. 5. We additionally estimate wet-lab overhead from the oracle counts, assuming $50 and two days per evaluation with sequential execution [1,2,3]. As shown in Table C.5, although PRINCIPIA incurs higher LLM costs, this additional expense is small compared with the estimated savings from fewer experimental evaluations. Consequently, investing in knowledge construction reduces estimated total cost and time, supporting the practical efficiency of PRINCIPIA in settings where experimental evaluations dominate the discovery budget.

MethodToken Usage (million)LLM CallsLLM-API Cost ($)Wet-lab + LLM-API Cost ($)LLM Wall-Clock (day)Wet-lab Runtime + LLM Wall-Clock (day)
MolLEO4.08487.6350,007.630.0332,000.033
Ours4.66573.337,003.330.010280.010
ExLLM4.08737.3350,007.330.0352,000.035
Ours27.92,80416.4822,316.480.040892.040
WebSearch5.517609.1450,009.140.0572,000.057
Ours6.28254.298,204.290.012328.012
DeepEvolve9.15,04012.7850,012.780.1542,000.154
Ours4.66573.336,903.330.010276.010

Table C.5: Cost comparison at matched performance.

[1] Kosuri, Sriram, and George M. Church. "Large-scale de novo DNA synthesis: technologies and applications." Nature methods 11.5 (2014): 499-507.

[2] Wittmann, B. J., et al. "evSeq: Cost-effective amplicon sequencing of every variant in a protein library. ACS Synth Biol 11: 1313–1324." 2022,

[3] Fowler, Douglas M., Jason J. Stephany, and Stanley Fields. "Measuring the activity of protein variants on a large scale using deep mutational scanning." Nature protocols 9.9 (2014): 2267-2284.



We thank the reviewer for your time and feedback and hope that our responses have addressed your concerns. If you have any further questions, we are happy to address them. If your concerns are appropriately addressed, we hope you kindly consider updating the rating accordingly.

Reviewer_LtxCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PRINCIPIA is an LLM framework that builds and revises textual knowledge during scientific optimization and prediction. It generates hypotheses from comparisons between related inputs, groups these hypotheses into cluster-level principles, and revises them using subsequent observations. The framework is evaluated on molecular optimization, protein engineering, gene perturbation, and eight molecular property prediction tasks. The central contribution is an explicit, evolving hierarchy of comparative hypotheses and reusable principles for guiding future decisions.

Critical strengths
  1. The framework addresses a meaningful problem with a coherent design. Connecting individual comparisons to broader principles, then revising those comparisons using new evidence, is a sensible approach to organizing scientific experience. The distinction between local hypotheses and more general guidance is useful and clearly motivated.

  2. Several empirical improvements are substantial. In Table 3, average macro-F1 increases from 62.30 for the strongest few-shot configuration to 69.03. In Table 2, GB1 AUC@10 improves from 0.277 for ExLLM to 0.383. Table 1 also reports the highest average molecular optimization scores. These results provide credible evidence that the complete framework offers useful guidance.

  3. The evaluation includes meaningful component analyses. Figures 3–6 examine comparative evidence, principle construction, hypothesis revision, and retrieval. Appendix E extends the experiments to additional backbones. These analyses strengthen the case that the results are not confined to one model or one design choice.

Critical weaknesses
  1. Causal claims exceed the evidence. Better optimization and prediction do not establish that the generated explanations are causally correct. Tables 5 and 8 provide plausible examples, but a systematic evaluation of principle validity is missing.
  2. The source of the gains remains unclear. PRINCIPIA uses richer memory and more elaborate retrieval than the summary and few-shot baselines. A control with the same evidence and retrieval, but without causal hypotheses, is needed to isolate the contribution of principle construction.
  3. Small improvements lack sufficient statistical support. Tables 1–3 report three-run averages without uncertainty estimates, and prediction uses at most 50 test molecules per class per run. Small gains, such as Syn-3bfo AUC@10 increasing from 0.738 to 0.741, are therefore difficult to assess.
  4. Overall cost-effectiveness is not established. The reported 55% reduction in oracle calls comes from one Syn-3bfo experiment. Without token usage and runtime measurements, it is unclear whether these savings outweigh the additional LLM computation.
Questions for authors
  1. What are PiFlow’s actual unique oracle-evaluation counts and initial populations in Tables 1–2? Does its performance gap persist under the same evaluation budget and duplicate-handling policy?
  2. How much improvement remains when the same evidence, clustering, and retrieval are retained, but causal hypotheses are replaced with factual descriptions of input differences and observed outcomes?
  3. Can the generated principles make accurate, falsifiable predictions on held-out comparisons—for example, predicting the direction of fitness changes? Does revision improve this accuracy?
  4. Can you provide paired per-run results or confidence intervals for the main comparisons, particularly molecular prediction and the smaller optimization gains?
  5. What are the total LLM calls, token usage, and runtime relative to ExLLM at comparable optimization quality?
4 comments
AuthorsOpenReview ↗

Initial Response by Authors (1/3)

Thank you for your constructive comments. We appreciate your recognition of the following strengths:

  • A coherent framework connecting comparative hypotheses, broader principles, and revision using new observations.
  • Substantial performance improvements on several optimization and molecular property prediction tasks.
  • Meaningful component analyses and evaluation across additional backbone models.

We address your concerns below.



Comment 1: Causal claims exceed the evidence. Better optimization and prediction do not establish that the generated explanations are causally correct. Tables 5 and 8 provide plausible examples, but a systematic evaluation of principle validity is missing.

Response 1: We agree that a systematic evaluation of hypothesis and principle validity would strengthen the evidence supporting our framework. However, such an evaluation is challenging due to the limited scientific literature that fully covers the relevant causal relationships for specific inputs and experimental settings. Accordingly, we use optimization and prediction performance to assess the practical utility of the generated hypotheses and to examine their scientific plausibility.

On the other hand, in real-world scientific discovery, observations cover only a small fraction of the input space, leaving hypotheses and principles potentially incomplete or incorrect, even when developed by experts. PRINCIPIA addresses this practical challenge by proposing a revision procedure as new observations accumulate.


Comment 2: Regarding the source of PRINCIPIA’s gain: a controlled comparison with raw experimental observations in place of causal hypotheses.

Response 2: We appreciate your valuable suggestion. To isolate the contribution of causal hypothesis generation in PRINCIPIA, we conduct an additional ablation in which the hypotheses are replaced with raw input-outcome pairs, while other procedures are retained. As shown in Table B.1, generating causal hypotheses improves performance compared to a variant that only uses raw input-outcome pairs, indicating that the benefit of PRINCIPIA comes from causal hypotheses.

JNK3GSK3BValsartan-SMARTSAvg.
Ours w/ Input-outcome pairs0.3370.3800.0950.271
Ours w/ Causal Hypotheses0.3500.4520.1060.303

Table B.1: Top-10 AUC comparison between using simple input-outcome pairs and causal hypotheses. We report the mean over three independent trials.


Comment 3: Small performance gains lack sufficient statistical support. Please provide paired per-run results or confidence intervals, particularly for molecular property prediction and the smaller optimization gains.

Response 3: We would like to note that Fig. 9 includes the standard deviation, and we additionally provide standard deviations for molecular property prediction in Table 9 of the revised paper. To further assess the consistency of the gains across paired runs, we conduct one-sided exact sign tests [1] using results matched by task and random seed. We aggregate these pairs separately for optimization and molecular property prediction, exclude ties, and report p-values. As shown in Tables B.2 and B.3, PRINCIPIA statistically achieves higher scores in the majority of paired runs against every baseline in both settings, indicating consistent improvements for paired per-run.

BaselinePerformance W/L/Tp-valueAUC@10 W/L/Tp-value
MolLEO27 / 2 / 18.12e-725 / 5 / 01.62e-4
ExLLM26 / 3 / 17.62e-623 / 6 / 11.16e-3
PiFlow28 / 2 / 04.34e-724 / 5 / 12.73e-4
WebSearch28 / 2 / 04.34e-724 / 6 / 07.15e-4
DeepEvolve27 / 3 / 04.22e-627 / 3 / 04.22e-6

Table B.2: PRINCIPIA’s wins, losses, and ties against each baseline and one-sided exact sign-test p-values on optimization tasks.

BaselineAccuracy W/L/Tp-valueMacro-F1 W/L/Tp-value
Direct23 / 1 / 01.49e-623 / 1 / 01.49e-6
Few-shot (k = 1)22 / 2 / 01.79e-522 / 2 / 01.79e-5
Few-shot (k = 3)22 / 0 / 22.38e-723 / 1 / 01.49e-6
Few-shot (k = 5)22 / 2 / 01.79e-523 / 1 / 01.49e-6
Few-shot (k = 7)22 / 1 / 12.86e-622 / 2 / 01.79e-5
Internal principles22 / 1 / 12.86e-622 / 2 / 01.79e-5
External principles23 / 1 / 01.49e-623 / 1 / 01.49e-6
Textual summary21 / 2 / 13.30e-522 / 2 / 01.79e-5

Table B.3: PRINCIPIA’s wins, losses, and ties against each baseline and one-sided exact sign-test p-values on molecular property prediction tasks.

[1] Demšar, Janez. "Statistical comparisons of classifiers over multiple data sets." Journal of Machine learning research 7.Jan (2006): 1-30.


Due to character limits, the response continues below.

AuthorsOpenReview ↗

Initial Response by Authors (2/3)

Comment 4: Regarding PRINCIPIA’s overall cost-effectiveness: whether oracle-call savings outweigh additional LLM computation, based on total LLM calls, token usage, and runtime relative to ExLLM at comparable optimization quality.

Response 4: As wet-lab experiments can be substantially more expensive and time-consuming than LLM inference, we primarily focused on the reduction of oracle calls. To rigorously compare the overall cost-effectiveness, we compare total LLM token usage, API cost, wall-clock time, and experimental overhead. Specifically, we use the oracle counts at matched performance on Syn-3bfo (Fig. 5) to estimate wet-lab overhead, assuming $50 and two days per evaluation under sequential execution [2,3,4] as a minimal cost. As shown in Table B.4, PRINCIPIA requires more LLM computation but substantially fewer oracle evaluations to reach comparable optimization quality. The additional LLM expense is small relative to the estimated experimental savings, yielding lower total estimated cost and time.

MethodToken Usage (million)LLM CallsLLM-API Cost ($)Wet-lab + LLM-API Cost ($)LLM Wall-Clock (day)Wet-lab Runtime + LLM Wall-Clock (day)
MolLEO4.08487.6350,007.630.0332,000.033
Ours4.66573.337,003.330.010280.010
ExLLM4.08737.3350,007.330.0352,000.035
Ours27.92,80416.4822,316.480.040892.040
WebSearch5.517609.1450,009.140.0572,000.057
Ours6.28254.298,204.290.012328.012
DeepEvolve9.15,04012.7850,012.780.1542,000.154
Ours4.66573.336,903.330.010276.010

Table B.4: Cost comparison at matched performance.

[2] Kosuri, Sriram, and George M. Church. "Large-scale de novo DNA synthesis: technologies and applications." Nature methods 11.5 (2014): 499-507.

[3] Wittmann, B. J., et al. "evSeq: Cost-effective amplicon sequencing of every variant in a protein library. ACS Synth Biol 11: 1313–1324." 2022,

[4] Fowler, Douglas M., Jason J. Stephany, and Stanley Fields. "Measuring the activity of protein variants on a large scale using deep mutational scanning." Nature protocols 9.9 (2014): 2267-2284.


Due to character limits, the response continues below.

AuthorsOpenReview ↗

Initial Response by Authors (3/3)

Comment 5: What are PiFlow’s actual unique oracle-evaluation counts and initial populations in Tables 1–2? Does its performance gap persist under the same evaluation budget and duplicate-handling policy?

Response 5: We would like to clarify that PiFlow uses a different optimization algorithm from the MolLEO-based methods: it generates candidates sequentially, whereas PRINCIPIA and the other baselines use a population-based genetic algorithm. We followed PiFlow’s original settings, using 24 iterations for molecular optimization and protein engineering. This resulted in an average of 15.73 and 20.67 unique oracle evaluations, respectively. For gene perturbation, we matched the number of distinct genes used for Hit Ratio evaluation across methods.

To address your concern, we conducted additional experiments under the same maximum oracle-evaluation budgets as PRINCIPIA, with three runs per task. PiFlow tends to generate duplicate candidates in the GB1 task of protein engineering, making it difficult to accumulate additional unique oracle evaluations. We therefore additionally provided PiFlow with previously proposed sequences in its prompts and instructed it not to propose them again. This introduces a potentially unfair advantage for PiFlow, as PRINCIPIA did not receive the same explicit list. Nevertheless, as shown in Tables B.5 and B.6, PRINCIPIA achieves higher performance on average, indicating that PRINCIPIA still outperforms PiFlow under the same oracle-evaluation budgets. We have updated Tables 1 and 2 in the revised paper to report these results under matched maximum oracle-evaluation budgets.

MethodJNK3GSK3BMedian1Median2Valsartan SmartsAvg.
Avg@100AUC@10Avg@100AUC@10Avg@100AUC@10Avg@100AUC@10Avg@100AUC@10AUC@10
PiFlow0.1580.2540.1840.2750.2510.2830.1570.1640.0000.0000.195
Ours0.4480.3500.6170.4520.2980.2840.2200.2030.1170.1060.279

Table B.5: Molecule optimization under matched maximum oracle budgets for three runs.

MethodSyn-3bfoAAVGB1Scharenberg22IFNGAvg.
Avg@50AUC@10Avg@50AUC@10Avg@50AUC@10Hit RatioAUC@10Hit RatioAUC@10AUC@10
PiFlow-29.8130.1640.1400.1010.4680.4620.2890.5570.1110.6820.393
Ours0.0840.7420.6810.5270.3980.3830.3080.6300.1140.7420.605

Table B.6: Protein engineering results under matched maximum oracle budgets. Gene perturbation results are adapted from the original evaluation, which already used matched maximum oracle budgets.


Comment 6: Can the generated principles make accurate, falsifiable predictions on held-out comparisons—for example, predicting the direction of fitness changes? Does revision improve this accuracy?

Response 6: We appreciate your insightful suggestion. In response, we conducted additional experiments to evaluate whether the generated principles from one discovery process are helpful to predict fitness changes from separate runs. As shown in Table B.7, providing the generated principles consistently improves the prediction accuracy, suggesting the correctness of test-time principle construction and its effectiveness in providing reusable knowledge.

MethodJNK3Median1Median2
w/o principles69.6781.6774.33
w/ principles74.6784.3384.00

Table B.7: Fitness direction prediction accuracy (%) with and without PRINCIPIA principles

Additionally, to assess the contribution of the revision stage, we conduct the prediction of fitness changes based on principles constructed with or without revision. As shown in Table B.8, revision improves prediction accuracy, suggesting that it helps construct principles more accurately.

MethodAccuracy (%)
w/o Revision70.33
w/ Revision74.67

Table B.8: Fitness direction prediction accuracy (%) on held-out JNK3 comparisons with and without revision, averaged over three independent trials.



We thank the reviewer for your time and feedback and hope that our responses have addressed your concerns. If you have any further questions, we are happy to address them.

AuthorsOpenReview ↗

Brief Summarization of Rebuttal for Reminder

Dear Reviewer LtxC,

We sincerely appreciate your careful review and constructive feedback. To facilitate your review, we briefly summarize our rebuttal below.



C1 & C6. Validity of generated principles

We explained that relevant causal relationships may not be fully established in the literature, limiting systematic validation. Limited observations can also leave even expert-developed hypotheses and principles incomplete, motivating our revision procedure (Response 1). Following your suggestion, we compared fitness-change direction predictions with and without principles constructed in a separate discovery run, and separately compared revised and unrevised principles (Response 6). The principles improved prediction on held-out comparisons, supporting their predictive validity and reusability. Revision further improved accuracy, demonstrating its value in refining knowledge formed from incomplete observations.


C2. Source of the gains: a controlled comparison with raw experimental observations in place of causal hypotheses

To isolate hypothesis generation’s contribution, we retained the evidence, clustering, and retrieval procedures while replacing hypotheses with raw input-outcome pairs. The hypothesis-based variant performed better, supporting a contribution from explanatory hypotheses beyond access to the same observations.


C3. Statistical support

We conducted sign tests using results matched by task and random seed, separately for optimization and molecular property prediction. These analyses support consistent overall gains against every baseline across paired runs in both settings.


C4. Overall cost-effectiveness

At comparable optimization quality, we measured LLM usage, API cost, and runtime and estimated the corresponding experimental overhead. Estimated experimental savings outweighed the additional LLM overhead relative to ExLLM. PRINCIPIA also incurred lower LLM cost and runtime than several other baselines.


C5. PiFlow comparison under matched budgets

We evaluated PiFlow under the same maximum oracle budgets and provided additional support for avoiding duplicate candidates where needed. PRINCIPIA maintained higher average performance, showing that its advantage remains under matched budget limits.



We sincerely thank you again for your constructive feedback and hope that the additional results and clarifications have helped address your concerns. As the discussion period approaches its end, we would greatly appreciate it if you could let us know whether our responses have sufficiently addressed your main concerns.

Best regards,

Authors

Reviewer_qe39OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PRINCIPIA, a large language model framework for constructing and continually updating task-specific scientific principles during test-time discovery. Rather than relying solely on pretrained knowledge, web retrieval, or summaries of prior experiments, PRINCIPIA compares related inputs and observed outcomes to propose causal hypotheses. It clusters similar inputs, synthesizes recurring hypotheses into broader inductive principles, and retrieves both forms of knowledge to guide subsequent predictions or candidate optimization. New observations then revise the hypotheses and regenerate the principles, creating an iterative discovery loop. The authors evaluate the method on five molecular optimization tasks, three protein engineering tasks, two gene perturbation tasks, and eight molecular property prediction benchmarks. PRINCIPIA generally outperforms relevant optimization and prediction baselines; on property prediction, average accuracy reaches 69.75%. Ablation studies support the contribution of comparative reasoning, principle induction, retrieval, and revision. Sample-efficiency analyses report fewer oracle evaluations, while qualitative examples show consistency with scientific literature.

Critical strengths

One strength is the clear formulation of test-time scientific knowledge construction as a distinct problem from retrieving external knowledge or summarizing previous experiments. PRINCIPIA integrates comparative hypothesis generation, inductive principle construction, and observation-driven revision into a coherent framework. This is meaningful because it enables accumulated observations to guide subsequent scientific decisions through explicit, reusable knowledge.

A second strength is the relatively broad empirical evaluation across molecular optimization, protein engineering, gene perturbation, and molecular property prediction. Comparisons against multiple knowledge-augmented baselines demonstrate consistent performance improvements. The ablation studies further support the usefulness of individual components, while the reported reduction in oracle evaluations suggests potential practical benefits for resource-constrained discovery.

Nevertheless, these strengths primarily establish empirical effectiveness rather than substantial methodological novelty. Moreover, the scientific validity of the generated causal explanations remains insufficiently demonstrated. Consequently, while the framework is promising and experimentally well motivated, these contributions provide only limited support for acceptance.

Critical weaknesses

The first concern is that the paper's central claim of constructing causal scientific principles is insufficiently validated. Hypotheses are inferred from input-outcome comparisons that cannot isolate causal effects when multiple features differ. Qualitative examples and agreement with existing literature establish plausibility, but do not demonstrate whether the induced principles are mechanistically correct. This weakens the paper's main conceptual contribution.

Second, the practical efficiency claims are incomplete. Reported savings focus on oracle evaluations, while PRINCIPIA requires repeated LLM calls for hypothesis generation, principle induction, retrieval, and revision. Without end-to-end computational cost or wall-clock comparisons, it is difficult to assess overall efficiency relative to simpler memory-based baselines. Furthermore, evaluations rely primarily on benchmark scoring functions, predictive models, or lookup-based oracles rather than prospective laboratory experiments, limiting confidence in real-world applicability. These concerns do not negate the promising empirical results, but leave sufficient uncertainty regarding scientific validity and practical benefits.

Questions for authors

see weakness

4 comments
AuthorsOpenReview ↗

Initial Response by Authors (1/2)

Thank you for your constructive comments. We appreciate your recognition of the following strengths:

  • A clear problem formulation and a coherent framework for test-time scientific knowledge construction.
  • Broad empirical evaluation with consistent improvements over knowledge-augmented baselines.
  • Useful component analyses and potential practical benefits from reducing oracle evaluations.

We address your concerns below.



Comment 1-1: The first concern is that the paper's central claim of constructing causal scientific principles is insufficiently validated.

Response 1-1: Unfortunately, due to a lack of scientific literature that fully addresses causal relationships, the rigorous validation of the scientific correctness of the generated hypotheses and principles is limited. Accordingly, we provide optimization/prediction performance to evaluate the generated explanations, where our PRINCIPIA outperforms baselines.

Nevertheless, to complement these evaluations, we additionally assess whether principles constructed in one discovery run improve the prediction of fitness-change directions on held-out comparisons from separate runs. As shown in Table A.1, providing these principles consistently improves prediction accuracy, providing quantitative evidence supporting the correctness of the input–outcome relationships captured by the generated principles.

MethodJNK3Median1Median2
w/o principles69.6781.6774.33
w/ principles74.6784.3384.00

Table A.1: Fitness direction prediction accuracy (%) with and without PRINCIPIA principles

On the other hand, even hypotheses and principles developed by human experts can be incomplete, as observations cover only a fraction of the input space in real-world drug discovery. To address this limitation, we propose a revision algorithm that reassesses previously used hypotheses as new observations accumulate and reconstructs principles from the updated explanations. Accordingly, our contribution within Applications to Sciences addresses the practical need to construct and continually update scientific knowledge from incomplete observations.


Comment 1-2: Qualitative examples and agreement with existing literature establish plausibility, but do not demonstrate whether the induced principles are mechanistically correct.

Response 1-2: To avoid potential misunderstanding, we would like to clarify the distinction between causality and mechanisms. Causality is a more comprehensive conceptual principle for understanding causes and their resulting effects with or without understanding intermediates factors, whereas a mechanism refers to a more detailed (lower-level) principle for understanding the intermediate elements that exist between a cause and its effect.

While both are important, researchers often seek to understand high-level causal relationships before investigating the underlying mechanisms in detail, when studying all the intermediate factors requires substantial cost. For example, when new disease targets are discovered, drug developers might first study high-level causality during discovering effective drugs, and then deepen their understanding towards mechanisms based on the high-level causality.

Our work primarily focuses on this high-level causality of which input changes affect the final, targeted outcome, proposing an LLM-based framework that constructs observational knowledge at test time. By automating this knowledge construction process and using the resulting principles to guide subsequent investigation, PRINCIPIA is able to reduce expert labor and experimental costs. Its practical contribution is to support scientific discovery through high-level empirical principles even when the intermediate processes are not yet fully understood.


Due to character limits, the response continues below.

AuthorsOpenReview ↗

Initial Response by Authors (2/2)

Comment 2: Second, the practical efficiency claims are incomplete. Reported savings focus on oracle evaluations, while PRINCIPIA requires repeated LLM calls for hypothesis generation, principle induction, retrieval, and revision. Without end-to-end computational cost or wall-clock comparisons, it is difficult to assess overall efficiency relative to simpler memory-based baselines.

Response 2: We primarily focused on the oracle efficiency as experimental costs and turnaround times can far exceed those of LLM inference. To rigorously assess it, we compare total LLM token usage, API cost, wall-clock time, and experimental overhead. To this end, we first conservatively estimate the wet-lab experimental overhead on Syn-3bfo structure, stability, and functionality where one evaluation costs $50 and takes 2 days at minimum [1,2,3], and report costs at matched performance as shown in Fig. 5. As shown in Table A.2, although PRINCIPIA incurs additional LLM overhead, it requires substantially fewer oracle evaluations to reach comparable performance. Therefore, the savings from fewer experimental evaluations outweigh the additional LLM overhead and PRINCIPIA is capable of reducing estimated total cost and time and improving overall efficiency in scientific discovery.

MethodToken Usage (million)LLM CallsLLM-API Cost ($)Wet-lab + LLM-API Cost ($)LLM Wall-Clock (day)Wet-lab Runtime + LLM Wall-Clock (day)
MolLEO4.08487.6350,007.630.0332,000.033
Ours4.66573.337,003.330.010280.010
ExLLM4.08737.3350,007.330.0352,000.035
Ours27.92,80416.4822,316.480.040892.040
WebSearch5.517609.1450,009.140.0572,000.057
Ours6.28254.298,204.290.012328.012
DeepEvolve9.15,04012.7850,012.780.1542,000.154
Ours4.66573.336,903.330.010276.010

Table A.2: Cost comparison at matched performance.

[1] Kosuri, Sriram, and George M. Church. "Large-scale de novo DNA synthesis: technologies and applications." Nature methods 11.5 (2014): 499-507.

[2] Wittmann, B. J., et al. "evSeq: Cost-effective amplicon sequencing of every variant in a protein library. ACS Synth Biol 11: 1313–1324." 2022,

[3] Fowler, Douglas M., Jason J. Stephany, and Stanley Fields. "Measuring the activity of protein variants on a large scale using deep mutational scanning." Nature protocols 9.9 (2014): 2267-2284.


Comment 3: Evaluations rely primarily on benchmark scoring functions, predictive models, or lookup-based oracles rather than prospective laboratory experiments, limiting confidence in real-world applicability.

Response 3: We agree that prospective laboratory experiments would provide stronger evidence of real-world applicability. Unfortunately, conducting such experiments requires specialized equipment, trained experts, and substantial costs, which are beyond the resources available for ML study. Given these practical constraints, benchmarks commonly used in ML-based scientific discovery mitigate these problems by leveraging scoring functions or predictive models which enable controlled and reproducible evaluation. Our evaluation follows this established practice, as exemplified by benchmarks such as PMO [4], allowing us to compare methods under limited oracle evaluation budgets.

[4] Gao, Wenhao, et al. "Sample efficiency matters: a benchmark for practical molecular optimization." Advances in neural information processing systems 35 (2022): 21342-21357.



We thank the reviewer for your time and feedback and hope that our responses have addressed your concerns. If you have any further questions, we are happy to address them. If your concerns are appropriately addressed, we hope you kindly consider updating the rating accordingly.

AuthorsOpenReview ↗

Brief Summarization of Rebuttal for Reminder

Dear Reviewer qe39,

We sincerely appreciate your careful review and constructive feedback. To facilitate your review, we briefly summarize our rebuttal below.



C1. Scientific validity of generated principles

We explained that rigorous validation against scientific literature is limited as the relevant causal relationships are not fully established. To mitigate this limitation, we evaluated whether principles from one discovery run improve fitness-change predictions on held-out comparisons from separate runs. The principles consistently improved prediction, supporting the predictive validity of the generated principles. We further explained that limited observations can leave even expert-developed hypotheses incomplete. Our revision procedure addresses this limitation by reassessing hypotheses and reconstructing principles as new evidence becomes available.


C2. Overall cost-effectiveness

To assess overall efficiency, we compared LLM cost and runtime at matched optimization performance and incorporated estimated wet-lab overhead. PRINCIPIA achieved lower LLM cost and runtime than several baselines. Even where LLM overhead increased, savings from fewer experimental evaluations reduced estimated total cost and time.


C3. Practical constraints on wet-lab validation

We explained that prospective wet-lab experiments require specialized resources beyond those available for this AI study. We therefore followed established practice in AI-driven scientific discovery, using benchmark scoring functions and predictive models for controlled, reproducible evaluation under limited oracle budgets.



We sincerely thank you again for your constructive feedback and hope that the additional results and clarifications have helped address your concerns. As the discussion period approaches its end, we would greatly appreciate it if you could let us know whether our responses have sufficiently addressed your main concerns.

Best regards,

Authors

Dear Reviewer qe39,

As the discussion period nears its end, we would like to follow up on our rebuttal responses. We have provided additional held-out prediction experiments to evaluate the generated principles, cost analyses at matched performance, and clarifications on the scope of our benchmark-based evaluation. We hope these results and explanations have helped address your concerns regarding scientific validity and practical applicability.

If you have any further questions, we are happy to address them. If you find that our responses have adequately addressed your concerns, we would greatly appreciate it if you could consider revisiting your initial rating accordingly.

Thank you again for your time and thoughtful feedback.

Sincerely,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.