acceptodds
Under review as a conference paper at ICLR 2027

Towards Literature-Grounded Agentic Co-Crystal Synthesis Design

Abstract

Co-crystallization has become an important approach for modulating the physicochemical properties of drug molecules, yet experimental development remains challenging. While recent machine-learning methods have advanced co-crystal screening, the downstream problem of synthesis design remains largely unresolved. We present CoSyn, a literature-grounded LLM-based agentic framework for supporting co-crystal synthesis design that combines (i) hierarchical extraction of method-specific synthesis information, (ii) a large-scale machine-readable database of 20,783 co-crystal synthesis records across multiple methods, and (iii) method-specific starting-condition prediction for new molecular pairs. We systematically evaluate each component using manually annotated datasets and prospectively validate recommended conditions, achieving phase-pure co-crystals for three of four held-out pairs. At the system level, tool-augmented orchestration improves the average score on our targeted domain benchmark from 1.88 for an LLM without tool use to 3.58 on a 1–5 scale. These results demonstrate that structured, literature-derived experimental evidence can support co-crystal synthesis decision-making beyond pair-level screening.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Co-crystals, Synthesis design, Large language models, Agentic AI, Information extraction, Condition prediction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GZnbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CoSyn, a literature-grounded framework for pharmaceutical co-crystal synthesis design. The authors construct CoSyn-DB, containing 20,783 structured synthesis records extracted from scientific literature through a hierarchical, method-aware extraction pipeline. The database is subsequently used for method-specific synthesis-condition prediction with retrieval-augmented few-shot LLMs. Finally, the authors integrate literature analysis, database retrieval, and condition prediction into single-agent and multi-agent systems.

The proposed framework is evaluated through manually annotated extraction datasets, comparisons with classical machine-learning baselines, prospective synthesis experiments on four molecular pairs, and a newly constructed 30-question benchmark (CoSyn-Bench). The results demonstrate the feasibility of literature-grounded condition recommendation and improved benchmark performance through tool-augmented workflows.

Critical strengths

S1. A useful structured resource for an underexplored downstream task.

The construction of CoSyn-DB, comprising 20,783 method-specific synthesis records, addresses the practical challenge of organizing heterogeneous experimental information from the literature. Unlike conventional co-crystal screening tasks, the proposed resource connects molecular pairs with synthesis conditions, potentially enabling further research on synthesis-condition recommendation.

S2. Experimental validation beyond purely computational evaluation.

The paper evaluates extraction and prediction components independently and includes prospective experimental validation. Obtaining phase-pure co-crystals for three of four tested molecular pairs provides preliminary evidence that literature-grounded recommendations can support practical synthesis experiments. The inclusion of tool ablations and independent expert evaluation also strengthens the empirical assessment.

Critical weaknesses

W1. The necessity and distinctive contribution of LLM-based prediction and agentic orchestration remain insufficiently established.

The strongest contribution appears to be the construction of a structured synthesis database rather than a novel LLM-based methodology. In Table 3, the LLM predictors achieve an Overall F1 comparable to the Random Forest baseline, making their additional predictive value unclear. Furthermore, although the agentic systems outperform LLM-only and static-context baselines, the experiments do not fully distinguish the benefits of agentic orchestration from those of accessing domain-specific tools and structured evidence.

W2. The system-level evaluation has limited scope and generalizability.

CoSyn-Bench contains only 30 questions designed around the capabilities of the proposed system. Some tasks primarily test database lookup or predefined combinations of retrieval and prediction, raising concerns about whether the benchmark captures broader synthesis-design reasoning. Although LLM-as-a-judge evaluation is supported by human agreement analysis, the expert evaluation covers only the two full agentic architectures, leaving the reported improvements over non-agentic baselines without direct human validation.

W3. The evidence for general synthesis-design capability remains preliminary.

The prospective evaluation is encouraging but includes only four literature-confirmed molecular pairs under a fixed synthesis method. This demonstrates the potential usefulness of predicted starting conditions, but does not establish generalization to broader synthesis methods, previously unconfirmed molecular pairs, or more challenging experimental settings. The relatively limited record-level exact-match performance also raises questions about the reliability of downstream recommendations for sensitive experimental parameters.

Questions for authors

Q1. Necessity of LLM-based condition prediction.

In Table 3, the LLM predictors achieve Overall F1 scores comparable to the Random Forest baseline, despite using additional retrieved literature examples. This raises concerns about whether the LLM component provides a meaningful predictive advantage over conventional methods, particularly given its additional computational cost and complexity. Since LLM-based condition prediction is a central component of the proposed framework, the lack of a clear performance improvement makes its methodological contribution difficult to assess. How does CoSyn-Predictor compare against a simple retrieval-only baseline that directly transfers or aggregates synthesis conditions from the nearest literature examples?

Q2. Contribution of agentic orchestration.

Although the tool ablations demonstrate the importance of individual capabilities, the comparisons do not fully isolate the benefit of agentic orchestration. How would a deterministic, non-agentic workflow with access to the same retrieval, analysis, and prediction tools perform under a comparable execution budget?

Q3. Generalizability beyond the self-constructed benchmark.

Since CoSyn-Bench is designed around the predefined capabilities of the proposed framework, strong performance may primarily reflect successful execution of these specific workflows rather than generalizable scientific reasoning or synthesis-design ability. Have the authors evaluated the system on an independent, more general, or out-of-distribution benchmark involving synthesis-design tasks beyond those explicitly covered by CoSyn's tools?

Q4. Novelty of synthesis recommendations.

The prospective experiments validate starting conditions for four literature-confirmed co-crystal-forming pairs under a fixed synthesis method. However, it remains unclear whether the framework can scientifically reason about and propose novel synthesis conditions beyond transferring or adapting existing literature precedents. Can the authors provide experimental evidence or concrete examples where CoSyn successfully proposes and validates novel synthesis conditions that are not directly supported by closely related literature records?

5 comments
AuthorsOpenReview ↗

Author Response [Part 1/2]

We thank the reviewer for recognizing the value of CoSyn-DB and our prospective experimental validation. To address the concerns raised, we added controlled baselines, expanded expert evaluation, and clarified the scope and reliability of our synthesis recommendations.

[W1/Q1]. Value of LLM prediction

CoSyn-DB provides the structured experimental evidence needed to study protocol-level synthesis-condition prediction. Our goal is to assess how effectively LLMs use this evidence compared with simpler approaches. We therefore added two retrieval-only baselines: NN-1 directly transfers conditions from the nearest literature record, while kNN-3 aggregates conditions from the three nearest records.

On the held-out grinding set, Gemini-2.5-Flash achieves an Overall-Single F1 of 0.49, compared with 0.46 for NN-1, 0.47 for kNN-3, and 0.46 for Random Forest. Its Numerical F1 is slightly higher than the retrieval-only baselines (0.48 compared with 0.46), while Random Forest performs better on solvent and categorical parameters (Table 3). We also added a molecular-similarity analysis, showing that Gemini-2.5-Flash maintains an Overall F1 of 0.48 in both the lower- and higher-similarity subsets (Appendix B.12).

These results show that LLM-based prediction is competitive but does not consistently outperform simpler methods. The added comparisons provide a clearer understanding of where LLMs may offer benefits and where direct retrieval remains sufficient for literature-grounded synthesis-condition recommendation.

[W1/Q2]. Value of agentic coordination

To isolate the value of agentic coordination beyond tool access, we added two non-agentic baselines using the same GPT-4o-mini backbone, domain tools, and evidence as Single-Agent (SA) and Multi-Agent (MA). Fixed Workflow (FW) executes a predefined tool sequence, while LLM-Routed (LR) selects a predefined workflow without replanning.

The results reveal a clear distinction by task complexity. On multi-capability questions, SA and MA achieve scores of 3.25 and 3.35, compared with 2.15 for FW and 2.63 for LR. On single-capability questions, however, FW and LR remain competitive and slightly outperform the agents (Table 5). This suggests that flexible coordination is most useful when a task requires combining several capabilities, rather than executing a predefined workflow.

We also examined the execution costs of this flexibility. FW and LR require fewer LLM calls, tokens, and less time than the agentic systems. Among the agents, MA uses considerably fewer tokens than SA (20,130 compared with 57,611), but requires more calls and longer execution time (Appendix C.6, Table 31). The added baselines thus reveal a clear quality-efficiency trade-off: non-agentic workflows are more economical for single-capability tasks, while agentic coordination is more beneficial for multi-capability tasks, with SA offering lower latency and MA lower token consumption.

AuthorsOpenReview ↗

Author Response [Part 2/2]

[W2/Q3]. Benchmark validity

CoSyn-Bench was deliberately designed to evaluate literature-grounded co-crystal synthesis assistance. To our knowledge, no established external benchmark directly assesses method-specific synthesis-condition recommendation and the coordination of supporting tools. We therefore developed 30 expert-curated single- and multi-capability questions to evaluate these workflows in a consistent setting. We have clarified this intended scope in Section 5.1.

To strengthen the reliability of this evaluation, we expanded the independent expert assessment to 120 responses from four systems, evaluated by two experts. The LLM-expert Spearman correlation (0.806) was comparable to inter-expert agreement (0.792). Beyond measuring agreement, we also compared expert-assigned scores across systems, with MA achieving the highest mean Overall Score of 3.63 (Appendix C.7, Tables 32-33).

The expanded expert assessment strengthens confidence in the comparisons made within CoSyn-Bench. Assessing broader applicability would require testing CoSyn across a wider range of co-crystal syntheses, including more challenging molecular systems, and evaluating its use in experimental workflows.

[W3/Q4]. Experimental applicability

Adapting established synthesis knowledge to specific co-crystal pairs is precisely the task CoSyn addresses. CoSyn-DB organizes a broad range of experimentally reported conditions, providing a foundation for selecting suitable quantities, solvents, and processing parameters for a target pair, even when its own synthesis protocol is unavailable.

We tested this capability prospectively, excluding all records for four target pairs before generating recommendations. The proposed conditions were pair-specific: CoSyn recommended an acetonitrile–water mixture with 12 h stirring for caffeine–oxalic acid, and water with 48 h stirring for urea–oxalic acid. Both yielded phase-pure co-crystals (Appendix B.13). Overall, CoSyn produced three phase-pure co-crystals in four experiments, while the baseline yielded two mixed products and two failures (Table 4). As clarified in Section 4.3, unavailable ethyl acetate was replaced with 1,2-dimethoxyethane based on similar relative polarity, which limits the comparison.

We also examined extraction reliability. The 25.3% Full-record Exact Match requires agreement across all protocol components, including processing details. Our new hierarchical metrics show higher agreement for chemical composition (65.2%) and quantitative composition (52.5%) (Appendix A.11, Table 12). Our error-propagation analysis also found comparable downstream Overall F1 using manually annotated and extracted records (0.23 compared with 0.24, Appendix B.11, Table 23), indicating no substantial aggregate degradation in the evaluated setting.

These findings support the practical value of CoSyn-DB as a structured evidence resource and demonstrate the initial feasibility of literature-grounded synthesis-condition recommendation. Broader prospective validation across synthesis methods and more challenging molecular systems remains an important next step.

We believe these additions strengthen the evaluation of CoSyn and clarify its contributions to literature-grounded synthesis design. We would be grateful if the reviewer would reconsider their rating in light of the new evidence.

Revision: Table 3 (L277–L285), Section 4 (L297–L315), Sections 4.2–4.3 (L345–L385), Sections 5.1–5.2 and Table 5 (L399–L442), Section 6 (L516–L517), Appendix A.11 (L1694–L1707), Appendix B.12 (L2291–L2314), Appendix B.13.1 (L2324–L2325), Appendix C.3.2 (L2634–L2747), Appendix C.6–C.7 (L3090–L3156).

Thank you for the detailed response and additional experiments. I have two remaining questions.

Q2/Q3. Statistical significance of agentic improvements

Given the small size of CoSyn-Bench (30 questions, including only 10 multi-capability questions), could the authors report 95% confidence intervals for the performance differences between agentic and non-agentic baselines, using paired bootstrap resampling over questions? This would help establish whether the reported improvements from agentic coordination are statistically reliable.

Q4. Novelty of the recommended synthesis conditions

Is it possible to quantify the number of possible and unique synthesis-condition combinations based on the parameter values in CoSyn-DB, and assess how many previously unseen combinations were experimentally validated by CoSyn? For instance, comparing the recommended conditions (e.g., solvents, temperature, time, and stoichiometric ratios) with the top-3 retrieved examples and broader CoSyn-DB records, alongside molecular similarity, could help clarify the added value of LLM-based prediction beyond direct retrieval of existing protocols.

Reviewer_i9eEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces COSYN, a system for assisting co-crystal synthesis by recommending specific initial experimental conditions for a given molecular pair and synthesis method. The system constructs the COSYN-DB dataset from the relevant literature through a three-stage pipeline comprising synthesis procedure detection, method classification, and parameter extraction. The condition module retrieves the three most similar molecular pair records associated with the same synthesis method and uses an existing LLM to fill a specific procedural template. The framework further integrates literature analysis, database retrieval, and condition prediction into both single-agent and multi-agent systems.

Critical strengths
  1. COSYN-DB provides a valuable resource for training LLMs on scientific tasks.
  2. The evaluation of the proposed method spans multiple stages, from literature extraction to experimental validation, and is generally rigorous.
Critical weaknesses
  1. The input information is insufficient to determine some prediction targets, resulting in a partial mismatch between the task formulation and the evaluation protocol. Specifically, the targets include absolute quantities that depend on experimental scale and experimenter choices, yet these constraints are not explicitly specified in the input. Consequently, agreement with reference conditions may not accurately reflect the appropriateness of the recommended protocol. Moreover, multiple valid combinations of conditions may exist for the same molecular pair, making the task inherently multi-solution.
  2. The definition of the system’s Pass Rate does not guarantee answer correctness, as substantial disparities across the four criterion scores can coexist with a high overall mean.
Questions for authors

See the weaknesses.

2 comments
AuthorsOpenReview ↗

Author Response

We thank the reviewer for the positive assessment of CoSyn-DB and our evaluation. We have addressed limitations in reference-based synthesis prediction and strengthened system-level evaluation with additional analyses and stricter metrics.

[W1]. Validity of prediction evaluation

Experimental scale. We agree that experimental scale is not specified by the molecular pair and synthesis method, although predictions include absolute quantities. We have therefore clarified in Section 4 that CoSyn recommends literature-grounded starting conditions and that numerical F1 measures agreement with reported values rather than experimental feasibility. The latter is assessed separately through prospective validation, where three of four held-out pairs yielded phase-pure co-crystals (Section 4.3).

To examine the influence of scale on retrospective evaluation, we added an analysis that adjusts predicted coformer and solvent quantities to the reference API amount. Other numerical parameters were left unchanged because scale dependence is not necessarily proportional. For example, solvent volume affects concentration and may also influence evaporation time. Numerical F1 increased by up to 0.042 across the three methods, although the changes were not statistically significant (Appendix B.15). This analysis clarifies the limitations of adjusting quantities independently and motivates a more comprehensive approach to scale adaptation.

We propose making the target API amount an explicit user input. Besides specifying the intended scale, it could guide retrieval of structurally similar literature protocols conducted at comparable quantities, providing a basis for recommending the remaining numerical conditions at that scale (Section 6).

Multiple valid protocols. We also examined how frequently the literature reports alternative synthesis conditions for the same molecular pair. Across the three dominant methods, 18.4% to 26.8% of pairs have multiple distinct synthesis records (Appendix B.14, Figure 10). These records do not always agree. For grinding, mean agreement across all eligible protocol comparisons was 0.815, but dropped to 0.550 when considering only protocols from different publications. This difference shows that reported conditions can vary substantially even for the same molecular pair and synthesis method (Appendix B.14, Table 25).

To reflect this variability, we added Overall-Single and Overall-Best columns to Table 3. Overall-Best evaluates predictions against the best-matching reported protocol for the same pair and method. For Gemini-2.5-Flash on grinding, Overall F1 increases from 0.485 to 0.502 on the full test set, and from 0.509 to 0.563 among pairs with multiple protocols. Similar trends were observed for slurry and solution crystallization (Appendix B.14, Table 26).

Reporting both metrics provides a more representative assessment of literature agreement without treating a single reported protocol as the only acceptable reference.

[W2]. Correctness-aware evaluation

We agree that averaging four criteria can mask failures, particularly in Correctness. We therefore added two success metrics in Section 5.1. Correctness Pass Rate requires a Correctness score of at least 4, while Strict Pass Rate requires all four criteria to meet this threshold. Together with the original Pass Rate, these explicitly assess correctness and response reliability.

The additional metrics reveal differences that were previously hidden by aggregate scoring. For GPT-4o-mini Single-Agent, Pass Rate is 0.47, while Correctness Pass Rate and Strict Pass Rate are 0.33 and 0.32, respectively. Multi-Agent shows a similar pattern (0.48, 0.33, and 0.32). We also added failure rates for each evaluation criterion, identifying Correctness as the most frequent source of failure across tool-using systems (Appendix C.8, Tables 35-36). These results confirm the reviewer's concern and demonstrate that the revised evaluation can distinguish high average scores from responses that meet explicit correctness requirements.

To independently validate the automated evaluation, two experts assessed 120 responses. LLM-expert Spearman correlation (0.806) was comparable to inter-expert agreement (0.792). Experts also identified Correctness as the weakest criterion across all four systems, consistent with automated results (Appendix C.7, Tables 32-33).

Together, the new correctness-aware metrics and independent expert validation strengthen the reliability of our evaluation by explicitly distinguishing successful responses from those with remaining errors.

We hope these additional results address both concerns and, in light of your positive initial assessment, would be grateful if you would consider increasing your rating.

Revision: Table 3 (L272–L286), Section 4 (L297–L299), Sections 4.1–4.3 (L335–L385), Sections 5.1–5.2 (L417–L467), Section 6 (L522–L523), Appendix B.14–B.15 (L2430–L2537), Appendix C.7–C.8 (L3122–L3238).

Thanks for the detailed responses, which have addressed my concerns. I would like to maintain my positive score for acceptance.

Reviewer_ddbhOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoSyn, a literature grounded agentic framework for co-crystal synthesis design that integrates literature extraction, condition prediction, and agent-based decision support. Instead of focusing on co-crystal formation prediction, the authors introduce a three-stage extraction pipeline to construct CoSyn-DB, containing 20,783 synthesis records across 12 methods. These records are used by a retrieval augmented predictor that combines molecular similarity with few-shot LLM prompting to recommend synthesis conditions for new molecular pairs. The framework further integrates literature analysis, database retrieval, and condition prediction into single-agent and multi-agent systems. The paper demonstrates its effectiveness through extraction evaluations, condition prediction experiments, prospective synthesis validation, and an agentic benchmark.

Critical strengths
  • The paper presents a unified, literature grounded framework for co-crystal synthesis design that goes beyond conventional molecular pair screening and focuses on downstream experimental conditions.

  • The construction of CoSyn-DB, containing 20,783 synthesis records across 12 methods, provides a potentially valuable structured resource for synthesis condition prediction and scientific decision support.

  • The framework is reasonably engineered, with systematic evaluations of literature extraction, condition prediction, and agentic tool use, as well as prospective experimental validation that demonstrates its potential practical utility.

Critical weaknesses
  • The proposed agentic framework largely relies on standard tool-calling and orchestration over existing modules. The paper provides little methodological novelty in planning or collaboration, and the multi-agent system performs similarly to the single-agent counterpart despite higher execution costs.

  • The paper frames CoSyn as an agentic synthesis-design framework, yet the evaluation relies on only 30 self-constructed questions, mostly assessing individual tool capabilities. There are no comparisons with alternative agentic architectures using the same tools, nor systematic evaluations under more challenging synthesis-design scenarios.

  • The condition predictor mainly relies on similarity-based retrieval and few-shot LLM prompting, but lacks comparisons with simpler retrieval-based methods. The best LLMs only match Random Forest in overall F1, while the evaluation measures agreement with historical conditions rather than actual synthesis feasibility.

  • The reliability and practical utility of CoSyn remain insufficiently validated. Only 25.3% of matched synthesis records achieve exact agreement, and prospective experiments cover just four known molecular pairs under one synthesis method. The solvent substitution in the baseline further weakens the experimental comparison.

Questions for authors
  • What are the key methodological contributions of the proposed agentic architecture beyond standard tool-calling and orchestration? Given the comparable performance of single-agent and multi-agent systems, what justifies the additional complexity of the latter?

  • How would CoSyn perform against existing general-purpose agentic frameworks equipped with the same domain-specific tools? Can the authors provide more comprehensive evaluations on challenging multi-step synthesis design tasks to demonstrate the robustness and generalizability of the framework?

  • How does the proposed condition predictor compare with simpler similarity-based retrieval methods, such as nearest neighbor condition transfer? To what extent do the reported prediction metrics reflect actual synthesis feasibility rather than agreement with historical records?

  • How do extraction errors affect the reliability of downstream synthesis recommendations? Can the authors provide stronger experimental evidence beyond four known molecular pairs and clarify the impact of solvent substitution on the baseline comparison?

3 comments
AuthorsOpenReview ↗

Author Response [Part 1/2]

We thank the reviewer for recognizing the value of CoSyn-DB and the focus on synthesis conditions beyond molecular-pair screening. We have clarified the scientific contribution and added comparisons that directly examine the value of LLM prediction, agentic coordination, and the reliability of the underlying evidence.

[W1/Q1] Scientific contribution

We agree that standard tool calling and orchestration do not constitute new planning algorithms. CoSyn’s scientific contribution lies in a method-aware extraction pipeline, CoSyn-DB with 20,783 synthesis records across 12 methods, and the formulation and evaluation of literature-grounded synthesis-condition recommendation as a downstream task. Predictive models and agentic architectures are investigated as approaches to using this structured evidence. We revised the contributions in the Introduction to make this distinction explicit.

Within this framework, comparing SA and MA allows us to assess whether distributing scientific capabilities among specialist agents improves performance or efficiency. With GPT-4o-mini, SA and MA achieve similar overall scores of 3.58 and 3.56. MA uses substantially fewer median tokens (20,130 compared with 57,611), but requires more calls and longer execution time. Extending the comparison to Claude Sonnet 5.5 shows that SA achieves a higher overall score (3.76 compared with 3.44), with lower latency and similar token usage (Tables 5–6, Appendix C.6).

These findings support SA as a simpler implementation of CoSyn’s scientific capabilities. MA offers a token-efficiency benefit in the GPT-4o-mini setting, but its additional complexity is not consistently rewarded across backbones. The revised manuscript therefore presents the architectures as alternatives with measured trade-offs, without making the contribution depend on multi-agent superiority.

[W2/Q2] Agent evaluation

To distinguish the benefits of coordination from access to domain tools, we added two non-agentic baselines using the same GPT-4o-mini backbone, tools, and evidence as SA and MA. Fixed Workflow (FW) executes capabilities in a predefined order, while LLM-Routed (LR) selects a predefined workflow without replanning. On multi-capability questions, SA and MA achieve scores of 3.25 and 3.35, compared with 2.15 for FW and 2.63 for LR. On single-capability questions, the simpler workflows achieve slightly higher mean scores (Table 5). These results indicate that flexible coordination is most useful when a task requires combining several capabilities.

We also evaluated Claude Agent SDK alongside SA and MA using the same Sonnet 5.5 backbone and domain tools. SA achieves an Overall Score of 3.76 and Pass Rate of 0.50, compared with 3.53 and 0.47 for SDK, although SDK has lower latency (Table 6, Appendix C.6). This comparison directly addresses the architectural question and demonstrates that CoSyn’s SA implementation is competitive with a general-purpose agentic framework.

Regarding evaluation scope, to our knowledge, no established external benchmark directly assesses method-specific co-crystal synthesis-condition recommendation together with the supporting literature-analysis and retrieval workflows. Domain specialists therefore developed CoSyn-Bench around practical tasks in this setting, comprising 20 single-capability questions and 10 multi-capability questions requiring coordination. Reporting these groups separately allows us to assess the contribution of coordination specifically on composite tasks.

Given the benchmark’s modest size, we expended the evaluation through independent assessment of 120 responses by two experts. LLM–expert agreement was comparable to inter-expert agreement (Spearman 0.806 and 0.792), with MA receiving the highest mean expert score of 3.63 (Appendix C.7). This agreement strengthens confidence in the system comparisons on CoSyn-Bench. The revised evaluation provides evidence of the utility of coordination for the practical synthesis-assistance workflows represented in the benchmark, while its limited size warrants caution in extending these conclusions to more complex or broader synthesis-design scenarios.

AuthorsOpenReview ↗

Author Response [Part 2/2]

[W3/Q3] Value of condition prediction

We added nearest-neighbor condition transfer (NN-1) and top-3 retrieval aggregation (kNN-3) to directly assess the value of LLM-based adaptation beyond retrieval. On the held-out grinding set, Gemini-2.5-Flash achieves an Overall-Single F1 of 0.49, compared with 0.46 for NN-1, 0.47 for kNN-3, and 0.46 for Random Forest. Its Numerical F1 is slightly higher than the retrieval-only baselines (0.48 compared with 0.46). However, NN-1 performs better on solvent prediction (0.48 compared with 0.41), and Random Forest achieves the highest solvent and categorical scores (Table 3). These comparisons establish retrieval as a strong baseline and show that the additional benefit of LLM-based adaptation is modest and parameter-dependent.

From a synthesis perspective, established literature protocols provide a substantial foundation of experimentally demonstrated methods and conditions. CoSyn addresses the practical challenge of selecting and adapting this evidence to a specific molecular pair when its own protocol is unavailable. Agreement with reported conditions is therefore relevant to evaluating literature-grounded recommendation, but does not establish experimental feasibility. We assess these aspects separately through retrospective prediction and prospective synthesis. Three of four target pairs yielded phase-pure co-crystals after all pair-specific records were excluded from retrieval (Section 4.3), providing initial evidence that the recommendations can serve as actionable starting conditions.

[W4/Q4] Recommendation reliability

We examined both what the 25.3% Full-record Exact Match represents and whether extraction errors affect downstream recommendations. The added hierarchical metrics show that, among matched records, exact agreement reaches 65.2% for chemical composition, including component identities, stoichiometry, and solvents, and 52.5% when quantities are also required. Full-record agreement additionally requires matching all remaining method-specific protocol details (Appendix A.11, Table 12). These results distinguish chemical fidelity from complete-protocol reproduction, while confirming that extraction errors remain.

The error-propagation analysis reported in Appendix B.11 found similar Overall F1 using manually annotated and automatically extracted evidence on 85 held-out grinding queries (0.23 and 0.24, Table 23). This indicates no systematic aggregate degradation in the evaluated setting, although individual recommendations may still be affected.

The prospective experiments provide complementary evidence of practical applicability, but remain limited to four literature-confirmed pairs under slurry crystallization. With all target-pair records excluded from retrieval, CoSyn recommendations yielded three phase-pure co-crystals, while the baseline produced two mixed products and two failures. For the fourth pair, urea–salicylic acid, the components dissolved completely under the recommended slurry conditions, leaving no solid product for recovery. In the baseline experiments, unavailable ethyl acetate was replaced with 1,2-dimethoxyethane in all four trials. Similar relative polarity motivated this substitution, but does not establish experimental equivalence, so we explicitly acknowledge that it limits the comparison (Section 4.3, Appendix B.13).

These findings provide initial evidence for the practical utility of CoSyn’s literature-grounded recommendations, including successful synthesis with conditions generated after excluding the target pairs’ own records from retrieval.

The revised manuscript strengthens the evidence for CoSyn’s contribution as a structured scientific resource and framework for synthesis-condition recommendation, while clarifying when predictive adaptation and agentic coordination add value. We hope these additions address the reviewer’s concerns and support a more favorable assessment of the work.

Revision: Section 1 (L76–L80), Section 3.3 (L254–L256), Table 3 (L277–L285), Section 4 (L297–L299), Section 4.1 (L311–L339), Section 4.2 (L345–L352), Section 4.3 (L374–L385), Sections 5.1–5.2 and Tables 5–6 (L396–L467), Appendix A.11 (L1694–L1707), Appendix B.13.1 (L2324–L2325), Appendix C.3.2–C.3.3 (L2634–L2779), Appendix C.6–C.7 (L3090–L3156).

Thank you for the detailed response and additional experiments. The clarifications have helped me better understand the paper's contributions and experimental settings, and I appreciate the authors' efforts to address my concerns.

However, the methodological novelty of the agentic component remains limited. Although the additional comparisons strengthen the evaluation, the improvements over simple retrieval-based baselines remain modest. Moreover, the agentic evaluation still relies on a small benchmark with only 10 multi-capability questions, while prospective validation covers only four molecular pairs under a single synthesis method.

Therefore, I will maintain my current score, although I now consider the paper closer to the acceptance borderline than in my initial assessment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Author Summary of Revisions

We thank all three reviewers for their thoughtful and constructive feedback. Their comments helped us clarify the scientific contribution and strengthen the supporting evidence.

Contribution and scope

CoSyn advances literature-grounded synthesis assistance beyond molecular-pair screening by structuring experimental evidence and using it to recommend method-specific synthesis conditions. Its contributions include a method-aware extraction pipeline, CoSyn-DB with 20,783 synthesis records across 12 methods, and the formulation and evaluation of synthesis-condition recommendation as a downstream task. Predictive models and agentic architectures are investigated as approaches to this task, with prospective experiments providing initial evidence of practical applicability. We revised the contribution statement to make this scientific focus explicit.

Summary of revisions

We expanded the manuscript with the following analyses and clarifications.

  • Extraction reliability. Added hierarchical record-level metrics distinguishing chemical composition, quantitative composition, and full-record agreement (Appendix A.11).

  • Prediction baselines. Added nearest-neighbor condition transfer and top-3 retrieval aggregation to assess the contribution of LLM prediction beyond retrieval (Table 3).

  • Molecular similarity. Analyzed prediction performance across subsets defined by nearest-neighbor molecular similarity (Appendix B.12).

  • Multiple valid protocols. Added best-reference scoring and quantified variability among published protocols, including cross-publication comparisons (Appendix B.14).

  • Experimental scale. Evaluated sensitivity to retrospective rescaling and clarified the ambiguity of absolute quantities without a specified synthesis scale (Appendix B.15).

  • Controlled agent comparisons. Added Fixed Workflow, LLM-Routed, and Claude Agent SDK baselines with matched domain tools and backbone-controlled comparisons (Tables 5 and 6).

  • Execution efficiency. Expanded comparisons of token usage, LLM calls, and latency across architectures and backbones (Appendix C.6).

  • Independent expert evaluation. Extended human assessment and analyzed agreement with the LLM judge (Appendix C.7).

  • Criterion-level reliability. Added Correctness Pass Rate, Strict Pass Rate, balanced metrics, and criterion-specific failure analysis (Appendix C.8).

  • Prospective validation. Clarified target-pair retrieval exclusions, experimental conditions and outcomes, and the limitations of solvent substitution in the baseline comparison (Section 4.3, Appendix B.13).

We have updated the manuscript and highlighted all revisions in color. Each reviewer response also includes a revision list with specific line references to help locate the corresponding changes.

We hope these revisions address the reviewers’ principal concerns and strengthen the case for CoSyn as a structured scientific resource and a framework for a new synthesis-condition recommendation task, supported by prospective experiments. We would be grateful if the reviewers reconsidered their assessments and if the Area Chairs took the clarified contributions and additional evidence into account.

AuthorsOpenReview ↗

Follow-up on Author Responses and Revisions

Dear Reviewers,

Thank you for your time and thoughtful feedback on our work.

As the discussion period is nearing its end, we would greatly appreciate your thoughts on our responses, additional analyses, and revised manuscript. We have made substantial revisions to address the concerns raised.

We would be grateful to hear whether these additions have addressed your main concerns and if any concerns remain.

Best regards,

Authors of Submission 44351

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.