acceptodds
Under review as a conference paper at ICLR 2027

Large Language Models as Preprocessing Optimisers in Clinical Spectroscopy

Abstract

Clinical spectroscopy has emerged as a powerful tool across biomedical research, with applications spanning from disease diagnosis and screening, to biomarker detection, therapeutic monitoring and intraoperative assessment. However, despite substantial progress, the analysis of spectroscopic data remains a major bottleneck, with current workflows still heavily reliant on trial and error, expert intuition and prior experience. Here, we introduce a multi-agent, large language model (LLM) framework for automated optimisation of preprocessing pipelines for spectral data. Our system combines a memory-derived information set, or dossier, with LLM-based agents that reason over previously evaluated preprocessing pipelines in order to propose, rank, and iteratively refine preprocessing strategies by following a multi-reward rule. Unlike exhaustive, brute-force search over a parameter space, the agent constructs candidates sequentially, enabling more targeted exploration. We benchmark our approach across 18 scenarios on four public clinical datasets. Our agent matches the accuracy of exhaustive grid search while evaluating fewer than 4% as many pipelines, and outperforms sequential stepwise optimisation in 17 of 18 scenarios. Ablation across LLM backbones and dossier configurations further shows that access to optimisation history is a key contributor to performance. These results demonstrate the potential of history-informed agents to enable more automated, adaptive and reproducible preprocessing pipelines in spectroscopy.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
large language models, agents, AI for science, optimization, spectroscopy

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4QkxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents an LLM-based multi-agent approach to automating the optimization of preprocessing pipelines for clinical spectroscopy. The proposed system employs a proposer agent and a reflector agent, supported by a structured memory component termed the dossier, to iteratively explore and refine preprocessing configurations using feedback from previous evaluations. A multi-reward racing mechanism is incorporated to facilitate candidate selection. The framework is evaluated across 18 classification scenarios spanning four publicly available clinical spectroscopy datasets and three LLM backbones. Experimental results suggest that the approach achieves classification performance comparable to exhaustive grid search while requiring substantially fewer pipeline evaluations, and generally outperforms greedy stepwise optimization. Further ablation studies investigate the effects of different dossier components, accumulated optimization history, and LLM model size on overall performance.

Critical strengths

The paper addresses a practical and important problem in clinical spectroscopy, the selection and optimization of preprocessing pipelines. It presents a coherent and well-motivated study, with a clear connection between the practical problem, the proposed method, and its evaluation. I particularly appreciate that the authors make a serious effort to formalize preprocessing optimization as a well-defined search problem, rather than simply using an LLM as an unconstrained natural-language assistant. I appreciate the practical relevance of this work.

The experimental evaluation is also relatively comprehensive. The authors evaluate the framework across 18 scenarios involving different spectroscopic datasets and classifiers, demonstrating competitive classification performance while evaluating substantially fewer pipelines than exhaustive grid search. They further investigate the optimization process through search trajectories, LLM backbones and detailed ablation studies. the paper provides a convincing demonstration of the practical value of history-guided LLM optimization for spectral preprocessing.

Critical weaknesses

Insufficient comparison with established optimisation methods. The paper demonstrates that LLM-guided optimisation achieves competitive performance with substantially fewer evaluations than exhaustive grid search. However, the absence of budget-matched comparisons with established optimisation methods makes it difficult to assess the actual contribution of LLM-guided reasoning. Random search is a well-established baseline for hyperparameter optimisation (Bergstra & Bengio, 2012). More importantly, genetic algorithms have already been applied to Raman preprocessing optimisation (Bocklitz et al., 2011), and Brunel et al. (2021) explicitly compared genetic algorithms against random search, ant colony optimisation, and TPE in vibrational spectroscopy. Bayesian optimisation has also been applied to automated spectral preprocessing (Babatunde et al., 2025). Therefore, outperforming grid search in evaluation efficiency alone is insufficient to establish an advantage over existing optimisation strategies. Comparisons with budget-matched random search and at least one adaptive optimiser would substantially strengthen the paper's central claim.

Questions for authors
  1. Could the authors compare the proposed approach with budget-matched random search and a representative adaptive optimiser (e.g., TPE or GA) to better establish the benefits of LLM-guided optimisation?

  2. Appendix E (Table 13) reports pipelines violating the predefined stage constraints. Could the authors clarify how these were admitted and whether they affect the main conclusions?

  3. Table 9 suggests that removing certain dossier components can maintain or improve performance. Could the authors clarify the necessity of these components, particularly the reflector agent?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 4Qkx (1/4)

We thank the reviewer for their constructive feedback and encouraging assessment of our work. We particularly appreciate their recognition of its practical relevance and the breadth of our experimental evaluation. We address the three concerns below.

1. Comparison with established optimisers

Comparisons with budget-matched random search and at least one adaptive optimiser would substantially strengthen the paper's central claim.

We have added four independent baselines: random search, Tree-structured Parzen Estimator (TPE) and genetic algorithm (GA), as previously studied by Brunel et al. (2021), and Gaussian process-based Bayesian optimisation (GP-BO), as used for spectral preprocessing by Babatunde et al. (2025).

We evaluated each of the four baselines across the same 18 scenarios. To ensure comparability with our agentic methods, they use the same data splits, preprocessing method catalogue, parameter bounds and a budget of 120 pipeline evaluations per scenario. Like our agentic optimisers, these additional baselines explore continuous parameter ranges within the specified bounds, allowing direct comparisons with agentic search. We assessed each candidate using the same internal cross-validation procedure.

The results are presented in the updated Table 1, evaluated on held-out test data (see point 1 in our response to Reviewer wsKD).

Updated Table 1 — Out-of-sample AUC

MethodDRS skinFTIR plasma (+)FTIR plasma (−)FTIR serum (+)FTIR serum (−)SERS plasma
Reference methods
Raw data0.6750.6140.6810.6760.6990.717
Grid search0.7390.7190.7720.6750.7590.800
Stepwise0.7290.7530.7530.6840.8270.850
Additional baselines
Random search0.7650.7770.7570.6890.8020.840
TPE (Brunel et al., 2021)0.7870.7460.7570.7180.8030.837
GA (Brunel et al., 2021)0.7840.7480.7370.7030.7950.842
GP-BO (Babatunde et al., 2025)0.7720.7480.7500.7040.7990.818
Agentic (ours)
DeepSeek-R1 (8B)0.7930.7410.7730.6990.7790.917
Llama 3.1 (8B)0.8380.7220.7850.6960.8150.867
Llama 3.1 (70B)0.8400.7430.7230.7430.7530.867

Across the six datasets, our agentic approach achieved the highest mean held-out AUC in four settings and outperformed the best of the three established optimisers (TPE, GA and GP-BO) in five. We have expanded our method comparisons in Section 5.2. and added full information about the baselines in Section 4. We provide the full in-sample and out-of-sample results for individual scenarios in the new Tables 5–8 (Appendix C).

We are excited to see how future advances in LLM reasoning and agent orchestration could further improve these results. We have better highlighted this at the end of the Conclusion section of our updated paper.

AuthorsOpenReview ↗

Response to Reviewer 4Qkx (2/4)

2. Stage-constraint violations

Appendix E (Table 13) reports pipelines violating the predefined stage constraints. Could the authors clarify how these were admitted and whether they affect the main conclusions?

In our framework, we deliberately did not enforce hard constraints on method placement across preprocessing stages. While method validity and parameter bounds were checked, cross-stage proposals were allowed so that we could assess how reliably agents followed the stage definitions provided in their instructions. We agree that this design choice was not sufficiently clear and have clarified it in Section 4.

Following the revised evaluation protocol, we also repeated the stage-compliance analysis using the newly generated optimisation runs and inner vs. outer evaluations. The updated results are reported in Table 17, Appendix E (previously Table 13).

Updated Table 17 — Stage-compliance summary

MetricDeepSeek-R1 (8B)Llama 3.1 (8B)Llama 3.1 (70B)
Admitted pipelines4,1683,7164,049
With a cross-stage method66510
Share1.6%1.4%0.0%
Scenario-splits affected18/3617/360/36
occurrences, by family and the stage it occupied
Whittaker in baseline (registered: denoise)39——
SNV in baseline (registered: normalise)1412—
Whittaker in despike (registered: denoise)715—
ASLS in denoise (registered: baseline)312—
IASLS in denoise (registered: baseline)—9—
Vector in baseline (registered: normalise)52—
iModPoly in denoise (registered: baseline)—2—
SavGol in baseline (registered: denoise)1——
Selected pipelines affected1/360/360/36
Paired Δ in CV AUC (training)−0.026−0.005—
Paired Δ in held-out AUC−0.047−0.011—

Cross-stage proposals accounted for only 1.6% and 1.4% of admitted pipelines for DeepSeek-R1:8B and Llama 3.1:8B, respectively, and none for Llama 3.1:70B. More importantly, only one of the 108 final pipeline selections contains a method outside its registered stage. This occurs for DeepSeek-R1:8B on ATR-FTIR serum without chemotherapy, with a training CV AUC of 0.828 and held-out AUC of 0.819. We report this pipeline explicitly in the new Table 18 in Appendix E, as shown below.

Modeldeepseek-r1:8b
ScenarioATR-FTIR serum, no chemotherapy
Pipeline
- despikeWhitakerHayes (kernel_size=7, threshold=15)
- denoiseASLS‡ (diff_order=2, lam=1e5)
- baselineidentity
- normaliseSNV
CV AUC / held-out AUC0.828 / 0.819

‡ ASLS is a baseline-correction method, but the agent placed it in the denoise stage.

The paired comparisons in Table 17 show that cross-stage configurations were associated with lower AUC, both in cross-validation (−0.026 for DeepSeek, −0.005 for Llama 8B) and on held-out data (−0.047 and −0.011, respectively). Thus, these violations did not appear to improve performance. All held-out comparisons in Table 17 were computed post-hoc and were not used during optimisation or pipeline selection.

Several violations involved similarly named or related methods. For example, 'WhitakerHayes' is used for despiking and 'Whittaker' for denoising, with 22 instances of the latter being used for despiking. Other cases involved Whittaker smoothing and ASLS/IASLS baseline correction, which share similar mathematical foundations and can indeed be adapted for both preprocessing stages in practice.

Again, we emphasise that allowing cross-stage proposals was intentional, as we wanted to study how well agents follow the specified stage definitions. Strict constraints could be easily implemented, just as we already enforce parameter bounds. We agree that rerunning the optimisation with these constraints would provide a useful additional control, but the computational cost and limited rebuttal period did not allow us to complete this experiment. We hope that the expanded analysis, particularly the finding that only 1 of 108 selected pipelines was affected, helps address this concern.

Note: We observed fewer cross-stage violations in the revised experiments compared with the original results. We confirm that our pipeline generation and stage-validation logic were not changed during the revision.

AuthorsOpenReview ↗

Response to Reviewer 4Qkx (3/4)

3. Dossier components and Reflector Agent

Table 9 suggests that removing certain dossier components can maintain or improve performance. Could the authors clarify the necessity of these components, particularly the reflector agent?

As described in Section 3.1, our dossier builds on the meta-prompt used in Optimization by PROmpting (OPRO; Yang et al., 2024), which uses previously evaluated candidates and their scores to guide subsequent proposals. We extend this idea by organising successful and unsuccessful pipelines, performance comparisons, diagnostic feedback and task constraints into a structured record. The aim is to help agents identify promising configurations, learn from previous evaluations and avoid repeating unsuccessful choices.

We agree that not all dossier components contribute equally. As shown in Table 13 (previously Table 9), removing §7 diagnostics or the self-model (§6 and invariants) leaves mean proposal AUC essentially unchanged (0.650 vs. 0.649 for the full dossier). In contrast, removing §2 top-performing history reduces mean AUC by 0.020 ± 0.010, while removing all optimisation history leads to a larger reduction of 0.027 ± 0.012. These results highlight the importance of historical feedback, while suggesting that some supporting components offer limited additional benefit. We have expanded the rationale in Section 3.1 and the discussion of these findings in Section 5.4.

The use of a reflector agent was motivated by Nie et al. (2024), who used a separate LLM call to turn numerical evaluation results into natural-language feedback for subsequent proposals. They showed that adding an agent that provides directional feedback makes the agentic framework more robust and efficient for optimisation problems. Similarly, our reflector summarises predicted and measured AUC and sensitivity after each batch, providing feedback to the proposer through the updated dossier. We have clarified this rationale in Section 3.2.

To assess its contribution, we conducted an additional ablation on 12 optimisation runs (3 models, 2 scenarios and 2 seeds), disabling the reflector while keeping all other components and evaluation settings fixed. The results of these experiments are presented in Table 13 (previously Table 9) in Appendix E of the updated paper.

Table 13 — Reflector ablation

ConfigurationMean proposal AUCBest AUCBounds complianceContract compliance
Full dossier0.6490.72188.7%100.0%
Without reflector0.6400.72393.5%100.0%

Removing the reflector reduced the mean proposal AUC by 0.009 (±0.005), with lower performance in 10 of the 12 optimisation runs. However, the best AUC remained similar (0.721 vs. 0.723), and bounds compliance improved without the reflector. These results suggest that the reflector helps improve average proposal quality, but is not essential for finding a competitive pipeline. We have added this analysis and discussed its implications in Section 5.4.

The modest contribution of the reflector also raises the question of whether simpler agent architectures could achieve comparable performance at lower computational cost. Exploring this trade-off further would be an interesting direction for future work.

AuthorsOpenReview ↗

Response to Reviewer 4Qkx (4/4)

Conclusion

We hope that the new budget-matched baselines, stage-compliance analysis, expanded discussions of dossier components and reflector ablations address the reviewer's concerns. We believe these additions substantially strengthen the paper and would appreciate a reconsideration of the rating in light of the revised results.

Reviewer_i5LZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Clinical spectroscopy has emerged as a powerful tool for the biomedical domain. However, the analysis of spectroscopic data remains a major bottleneck. To this end, the paper proposes a multi-agent, large language model (LLM) framework for automated optimisation of preprocessing pipelines for spectral data. Experiments across 18 scenarios on four public clinical datasets demonstrate that the proposed agentic method matches the accuracy of exhaustive grid search while evaluating fewer than 4% as many pipelines, and outperforms sequential stepwise optimisation in 17 of 18 scenarios.

Critical strengths
  • The studied problem is practical and meaningful in the real world
  • The writing of the paper is, in general, good.
Critical weaknesses
  • Missing necessary baselines. The paper only compares its agentic method with grid search and greedy stepwise optimization. However, some other traditional search methods, such as random search, Bayesian optimization, and genetic algorithms, are not included.
  • Although the search efficiency (number of evaluated pipelines) is improved with the agentic method, Table 5 shows that the LLM-based proposal stage accounts for more than 90% of the total runtime. This raises concerns about the practical computational efficiency of the proposed framework.
  • The ablation studies on the orchestration of the agents remain limited. For example, why is the Reflector Agent required?
Questions for authors

See the Weakness part.

3 comments
AuthorsOpenReview ↗

Response to Reviewer i5LZ (1/3)

We thank the reviewer for recognising the significance and practical relevance of our work, and for their constructive feedback. We address the three concerns below.

1. Additional optimisation baselines

Missing necessary baselines. The paper only compares its agentic method with grid search and greedy stepwise optimization. However, some other traditional search methods, such as random search, Bayesian optimization, and genetic algorithms, are not included.

We have added four independent baselines: random search, Tree-structured Parzen Estimator (TPE) and genetic algorithm (GA), as previously studied by Brunel et al. (2021), and Gaussian process-based Bayesian optimisation (GP-BO), as used for spectral preprocessing by Babatunde et al. (2025).

We evaluated each of the four baselines across the same 18 scenarios. To ensure comparability with our agentic methods, they use the same data splits, preprocessing method catalogue, parameter bounds and a budget of 120 pipeline evaluations per scenario. Like our agentic optimisers, these additional baselines explore continuous parameter ranges within the specified bounds, allowing direct comparisons with agentic search. We assessed each candidate using the same internal cross-validation procedure.

The results are presented in the updated Table 1, evaluated on held-out test data (see point 1 in our response to Reviewer wsKD).

Updated Table 1 — Out-of-sample AUC

MethodDRS skinFTIR plasma (+)FTIR plasma (−)FTIR serum (+)FTIR serum (−)SERS plasma
Reference methods
Raw data0.6750.6140.6810.6760.6990.717
Grid search0.7390.7190.7720.6750.7590.800
Stepwise0.7290.7530.7530.6840.8270.850
Additional baselines
Random search0.7650.7770.7570.6890.8020.840
TPE (Brunel et al., 2021)0.7870.7460.7570.7180.8030.837
GA (Brunel et al., 2021)0.7840.7480.7370.7030.7950.842
GP-BO (Babatunde et al., 2025)0.7720.7480.7500.7040.7990.818
Agentic (ours)
DeepSeek-R1 (8B)0.7930.7410.7730.6990.7790.917
Llama 3.1 (8B)0.8380.7220.7850.6960.8150.867
Llama 3.1 (70B)0.8400.7430.7230.7430.7530.867

Across the six datasets, our agentic approach achieved the highest mean held-out AUC in four settings and outperformed the best of the three established optimisers (TPE, GA and GP-BO) in five. We have expanded our method comparisons in Section 5.2. and added full information about the baselines in Section 4. We provide the full in-sample and out-of-sample results for individual scenarios in the new Tables 5–8 (Appendix C).

We are excited to see how future advances in LLM reasoning and agent orchestration could further improve these results. We have better highlighted this at the end of the Conclusion section of our updated paper.

AuthorsOpenReview ↗

Response to Reviewer i5LZ (2/3)

2. Computational efficiency

Although the search efficiency (number of evaluated pipelines) is improved with the agentic method, Table 5 shows that the LLM-based proposal stage accounts for more than 90% of the total runtime. This raises concerns about the practical computational efficiency of the proposed framework.

In Table 5 (now Table 9, Appendix C), what we previously termed the proposal stage includes candidate generation, reflection, dossier construction and updates, and agent orchestration. As these activities account for most of the agentic runtime, we have renamed this component LLM & orchestration for clarity.

To assess practical computational efficiency, we focus on total wall-clock time across methods. We have expanded Table 9 to include the new optimisation baselines and average runtime per scenario, rather than only the totals across all 18 scenarios. Runtime measurements were recorded for a single repeat.

Updated Table 9 — Computational cost

MethodEvaluationsTotal (h)LLM & orchestration (h)Per scenario
Raw18<1 min—<1 min
Grid search43,56070.00—3.9 h
Random search2,1601.74—5.8 min
Stepwise6350.50—1.7 min
TPE2,1602.04—6.8 min
GA2,1601.86—6.2 min
GP-BO2,1601.98—6.6 min
DeepSeek-R1 (8B)2,1607.536.1525.1 min
Llama 3.1 (8B)2,1609.517.9431.7 min
Llama 3.1 (70B)2,16011.9110.6039.7 min

Our agentic optimisation took approximately 25–40 minutes per scenario on average, compared with 6–7 minutes for budget-matched conventional optimisers and 3.9 hours for exhaustive grid search. While LLM inference adds overhead compared with conventional optimisers, the agentic approach remains substantially faster than exhaustive search. We believe this trade-off may be worthwhile in many clinical applications, where even modest improvements in classification performance can justify additional optimisation time. We have updated our discussions in Section 5.2.

Note: During the rebuttal, we observed a reduction in DeepSeek runtime compared with our original results. After investigating, we believe this likely reflects an Ollama update that improved inference efficiency with thinking disabled. This also suggests that further improvements in LLM inference, agent orchestration and tool use could help narrow the runtime gap with conventional optimisers. We have better highlighted this at the end of the Conclusion section of our updated paper.

3. Reflector Agent ablation

The ablation studies on the orchestration of the agents remain limited. For example, why is the Reflector Agent required?

The use of a reflector agent was motivated by Nie et al. (2024), who used a separate LLM call to turn numerical evaluation results into natural-language feedback for subsequent proposals. They showed that adding an agent that provides directional feedback makes the agentic framework more robust and efficient for optimisation problems. Similarly, our reflector summarises predicted and measured AUC and sensitivity after each batch, providing feedback to the proposer through the updated dossier. We have clarified this rationale in Section 3.2.

To assess its contribution, we conducted an additional ablation on 12 optimisation runs (3 models, 2 scenarios and 2 seeds), disabling the reflector while keeping all other components and evaluation settings fixed. The results of these experiments are presented in Table 13 (previously Table 9) in Appendix E of the updated paper.

Table 13 — Reflector ablation

ConfigurationMean proposal AUCBest AUCBounds complianceContract compliance
Full dossier0.6490.72188.7%100.0%
Without reflector0.6400.72393.5%100.0%

Removing the reflector reduced the mean proposal AUC by 0.009 (±0.005), with lower performance in 10 of the 12 optimisation runs. However, the best AUC remained similar (0.721 vs. 0.723), and bounds compliance improved without the reflector. These results suggest that the reflector helps improve average proposal quality, but is not essential for finding a competitive pipeline. We have added this analysis and discussed its implications in Section 5.4.

The modest contribution of the reflector also raises the question of whether simpler agent architectures could achieve comparable performance at lower computational cost. Exploring this trade-off further would be an interesting direction for future work.

AuthorsOpenReview ↗

Response to Reviewer i5LZ (3/3)

Conclusion

We hope that the new budget-matched baselines, computational efficiency analyses and reflector ablations address the reviewer's concerns. We believe these additions substantially strengthen the paper and would appreciate a reconsideration of the rating in light of the revised results.

Reviewer_wsKDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an LLM-based multi-agent framework for optimising clinical spectral preprocessing. A proposer generates candidate pipelines from a structured dossier of previous evaluations, while a reflector produces short invariants to guide subsequent proposals. Candidates receive cross-validated AUC and sensitivity rewards, and a multi-reward racing rule controls retirement. Across 18 scenarios from four public datasets, DeepSeek-R1:8B achieves a mean top-1 cross-validated AUC of 0.816, compared with 0.804 for exhaustive grid search and 0.775 for greedy stepwise optimisation. Additional analyses examine optimisation trajectories, prediction calibration, model size, dossier components, and pipeline validity.

Critical strengths
  1. The application is practically relevant, and the results suggest promising evaluation efficiency. Spectral preprocessing involves interacting stages and a combinatorial configuration space. The framework finds competitive cross-validated scores while evaluating substantially fewer pipelines than exhaustive grid search. DeepSeek-R1:8B outperforms greedy stepwise optimisation in 17/18 scenarios. This is encouraging evidence for the approach as a preprocessing search tool, although the efficiency advantage is principally relative to exhaustive search.

  2. The empirical study covers several relevant settings and includes informative diagnostic analyses. The experiments span three spectroscopic modalities and three classifier families. Search trajectories, dossier ablations, and audits of proposal validity help expose both useful behaviour and failure modes. The observation that the larger Llama model has better prediction calibration without better optimisation performance is informative. The ablations also provide preliminary evidence that top-performing pipeline history contributes to proposal quality within the full configuration.

Critical weaknesses
  1. Final performance is not clearly evaluated independently of search feedback. Section 2 defines fixed, repeated cross-validation as the optimisation reward, while Table 1 reports the best cross-validated scores found during search. No independent outer evaluation is described. Patient-grouped splitting and fold-specific scaling are appropriate, but do not resolve selection-induced optimism from repeatedly using validation feedback. This does not establish that the relative advantage over grid search is biased; it means that the generalisation performance of the selected pipelines remains uncertain. Table 3 also includes label-dependent dossier diagnostics, whose data access should be specified within an independent evaluation protocol.

  2. The comparisons do not sufficiently isolate the advantage of LLM-guided search. There is no independent, budget-matched random search or established adaptive optimisation baseline. Table 12’s fallback draws occur within agentic campaigns and cannot replace complete baseline searches. Moreover, Section 2 explicitly permits real-valued parameter intervals, whereas grid search uses discrete values. Better scores may therefore reflect access to configurations outside the grid as well as differences in search strategy. The stage constraints introduce a further comparability problem. Table 13 reports admitted pipelines containing methods outside their registered stages: 16.4% across 11/18 scenarios for Llama3.1:8B, 2.4% in 1/18 for DeepSeek-R1:8B, and none for Llama3.1:70B. This already affects at least one reported top-1 pipeline: Table 6 places IASLS in the denoising stage for Llama3.1:8B on SERS plasma/LogReg, while Table 13 identifies this placement as a violation. The impact on aggregate results and the headline DeepSeek advantage remains unresolved. Controlled comparisons require consistent search spaces and strictly enforced stage constraints.

Questions for authors
  1. Are Table 1’s scores the same cross-validation rewards used during search? Is there an independent outer evaluation? Please specify how pipeline selection and label-dependent dossier diagnostics are confined to the outer training data.

  2. Why do the cross-stage configurations in Table 13 pass admission checks? Please report the compliance of the final selected pipelines and results from strictly stage-constrained searches. Could you also provide budget-matched random search and an established adaptive optimiser operating over the same configuration space?

4 comments
AuthorsOpenReview ↗

Response to Reviewer wsKD (1/4)

We thank the reviewer for their careful assessment and constructive feedback. We appreciate their recognition of the practical relevance of our approach and the breadth of our experimental analyses. We address the concerns below.

1. Independent evaluation and generalisation

Final performance is not clearly evaluated independently of search feedback.

We agree that the cross-validation scores used during optimisation do not provide an independent estimate of generalisation performance. We therefore reran all methods from scratch across all 18 scenarios, using an 80/20 train/test split with two repeats.

For each split, optimisation and pipeline selection were performed entirely on the 80% training partition, using internal cross-validation and the procedure in Algorithm 1. For each method and dataset, the classifier with the highest inner-cross-validation score was selected and subsequently evaluated on the held-out 20% test set. Table 1 reports these out-of-sample AUCs, with no test data used for pipeline or classifier selection. We have clarified this protocol in Section 4.

Updated Table 1 — Out-of-sample AUC

MethodDRS skinFTIR plasma (+)FTIR plasma (−)FTIR serum (+)FTIR serum (−)SERS plasma
Reference methods
Raw data0.6750.6140.6810.6760.6990.717
Grid search0.7390.7190.7720.6750.7590.800
Stepwise0.7290.7530.7530.6840.8270.850
Additional baselines
Random search0.7650.7770.7570.6890.8020.840
TPE (Brunel et al., 2021)0.7870.7460.7570.7180.8030.837
GA (Brunel et al., 2021)0.7840.7480.7370.7030.7950.842
GP-BO (Babatunde et al., 2025)0.7720.7480.7500.7040.7990.818
Agentic (ours)
DeepSeek-R1 (8B)0.7930.7410.7730.6990.7790.917
Llama 3.1 (8B)0.8380.7220.7850.6960.8150.867
Llama 3.1 (70B)0.8400.7430.7230.7430.7530.867

Under this independent evaluation, our agentic optimisation achieves the highest out-of-sample AUC in four of the six dataset settings, with random search and stepwise optimisation leading in one setting each. We provide the full in-sample and out-of-sample results for individual scenarios in the new Tables 5–8 (Appendix C) and have revised Section 5.2 accordingly. We are excited to see how future advances in LLM reasoning and agent orchestration could further improve these results. We have better highlighted this at the end of the Conclusion section of our updated paper.

Note: Due to the limited rebuttal period and the computational cost of rerunning all optimisation methods across 18 scenarios, we were able to complete two repeats.

2. Label-dependent dossier diagnostics

Table 3 also includes label-dependent dossier diagnostics, whose data access should be specified within an independent evaluation protocol.

The revised evaluation protocol also addresses this concern. During optimisation, all label-dependent diagnostics are computed exclusively from the 80% training partition, without access to the held-out test data or labels. We have clarified this in Section 3.1. and in the caption of Table 3.

AuthorsOpenReview ↗

Response to Reviewer wsKD (2/4)

3. Baselines and search-space comparability

The comparisons do not sufficiently isolate the advantage of LLM-guided search. There is no independent, budget-matched random search or established adaptive optimisation baseline.

We have added four independent baselines: random search, Tree-structured Parzen Estimator (TPE) and genetic algorithm (GA), as previously studied by Brunel et al. (2021), and Gaussian process-based Bayesian optimisation (GP-BO), as used for spectral preprocessing by Babatunde et al. (2025).

We evaluated each of the four baselines across the same 18 scenarios. To ensure comparability with our agentic methods, they use the same data splits, preprocessing method catalogue, parameter bounds and a budget of 120 pipeline evaluations per scenario. Like our agentic optimisers, these additional baselines explore continuous parameter ranges within the specified bounds, allowing direct comparisons with agentic search. We assessed each candidate using the same internal cross-validation procedure.

The results are presented in the updated Table 1, shown in our response to point 1 above, with per-scenario results in Tables 5–8 of the updated paper. Our agentic approach achieved the highest mean held-out AUC in four settings and outperformed the best of the three established optimisers (TPE, GA and GP-BO) in five. We have expanded our comparisons in Section 5.2. and added full information about the baselines in Section 4.

AuthorsOpenReview ↗

Response to Reviewer wsKD (3/4)

4. Stage-constraint violations

Why do the cross-stage configurations in Table 13 pass admission checks?

In our framework, we deliberately did not enforce hard constraints on method placement across preprocessing stages. While method validity and parameter bounds were checked, cross-stage proposals were allowed so that we could assess how reliably agents followed the stage definitions provided in their instructions. We agree that this design choice was not sufficiently clear and have clarified it in Section 4.

Following the revised evaluation protocol, we also repeated the stage-compliance analysis using the newly generated optimisation runs and inner vs. outer evaluations. The updated results are reported in Table 17, Appendix E (previously Table 13).

Updated Table 17 — Stage-compliance summary

MetricDeepSeek-R1 (8B)Llama 3.1 (8B)Llama 3.1 (70B)
Admitted pipelines4,1683,7164,049
With a cross-stage method66510
Share1.6%1.4%0.0%
Scenario-splits affected18/3617/360/36
occurrences, by family and the stage it occupied
Whittaker in baseline (registered: denoise)39——
SNV in baseline (registered: normalise)1412—
Whittaker in despike (registered: denoise)715—
ASLS in denoise (registered: baseline)312—
IASLS in denoise (registered: baseline)—9—
Vector in baseline (registered: normalise)52—
iModPoly in denoise (registered: baseline)—2—
SavGol in baseline (registered: denoise)1——
Selected pipelines affected1/360/360/36
Paired Δ in CV AUC (training)−0.026−0.005—
Paired Δ in held-out AUC−0.047−0.011—

Cross-stage proposals accounted for only 1.6% and 1.4% of admitted pipelines for DeepSeek-R1:8B and Llama 3.1:8B, respectively, and none for Llama 3.1:70B. More importantly, only one of the 108 final pipeline selections contains a method outside its registered stage. This occurs for DeepSeek-R1:8B on ATR-FTIR serum without chemotherapy, with a training CV AUC of 0.828 and held-out AUC of 0.819. We report this pipeline explicitly in the new Table 18 in Appendix E, as shown below.

Modeldeepseek-r1:8b
ScenarioATR-FTIR serum, no chemotherapy
Pipeline
- despikeWhitakerHayes (kernel_size=7, threshold=15)
- denoiseASLS‡ (diff_order=2, lam=1e5)
- baselineidentity
- normaliseSNV
CV AUC / held-out AUC0.828 / 0.819

‡ ASLS is a baseline-correction method, but the agent placed it in the denoise stage.

The paired comparisons in Table 17 show that cross-stage configurations were associated with lower AUC, both in cross-validation (−0.026 for DeepSeek, −0.005 for Llama 8B) and on held-out data (−0.047 and −0.011, respectively). Thus, these violations did not appear to improve performance. All held-out comparisons in Table 17 were computed post-hoc and were not used during optimisation or pipeline selection.

Several violations involved similarly named or related methods. For example, 'WhitakerHayes' is used for despiking and 'Whittaker' for denoising, with 22 instances of the latter being used for despiking. Other cases involved Whittaker smoothing and ASLS/IASLS baseline correction, which share similar mathematical foundations and can indeed be adapted for both preprocessing stages in practice.

Again, we emphasise that allowing cross-stage proposals was intentional, as we wanted to study how well agents follow the specified stage definitions. Strict constraints could be easily implemented, just as we already enforce parameter bounds. We agree that rerunning the optimisation with these constraints would provide a useful additional control, but the computational cost and limited rebuttal period did not allow us to complete this experiment. We hope that the expanded analysis, particularly the finding that only 1 of 108 selected pipelines was affected, helps address this concern.

Note: We observed fewer cross-stage violations in the revised experiments compared with the original results. We confirm that our pipeline generation and stage-validation logic were not changed during the revision.

AuthorsOpenReview ↗

Response to Reviewer wsKD (4/4)

Conclusion

We hope that the independent evaluation, additional budget-matched baselines and expanded stage-compliance analysis address the reviewer's concerns. We believe these revisions substantially strengthen the experimental evidence and clarify the contributions of the paper, and we would appreciate a reconsideration of the rating in light of the revised results.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision summary

We thank all three reviewers for their careful assessment and constructive feedback. We appreciate their recognition of the significance and practical relevance of our work, and the breadth of our experimental evaluation.

In response to their comments, we have revised the paper and made the following changes or additions:

  1. Independent evaluation: We reran all methods across 18 scenarios using two repeated 80/20 train/test splits, with optimisation restricted to the training data and final performance measured on held-out data. The results are presented in the updated Table 1 and new Tables 5–8 (Appendix C).

  2. Additional optimisation baselines: We added four established methods (random search, TPE, GA and GP-BO), evaluated under matched budgets and parameter bounds. Across the six datasets, our agentic approach achieved the highest mean held-out AUC in four settings and outperformed the best of the three established optimisers (TPE, GA and GP-BO) in five (Tables 1 and 5–8).

  3. Computational efficiency: We expanded the runtime comparison to include the four new baselines and reported per-scenario running times, providing a clearer picture of the computational trade-offs (updated Table 9).

  4. Stage-constraint analysis: We repeated the stage-compliance analysis and examined the impact of cross-stage configurations on performance and final pipeline selection (updated Table 17 and new Table 18).

  5. Reflector Agent ablation: We conducted an additional ablation to assess the Reflector Agent's contribution to proposal quality and optimisation performance (updated Table 13).

These additions are discussed in detail in the revised manuscript and our responses to each reviewer. We believe they substantially strengthen the experimental evidence and provide a clearer picture of the benefits and limitations of our approach, and we would, therefore, appreciate a reassessment of the provided ratings in light of the new results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.