acceptodds
Withdrawn from ICLR 2027

One Agent, Many Stains: Agentic Ensemble Design for Multi-Stain Computational Pathology

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pathological diagnosis routinely relies on multiple stains of the same tissue, including H&E and special or immunohistochemical stains, and multi-stain slide sets are increasingly available in digitized archives. Yet computational pathology models typically read a single stain or fuse stains using a single fixed architecture applied to every task, even though sections of different stains are unregistered and frequently missing, and labeled cohorts are small. We present ARISE (Agentic Rule Evolution for Integrating Stain-specific Experts), a simple and effective alternative: one well-trained multiple-instance model per stain is kept fixed, and the rule that integrates their predictions is designed separately for each task. ARISE builds a pool of candidate rules that abstain when a stain is missing and scores each candidate by its margin over the strongest conventional method. An LLM agent evolves the pool, the objective, and the search, while a numerical inner loop fits the weights. The output is a frozen program, a weighted list of pool members, that runs without an LLM or a GPU. We evaluate ARISE on four cohorts covering three organs (kidney, head and neck, lymphoma), panels of four to eight stains, both classification and time-to-event prediction, and one external site. The comparison methods, run under one protocol, include single-stain models, conventional ensembles, and published multi-stain fusion architectures. ARISE outperforms the strongest conventional method on every task in every cohort. On kidney biopsy lesion scoring, the margin holds in the internal cohort and at the external site (mean AUC and , both ). Applied without modification, the same procedure improves head-and-neck survival and recurrence prediction (AUC and ) and lymphoma survival prediction (C-index ). These results suggest that multi-stain integration can be treated as a task-specific design problem rather than a fixed architectural choice. ARISE provides a common procedure for constructing such integration programs across organs, stain panels, and prediction tasks.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
computational pathology, multi-stain histopathology, LLM agents, ensemble learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SiGXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a framework for combining predictions from multiple histopathological stains (H&E, IHC) using task-specific ensemble rules designed by an LLM agent. One base model is trained per stain using virchow2 tile features with an ABMIL model and then frozen. An LLM agent is then tasked with finding the best classifier for a given task. The methodology used by the agent is to propose a set of rules starting with a standard set, apply an optimizer to fit their combination weights and finally analyze the results to edit the set of rules and propose new ones created from learned combinations and cross-task members. The agent iterates over until it decides to stop. The method is evaluated on 4 cohorts (3 organs) both internal and external and shows an edge over fusion methods and learned ensembles.

Critical strengths
  • integrating information from multiple stains is an important problem in computational pathology. The proposed prediction-level fusion approach avoids the need for precise spatial registration between stained sections and accommodates incomplete stain panels, addressing two practical difficulties with multi-stain data.

  • separating stain-specific feature extraction and prediction from the integration procedure is appealing. Existing experts can be reused, new stains can be introduced without redesigning the entire architecture, and the final integration rule can run without an LLM. This is a potentially useful alternative to complex end-to-end fusion architectures, particularly when labeled datasets are small.

  • the evaluation covers three organs, multiple stain panels, classification and survival tasks, and an external validation cohort. The comparisons include conventional prediction-level ensembles, learned stacking approaches, and multi-stain architectures such as UNICORN and PathMoE.

  • the paper study the contribution of the candidate pool, agent-generated extensions, and search strategy and acknowledge that conventional greedy ensemble selection performs nearly as well as the agent-designed search when given the same candidate pool. This provides useful insight into where the improvements originate.

  • the evaluation separately examines patients with complete and incomplete stain panels. This helps establish that the improvements are not exclusively attributable to handling missing stains, except in the lymphoma cohort.

Critical weaknesses
  • i find that the contribution of the LLM agent is not sufficiently established. There is no ablation that replaces the LLM with other non-agentic techniques (Random Forests, SVM). Because the proposed method (the agentic part) is general (not tied to pathology) it should be validated more broadly against previously published methods such as CAAFE, FeatLLM, RuleFit, TPOT, RLIE. Also, for the same reason, 4 pathology cohorts is not sufficient to establish superiority of the approach over SOTA and established tabular benchmarks should be used.

  • the ablation is only done on 2 cohorts (both kidney), so it is hard to know if it generalizes to other organs.

  • hyperparameter tuning and optimization budgets for the baselines UNICORN and PathMoE are nor reported.

  • a concrete example is needed. The main contribution is the agentic iterative rules selection and generation but there are no example to illustrate the process (initial rules, diagnostic feedback, agent-proposed modifications, intermediate objective values, structure and weights of the final program). This makes it difficult to understand what the LLM actually invents and whether its modifications go beyond established ensemble techniques. If space is limited, it should be provided in the appendix.

  • Results are averaged over five splits without reporting standard deviations. Mean ± standard deviation, particularly for paired performance differences (figure 2), would help establish stability across splits and statistical significance.

  • On what data exactly is each stain virchow2 MIL model trained ? Is it with the same 4 datasets ? Is it with the same splits as for training the final classifier program ? I could not find this critical info.

  • How is the virchow prediction integrated over the tiles of the WSI slide ?

  • What is the complexity of training the final program with the agent ? specify number of tokens, total time.

  • The learned combiners are fitted exclusively on validation predictions rather than using out-of-fold predictions from training data. This is potentially data-inefficient, especially for small cohorts, so the authors should explain the choice. Also, combiners evaluated on one validation block are trained on other validation blocks. It should be clarified that these blocks are patient-stratified.

  • The manuscript claims that only the kidney cohorts provide multiple validation blocks, despite reporting five splits for all cohorts.

  • clarification needed: the authors describe five random 60/20/20 train/validation/evaluation splits but state that a single evolved integration program is shared across all splits. If the program is optimized jointly using the validation predictions and labels from these splits, patients used for validation in one split may also appear in the evaluation set of another. This would compromise evaluation independence. The authors must clarify the splitting protocol and demonstrate that evaluation patients have not influenced program selection.

  • The method uses within-cohort rank transformations. How does it work for inference on individual unseen patients. Please clarify.

  • MEST-C tasks are not explained nor referenced. Also it is not clear whether there's a separate ABMIL model trained for each task.

Questions for authors

see weaknesses.

Reviewer_SiBpOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ARISE, an agentic framework for multi-stain computational pathology that keeps stain-specific ABMIL experts fixed and uses an LLM agent to iteratively design task-specific prediction-level integration programs. The agent modifies candidate rule pools, optimization objectives, and search strategies, while a numerical inner loop optimizes weights. Evaluations across four clinical cohorts, three organs, and multiple classification/survival tasks indicate performance improvements over traditional baselines.

Critical strengths
  1. It naturally accommodates missing or serial-section stains without the need for registration or imputation, while keeping the stain-specific experts unchanged.

  2. Covers diverse clinical testing across various cohorts (kidney, head-and-neck, and lymphoma) towards classification and survival tasks

  3. Thorough ablations to investigate candidate-pool construction, agent-added rules, and agent-designed search, that help identify where performance improvements originate.

Critical weaknesses
  1. Once the agentic framing is removed, ARISE is essentially prediction-level ensemble selection/stacking over a large hand-engineered candidate library, building directly on long-established ensemble-selection methodology. The primary novelty is therefore deploying an LLM to modify this search space rather than introducing a fundamentally new multi-stain representation or fusion mechanism.

  2. The authors' own evaluations reveal that standard greedy or bagged selection over the expanded candidate pool achieves nearly identical performance to the agent-designed search, rendering the expensive LLM loop largely redundant.

  3. The paper fails to compare the LLM agent against a traditional algorithmic proposer (e.g., random search, genetic programming, or evolutionary algorithms)

  4. Relying on iterative LLM agent calls introduces high API costs, latency, and non-determinism, presenting barriers for clinical translation without reproducibility guarantees across random seeds.

  5. CureGN improves by only +0.014 mean AUROC, HANCOCK survival by +0.009, and the DLBCL improvement of +0.035 C-index has p=0.088p=0.088; several individual kidney lesions are also non-significant.

  6. Contemporary multi-stain architectures such as UNICORN and PathMoE explicitly model interactions between stain-specific representations, whereas ARISE reduces each WSI to a low-dimensional expert prediction before fusion. Potentially useful morphological and cross-stain information is lost.

  7. Much of the performance gain, in almost every kidney-based evaluation task, appears to originate from candidate-space engineering rather than agentic reasoning.

Questions for authors
  1. Is the LLM agent actually necessary for the reported performance? How about using an automated non-LLM scripted proposer?

  2. Given LLM generation is stochastic, how stable are the programs and performance metrics across independent agent runs with different LLM seeds?

  3. Is most of the final performance improvement an effect of conventional greedy selection?

  4. Can a program discovered on CureGN be applied directly to CMUH without re-running the agent?

  5. Estimate the financial cost required for an agentic search run per task compared to standard greedy ensemble selection.

Reviewer_7nK7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

ARISE first trains and freezes a prediction model for each stain. An LLM agent then modifies fusion rules based on validation feedback, while numerical algorithms select the rules and their weights. Once the final program is fixed, the fusion stage requires no LLM calls. The experiments cover kidney lesion scoring, head-and-neck cancer survival and recurrence classification, and lymphoma survival risk prediction.

Critical strengths

1. Independent external validation supports the practical value of the final fusion program

On the external CMUH cohort, which was excluded from training, validation, and search, ARISE achieves a mean AUROC gain of 0.037 across the five kidney scores over the strongest method in the predefined comparison panel, with a reported 95% confidence interval of 0.019–0.056 (Section 4.3, Figure 1). This cohort uses a different scanner, and the frozen program is evaluated without adaptation. The result therefore provides stronger evidence of effectiveness at a new institution than internal random splits alone. This is the strongest empirical consideration supporting acceptance in my assessment, although its scope remains limited to this external kidney cohort of 65 patients.

2. Designing fusion rules over fixed experts has practical value, with direct support from component comparisons

The approach can reuse existing stain-specific models, requires no precise registration between sections, and does not require an LLM at the final fusion stage. Table 3 further demonstrates the value of designing candidate rules: with greedy selection held fixed, adding cross-task rules improves external AUROC for M, E, S, and C from 0.750, 0.721, 0.788, and 0.759 to 0.767, 0.740, 0.800, and 0.784, respectively. This comparison provides clearer evidence of which additions are useful than a comparison of complete models alone, supporting prediction-level fusion rule design as a worthwhile research direction. It establishes the empirical value of the rules; whether generating them requires an agent remains subject to the controls discussed below.

Critical weaknesses

1. The experiments do not sufficiently separate the benefits of agentic design from those of expanding the candidate pool

The agent is central to the paper's contribution, but the existing controls mainly compare already-designed candidate pools and their search procedures. Table 3 shows that conventional selection recovers most of the performance, while the agent-designed search provides only limited, task-dependent additional gains. The effectiveness of cross-task rules does not itself establish that an LLM proposing them based on feedback is better than scripted or predefined expansion. The conclusion also acknowledges that a comparison against a scripted proposer under the same budget remains future work.

Thus, the evidence supports the effectiveness of the final fusion program more strongly than it establishes the agent's added value over existing automated design procedures. This directly affects my assessment of the core methodological contribution. Agent proposals should be compared with scripted or predefined rule expansion, starting from the same initial rules and validation information under comparable total budgets. Matching only the number of evaluations in the final search does not account for the complete design process.

2. Without appropriate conventional quantification baselines, the added value of complex fusion for kidney scoring remains unclear

Kidney MEST-C scoring assesses lesions already present on the slides, and some components are graded according to the proportion of affected glomeruli or tissue area. For components that can be quantified on a particular stain and graded using established criteria, an appropriate conventional image-quantification pipeline is an important baseline. Table 2 compares single-stain ABMIL, ensemble methods, and multi-stain fusion architectures, but does not establish how well direct quantification performs on these scores. The current results therefore demonstrate improvements over the tested learning-based models, but do not adequately establish the additional benefit over conventional scoring pipelines.

Kidney scoring constitutes a substantial portion of the experiments and provides the paper's only external institutional validation, making this omission consequential to my assessment of practical value. The authors should identify the applicable scores and conventional quantification methods, and compare scoring performance and required human input on the same test patients. If quantification pipelines already achieve similar performance, the value of complex fusion would need to be demonstrated through accuracy, degree of automation, or performance with missing stains.

Questions for authors

1. Is the fusion program used for each test split determined independently?

Table 1 describes five random patient-level splits, Section 3.2 jointly optimizes over multiple validation splits, and Section 3.3 states that the final members, weights, and fitted combiners are shared across all splits. Please clarify whether the program used for each test split is determined solely from that split's training and validation data or jointly from all five validation sets, and describe patient overlap across splits.

Patients changing roles across splits is not inherently problematic. However, if a shared program is selected using multiple validation sets, and a patient in one test set also appears in another validation set, that patient's label could indirectly influence the former test result. If such cross-split involvement occurs, evaluation should be repeated with test patients fully isolated from the design process.

2. Can the authors provide a complete agent execution trace and the settings needed for reproduction?

Please provide the feedback received by the agent, actual prompts, proposed changes, experimental results, criteria for retaining or reverting changes and stopping, and the final rules and weights. In particular, Section 3.3 allows changes to the optimization objective: how is improvement judged across different versions of that objective? Please also specify the parameters in Equation (1), search configurations, and the training settings and tuning ranges for the experts and comparison methods. The five numerical-search seeds do not constitute independent repetitions of the LLM design process. If the latter was repeated, please report variation in outcomes and any human intervention.

3. How do the confidence intervals account for repeated patients and correlated tasks?

If the same patient is evaluated across multiple splits, please explain how resampling preserves the dependence between those predictions and how results are aggregated across splits. Likewise, evaluating the same 65 external patients with five models does not create five independent patient cohorts. How is the interval for the mean gain across the five correlated kidney scores calculated? These details are necessary to interpret the reported intervals and significance; a large number of resamples does not itself address sample dependence.

4. Why does the comparison panel for significance testing exclude learned ensembles?

The reference in Figure 1 comes from the first three groups in Table 2, excluding the fourth. For example, the internal C-score gain is 0.018 relative to that panel, but 0.010 relative to ridge stacking with cross-task inputs. These support different comparative conclusions; please explain the choice of panel. Higher point estimates in all 13 task–cohort combinations should also be distinguished from significant evidence in every combination: the lymphoma result has a one-sided p=0.088, and the kidney T score does not reach one-sided p<0.05 in either the internal or external cohort.

5. Are methods compared on the same patients?

Table 1 states that all methods score every patient, whereas Table 2 notes that single-stain models on HANCOCK are evaluated only on patients with the corresponding stain. Please explain how paired comparisons involving these models are performed. Please also clarify whether the counts in Table 1 and Figure 2 represent unique patients or repeated evaluation records, and why sample sizes differ across tasks.

6. How are scores computed for new patients and patients with missing stains?

Section 3.1 uses within-cohort ranks. Please specify the reference distribution for new patients, whether prediction in different batches affects outputs, and how the neutral value for missing data is applied in different task-specific output spaces, such as multiclass log-probabilities.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.