acceptodds
Withdrawn from ICLR 2027

FOCAL: Agent-guided tissue masking by visual reconciliation of segmentation disagreements

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Tissue detection in whole-slide images (WSIs) is a critical preprocessing step for computational pathology pipelines. Classical approaches rely on thresholding and morphological operations, while deep learning can improve accuracy but depends on diverse annotated training data and can still struggle to generalize across organs and stains, particularly on challenging slides. We propose FOCAL (Foundation model and Oracle Classification for Annotation-free tissue Localization), an agent-guided tissue detection pipeline that combines candidate masks with visual reasoning to resolve segmentation disagreements without additional task-specific annotation or detector fine-tuning. FOCAL separates consensus from disputed regions, uses a vision-language agent to adjudicate tissue versus background in the original image context, and reconstructs the final mask while retaining regional decision reasons and artifact labels. We evaluate on four datasets (TWI, HEST, ACROBAT and KPMP) totaling 6,203 WSIs across diverse tissue types, stains and scanning platforms. FOCAL achieves strong overall performance and is especially robust on challenging slides with artifacts, weak staining and fragmented tissue, etc. Beyond mask prediction, we organize artifact-labelled regions into a traceable resource for targeted retrieval of challenging cases and future artifact-detection, quality-control and tissue-segmentation robustness benchmarks after human verification and necessary revision. Code, generated tissue masks, documented artifact annotations, and curated challenge sets will be released upon paper acceptance.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
vision language models, image segmentation, histology, quality control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_UBAVOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The method combines several candidate tissue masks for whole slide images on a grid. It separates pixels into unclear regions, then uses a VLM agent to decide between tissue and background for the larger unclear areas. Smaller or over-budget regions are classified using a mean-vote threshold. The approach is evaluated on several datasets and compared to seven threshold-based and learned detectors.

Critical strengths
  • tissue detection is relevant and often breaks downstream things. There is not much focus on this right now since reviewers think this is solved but it is not, espeically not across stains
  • some good results on public datasets, especially in the low tail on TWI
  • some reasonable QC, the per-region decision log could be useful
  • a small dataset of manual masks made public
Critical weaknesses
  • what are the candidate detectors?
  • using a VLM as judge for segmentation masks has been tried, see SegAE (Chen et al. ISBI 2026, doi:10.1109/ISBI61048.2026.11515717, Jain et al., MICCAI 2026 Workshops, LNCS 17251; arXiv:2606.16658), perhaps worth having a look.
  • what about STAPLE (TMI 2004;23(7):903-21) label fusion, doing agent off would give majority vote fusion? how much gain comes from the agent?
  • HEST seems circular since this comes from HEST's own tissue detectior. check if these cohorts link back to references made by models
  • if you review KPMP and find wrong labels, why not doing the same with your method. how does label noise compare?
  • any info on agent noise? how would that look like if you re-run in a couple of times?
  • how much is this if I run this through claude's API?
  • do you also test the non-H&E or IHC slides in TWI?
  • I'd also suggest to test the concept with a couple of available open-weight models.
Questions for authors
  1. Which candidate detectors does the approach use, and what settings are applied to them?
  2. How does the method compare to other work that uses a VLM to judge or create segmentation masks, such as SegAE (Chen et al. ISBI 2026) and Jain et al. (MICCAI 2026 Workshops, LNCS 17251; arXiv:2606.16658)?
  3. How much improvement is due to the agent? What results do you see if the agent is turned off, using mean-vote fusion over all regions, or when using STAPLE (TMI 2004;23(7):903-21)?
  4. Are any of the candidate detectors based on HEST's tissue model or related to it?
  5. How consistent are the agent's decisions? If you run the same slides multiple times, how does the Dice score change?
  6. What is the cost of running the method through the API per slide, in terms of tokens, time, and money?
  7. How does the method perform on TWI slides that are not H&E or IHC?
  8. Does FOCAL still work when using open-weight VLMs?
Reviewer_JcU3OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose a Claude orchestrated, Claude adjudicated tissue extraction pipeline, however, with much of the materials required for reproduction, the work is not verifiable, and in general, is far too application focused to be relevant to representation learning. The authors must consider alternate venues which are specifically geared towards this work.

Critical strengths

The authors show that agent pipelines can reliably segment tissue regions with the added benefit of adjudication, a feature currently lacking in standalone tissue segmentation approaches.

Critical weaknesses

The work has marginal novelty and does not address representation learning, rather, it combines existing classical approaches and uses an LLM orchestrator for adjudication. Such a work does not meet the guidelines or scope of ICLR.

Questions for authors

while the images shows basically just show that many other methods are not designed to extract patches from fat because the fat has limited useful information, are there are other tissue types with relevant tissue that should be included in a analysis.

Reviewer_Lz9kOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Paper tackles task of tissue segmentation in whole slide images and propose FOCAL. At core, FOCAL use vision-language agent to review regions with disagreement. FOCUS use multiple candidate masks to create consensus and disagreement region based on majority voting and VLM agents takes a call regarding foreground/background regions. Visual agent operates a two stage local (local visual adjudication) and global (global refinement). FOCUS is evaluate on four datasets and gains across the datasets.

Critical strengths

I think paper tackles an important problem of tissue segmentation in the whole slide images. Use of VLM agent to resolve the disagreement region is a natural solution and is well posed.

Critical weaknesses
  1. In terms of writing, contributions and methods should be clearly stated. I find the method section convoluted at places which at core is straightforward. There are details which are missing regarding the method and implementation which might creates confusion.
  2. Paper would require to strengthen the evaluation to justify the methodical contribution is actually significant. Currently, it looks incomplete and leave reader with move questions than answers.
Questions for authors
  1. Work looks similar to [1], where they work to reconcile the masks from two different sources using ILP formulation. Can authors use BoxCell or DenseCRF and see what gain does the visual agent provide? Did authors explore alternatives for the majority voting ? Maybe a weight sum of the candidates ?
  2. Can you provide additional information regarding the numbers calls on an average method made to the claude opus 5 (LA)? What is the average time and computational cost? Have you tried with the open source models like Qwen 3.8/MedGamma? Also what is the baseline performance of the VLM directly asked to perform tissue segmentation?
  3. What is the role/impact of the global refinement block? Can you provide an ablation regarding the impact of LA and GR. Is there an understanding regarding how many errors that LA missed were recovered by the GR stage? You can also provide the decision (baseline) based only on the majority voting.
  4. Which models were used to create the candidate mask, are these the same model or different model? Adding these details will improve the reproducibility aspect of the problem. Did you encounter cases of False negative where none of the masks detected the region or False positive where every mask segments an additional region? How does your method deal with these cases?
  5. Can you provide the details regarding the number of samples used for evaluation? How many samples were used to train the candidate models and what performance did they achieve?

[1]: BoxCell: Leveraging SAM for cell segmentation with box supervision

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.