RePath: Zero-Shot Whole-Slide Tumor Segmentation via Intra-Slide Relational Affinity

Zero-shot tumor segmentation of whole-slide images (WSIs) via vision–language models (VLMs) faces a fundamental obstacle: the gigapixel resolution of WSIs forces patch-level processing, with each patch independently scored against fixed text embeddings. This absolute patch-level scoring produces two failure modes: prompt dependency (sensitivity to prompt wording) and spatial inconsistency (adjacent patches of the same tissue receiving conflicting labels), both reflecting a mismatch between slide-level histopathological interpretation and independent patch-wise classification. We propose RePath, which reformulates inference from per-patch scoring against fixed text embeddings to relational comparison among patches within the slide, built on three design principles. First, slide-specific anchor selection draws tumor, normal, and non-tissue anchor patches from the slide itself, using text prompts only to seed the selection. Second, intra-slide relational affinity recasts isolated image-text similarity as the Jensen-Shannon Divergence (JSD) between each patch's class distribution and the anchor distributions, yielding tumor, normal, and non-tissue affinity maps. Third, Differential Affinity Integration (DAI), inspired by clinical differential diagnosis, contrasts the tumor affinity against the normal and non-tissue affinities to produce a coherent segmentation mask. On 1,647 test WSIs across four organs and three VLM backbones, RePath improves Dice by 13.0 and IoU by 14.5 over the vanilla zero-shot baseline and, on average, matches or exceeds few-shot supervised methods trained with 10 labeled WSIs, without annotation or training.