acceptodds
Under review as a conference paper at ICLR 2027

Look Beyond the Question: Preserve–Focus–Fuse for Whole-Slide Image Question Answering

Abstract

A question about a whole-slide image rarely names every morphological finding needed to answer it. Question-guided patch selection can therefore overlook supporting morphology, while slide-level compression can dilute focal evidence. Inspired by how pathologists combine broad slide review, question-directed examination, and integration of their findings, we introduce BeyondPath, a Preserve–Focus–Fuse framework for whole-slide image question answering. Preserve learns a question-agnostic Morphology Bank through answer supervision across multiple questions about each training slide. Focus retrieves question-specific evidence from all foreground patches, including regions outside the Bank. Fuse integrates the preserved morphology at the retrieved evidence positions through Morphology-on-Visual-Evidence (MoV) Fusion, producing a fixed 96-token visual prefix. On WSI-Bench, BeyondPath achieves an Avg-9 of 0.769 and leads all eight reported report-generation metrics using the same checkpoint. It also achieves 0.639 zero-shot overall accuracy on SlideBench-BCNB. Controlled ablations show complementary benefits from morphology preservation and question-directed retrieval, and support cross-question supervision and fusion at the evidence positions. These results support looking beyond the question’s explicit cues: preserving broader slide morphology while retaining the ability to seek evidence for each question.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision Language Model, Whole-Slide Image: Visual token selection, Visual question answering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YBE2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BEYONDPATH addresses whole slide image question answering through a question agnostic Morphology Bank, question conditioned retrieval over all foreground patches, and evidence queried fusion into 96 visual tokens. The Bank is trained using multiple answers per training slide before retrieval and fusion are learned. The paper reports 0.769 WSI Bench Avg-9, 0.639 zero shot SlideBench BCNB accuracy, and leading results across eight report generation metrics using the same checkpoint. The contribution is a useful organization of complementary visual information, supported by substantial controlled experiments.

Critical strengths
  • The architecture makes a meaningful distinction from HistoSelect, published at CVPR in June 2026. HistoSelect selects patches within question selected tissue groups. BEYONDPATH separately learns a question independent morphological representation and lets the question search the full foreground set. The Bank and evidence streams have clearly specified, different roles.

  • The controlled evidence goes beyond removing individual modules. The full model exceeds single stream scoring, Bank restricted retrieval, a learned resampler, and direct concatenation with twice as many language side visual tokens. Appendix C documents retraining versus inference interventions and supplies complete task metrics. These comparisons support the practical value of the proposed interface.

  • The evaluation includes focused answers, report generation, and transfer to BCNB. The appendix carefully specifies patient grouped validation, semantic prompts, selection recurrences, and frozen modules. It also distinguishes compact decoder context from full slide retrieval cost and reports inference timing including the Bank construction repeated for every request.

Critical weaknesses
  • Baseline provenance remains incomplete at the row level. Section 4.1 says published baselines follow their official protocols, and Appendix A.3 explains reproduced general model scores. However, the pathology entries in Tables 2 and 3 lack exact source tables or evaluation checkpoints and a clear indication of which results were imported or recomputed. This leaves the training cohort and evaluation comparability difficult to assess.

  • The prevalence of the proposed Relevance-Sufficiency Gap is not quantified. Random and wrong slide Banks are inference interventions, so their degradation shows Bank dependence without uniquely identifying recovery of missing supporting morphology. The retrained architecture controls and qualitative cases support the practical complementarity of the two streams, and Appendix F appropriately acknowledges the lack of systematic localization validation. The remaining limitation therefore concerns the scope of the proposed mechanism explanation rather than the validity of the system results.

Questions for authors
  • For each pathology baseline in Tables 2 and 3, which checkpoint, source table, or reproduced evaluation produced the score, and which training cohort and answer mapping were used?

  • If feasible, can the existing case analysis be extended to quantify when the Bank supplies cues complementary to Focus? This would provide additional support for the proposed mechanism explanation.

3 comments

We appreciate your valuable feedback and support of our work. We hope the following clarifications and additional results address your concerns. W = Critical Weaknesses, Q = Questions For Authors.

To Q1 & W1:

We have added the table below as Table 6 in Appendix A.3, marked in green. Tables 2 and 3 below refer to the manuscript tables.

Answer mapping. For generative BCNB baselines, we extract the final answer, normalize case, and remove non-semantic surrounding whitespace, quotation marks, and terminal punctuation while preserving symbols that form part of class labels. The normalized answer is mapped to the official label vocabulary of the corresponding task; ambiguous or unmappable final answers are counted as incorrect. When answers are represented as option letters, the mapping follows the option ordering of each individual question.

TITAN is evaluated as a frozen zero-shot image–text classifier using its official visual encoding and classification interfaces. Separate class text prototypes are constructed for the seven BCNB tasks. The class with the highest image–text similarity is mapped to the corresponding option in each question, accounting for question-specific option ordering. This procedure does not involve answer-text generation or generative answer normalization.

Report-generation evaluation. Report generation does not require class-label mapping. For report-generation results computed in this study, models are evaluated on the same official 208-WSI test list and corresponding reference reports. BLEU-1/2/3/4, ROUGE-L, and METEOR are computed using the official WSI-Bench evaluation scripts. WSI-Precision and WSI-Relevance use the GPT-4o claim-scoring prompts released with WSI-LLaVA at temperature zero. Results taken directly from prior publications retain the training and evaluation settings of their cited sources and are not re-normalized or re-scored in this study.

BaselineInference checkpoint / result sourceReported training / adaptation data
Quilt-LLaVATable 2: inference with wisdomik/Quilt-Llava-v1.5-7b; Table 3: taken from WSI-LLaVA Table 3Table 2: QUILT-1M and Quilt-Instruct; Table 3: WSI-Bench training split in the source experiment
SlideChatTables 2 and 3: inference with General-Medical-AI/SlideChat_WeightSlideInstruction: 4,181 TCGA WSIs and 175,753 QA pairs
WSI-LLaVATable 2: inference with Lucas-yuc/wsi-llava-v1.5-7b-e1; Table 3: taken from WSI-LLaVA Table 3WSI-Bench training split
MedDrTable 2: inference with Sunanhe/MedDr_0401Medical instruction mixture, including PathVQA and PCam200
TITANTable 2: inference with MahmoodLab/TITAN and CONCH v1.5Mass General Brigham pathology cohort: 335,645 WSIs
MedGemma-1.5-4BTable 2: inference with google/medgemma-1.5-4b-itPublic and private medical data, including TCGA and CAMELYON
Patho-R1 onlyTable 2: inference with WenchuanZhang/Patho-R1-7BPathology figure–caption pairs and reasoning data
MLLM-HWSITables 2 and 3: inference with Bastech/MLLM-HWSIWSI-Bench: 9,642 training WSI–report pairs and 175,450 QA pairs
HistoSelectTables 2 and 3: inference with histoselect_wsi-llava.pth and the official SlideChat stage-2 warmstartWSI-Bench training split; SlideChat warmstart trained on SlideInstruction
MI-GenTable 3: taken from WSI-LLaVA Table 3WSI-Bench training split in the source experiment
Hist-GenTable 3: taken from WSI-LLaVA Table 3WSI-Bench training split in the source experiment

To Q2 & W2:

We have added a blinded two-pathologist study in Section 4.4 and Bank-content interventions in Appendix C.7. We randomly sampled 60 test slides and two eligible closed-ended questions per slide. The pathologists independently assessed evidence sufficiency, agreeing on 106/120 questions (Cohen's κ=0.839\kappa=0.839), with the remaining assignments resolved by discussion.

Focus alone was insufficient for 55/120 questions. The Bank supplied complementary cues for 40 of these questions, accounting for 33.3% of the annotated sample. On these 40 questions, Full achieves 82.5% accuracy, compared with 42.5% for retrained Focus-only and 47.5% with Bank updates off in the same checkpoint.

Replacing the pathologist-identified supporting Bank patches reduces accuracy to 52.5%, whereas equal-count replacement of other patches retains 80.0%. Across all 120 questions, Full corrects 20/40 errors made with Bank updates off and changes 4/80 correct answers to errors. These results quantify the occurrence of complementary Bank cues and support their contribution to answer accuracy.

Thanks the authors' detailed rebuttal, including clarifications regarding the reviewer's concerns. I recommend the authors to include these details in the final version of the paper.

I will keep my score of 3 (weak acceptance).

Thank you for your support of our work, and we are very grateful for it.

Reviewer_PVFSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes BeyondPath, a Preserve–Focus–Fuse framework for whole-slide image question answering built on frozen CONCH encoders and Qwen2.5-7B-Instruct. Preserve selects a question-agnostic 96-token Morphology Bank, trained using answer supervision across multiple questions per slide. Focus independently selects 96 question-conditioned evidence tokens from all foreground patches, and expert-gated cross-attention fuses the Bank into these evidence tokens before answer generation. Training proceeds in two stages, first learning Preserve and then freezing it while optimizing Focus, fusion, and Qwen LoRA. Experiments report improvements on WSI-Bench VQA and report generation, as well as zero-shot transfer to SlideBench-BCNB, with ablations examining stream complementarity, multi-question supervision, and fusion design.

Critical strengths
  1. The architecture addresses a clear challenge in whole-slide VQA: preserving broader morphological context while retrieving focal evidence under a compact visual-token budget. Controlled ablations with the same backbone support the benefits of the two streams and their fusion.

  2. The evaluation covers both report generation and transfer to another cohort. The same VQA checkpoint leads all eight reported report-generation metrics. It also achieves 0.639 overall accuracy on SlideBench-BCNB without adaptation to that cohort. These results provide useful evidence that the visual representation works across tasks and cohorts.

Critical weaknesses
  1. The ablations show average gains, but leave the Bank's coverage and role unclear. Bank and Focus sizes are varied together. The paper does not report how many unique patches the Bank contains or how much it overlaps with Focus. It also lacks a direct comparison across questions using the same Bank. Most importantly, fusion is not directly tested when the Bank lacks the morphology needed for an answer. The residual connection and null gate may help in these cases, but their effectiveness is not established.

  2. The methodological novelty is limited. Token selection, expert gating, and residual cross-attention are established components. The main contribution is to combine a question-independent Bank with question-specific Focus and train the Bank across multiple questions per slide. This is a useful architecture for WSI-VQA, but the advance is mainly in how existing components are organized.

Questions for authors
  1. Can you vary the Bank size and Focus size separately, holding the other fixed? The current joint sweep does not show how much capacity each stream needs.

  2. How many unique source patches do the 96 Bank slots contain? Count patches selected by multiple experts only once. What fraction of Focus patches also appear in the Bank? Please report these distributions across test slides and questions.

  3. Can you show the full Bank and the Focus patches for several different questions on the same slide? Compare the full model with Focus-only for each question. This would show how one fixed Bank supports different questions.

  4. How do gains over Focus-only differ between low and high Focus–Bank similarity? For example, average each Focus patch's highest CONCH cosine similarity to a Bank patch. Also inspect cases where the Bank lacks the morphology needed for the answer. Low similarity alone does not establish this. Does fusion help or hurt in these cases? Does the null gate suppress irrelevant Bank updates?

  5. Can you replace the Focus visual queries with queries generated from the question alone? Keep the Bank and output-token count fixed. This would test whether visual features are needed in the queries.

5 comments

We appreciate your valuable feedback. We hope the following clarifications and additional results address your concerns. W = Critical Weaknesses, Q = Questions For Authors.

To Q1 & W1:

We have added independent Bank and Focus budget experiments in orange text in Appendix C.3. We first vary Bank size with Focus fixed at 96, then vary Focus size with the Bank fixed at 96. Bank-size changes retrain both stages; Focus-size changes retrain Stage II from the same Stage-I checkpoint.

With the LLM visual prefix fixed at 96 tokens, reducing the Bank from 96 to 48 lowers Avg-9 from 0.769 to 0.758. Increasing the Bank to 192 gives 0.772. Reducing Focus to 48 gives 0.750, while increasing it to 192 gives 0.771. Both streams contribute to performance, with smaller gains beyond the default budget of 96.

Table 1. Independent Bank and Focus Budget Experiments

BankFocusLLM visual tokensAvg-9
9696960.769
4896960.758
19296960.772
9648480.750
961921920.771

To Q2 & W1:

We have added Bank uniqueness and Focus–Bank overlap statistics in orange text in Appendix C.4. After deduplicating source-patch IDs across experts, the 96 Bank slots contain 89.31 distinct patches on average, with a duplicate-slot proportion of 6.97%. The mean proportion of Focus patches also selected by the Bank is 6.24%, leaving 93.76% outside the Bank. Bank statistics are averaged over slides, and Focus overlap is averaged over questions.

These results show that the two streams select largely different source patches. Semantic experts can share patches containing more than one tissue pattern. For example, the immune and necrosis experts in Figure 10 share three patches, as permitted by the expert-diversity design.

To Q3 & W1:

We have added the complete visualizations in Appendix E.1, with explanations in orange. Figure 7 shows three selected test slides, each with one fixed Bank and three question-specific Focus selections, together with reference answers and model outputs. Figures 8–10 show all 96 Bank slots by expert and all 96 Focus patches for each question, with source-patch IDs.

The full model corrects six Focus-only answers across these nine examples. In each correction, the identified supporting patch is in the Bank and outside the corresponding Focus selection. The table below summarizes the cases. The randomly sampled pathologist study described under Q4 provides the quantitative assessment of evidence sufficiency.

Table 2. One Fixed Bank across Three Questions per Slide

CaseQuestion topicFocus ∩ BankFocus-onlyFullSupporting Bank-only patch
S1-Q1Nuclear pleomorphism2/96IncorrectCorrect#4155, tumor
S1-Q2Epithelial infiltration of stroma4/96CorrectCorrect—
S1-Q3Resorptive inflammation / incipient scarring4/96IncorrectCorrect#677, immune
S2-Q1Necrosis in intraductal carcinoma6/96CorrectCorrect—
S2-Q2Fibrocystic changes4/96IncorrectCorrect#110, stroma
S2-Q3Calcifications in DCIS4/96IncorrectCorrect#1727, free expert 2
S3-Q1Tumor structural patterns9/96CorrectCorrect—
S3-Q2Peritumoral stroma6/96IncorrectCorrect#56, stroma
S3-Q3Tumor necrosis8/96IncorrectCorrect#35, necrosis

S2-Q3 illustrates the contribution of the free experts: free expert 2 retains calcification patch #1727, extending coverage beyond the four semantic anchors. All displayed Focus selections are from the full model.

To Q4 & W1:

We have added similarity-stratified results and Bank-content interventions in orange text in Section 4.4 and Appendices C.6–C.7. For each question, similarity is the mean of the highest CONCH cosine similarity between each Focus patch and any Bank source patch. We divide test questions into three equal-sized groups by this score. The Avg-9 gains over Focus-only are 0.068, 0.049, and 0.027 from low to high similarity, showing larger gains when the selected features differ more.

Table 3. Results by Focus–Bank Similarity

SimilarityFocus-onlyFullAvg-9 gain
Low0.6860.754+0.068
Medium0.7250.774+0.049
High0.7520.779+0.027

We also assessed evidence sufficiency directly. We randomly sampled 60 official test slides with at least two eligible closed-ended questions, then randomly selected two questions per slide using a fixed seed. Two pathologists independently inspected the Bank and the full model's Focus patches, blinded to model predictions. They agreed on the evidence group for 106/120 questions (88.3%; Cohen's κ=0.839\kappa=0.839), and resolved the remaining 14 by discussion.

For the 40 questions where the Bank supplies evidence missing from Focus, full-model accuracy is 82.5%, versus 42.5% for Focus-only and 47.5% with Bank updates off. Focus-only is retrained; Bank updates off and null gate disabled are inference-time interventions on the same checkpoint.

Table 4. Pathologist-Verified Evidence Sufficiency

Evidence groupnFocus-onlyBank updates offFullNull gate disabled
G1: Focus insufficient, Bank complementary4017 (42.5%)19 (47.5%)33 (82.5%)32 (80.0%)
G2: Focus sufficient, Bank insufficient3530 (85.7%)31 (88.6%)31 (88.6%)26 (74.3%)
G3: Both individually sufficient3026 (86.7%)27 (90.0%)28 (93.3%)27 (90.0%)
G4: Combined evidence insufficient153 (20.0%)3 (20.0%)4 (26.7%)2 (13.3%)
All12076 (63.3%)80 (66.7%)96 (80.0%)87 (72.5%)

Within G1, we replaced the pathologist-identified supporting Bank patches while keeping expert quotas and token counts fixed. Accuracy falls from 82.5% to 52.5%, compared with 80.0% when replacing an equal number of other Bank patches. This comparison supports the contribution of the specific morphology identified by the pathologists.

Table 5. Bank-Content Interventions in the 40 Complementary Cases

Bank interventionAccuracy
Original Bank82.5%
Equal-count random replacement of other patches80.0%
Targeted replacement of supporting patches52.5%
Bank updates off47.5%

When the Bank lacks the required evidence (G2 + G4, 50 questions), accuracy is 70.0% for Full, 68.0% with Bank updates off, and 56.0% with the null gate disabled. Relative to Bank updates off, Full corrects 3/16 errors and changes 2/34 correct answers to errors. Disabling the null gate corrects 2/16 errors and changes 8/34 correct answers to errors. Median null weights are 0.78 and 0.83 in G2 and G4. Normalized Bank-update norms are 0.036 and 0.024, compared with 0.153 and 0.139 without the gate. These results support the null path's role in reducing interference from irrelevant Bank content.

Bank updates off zeroes the Bank cross-attention residual while retaining self-attention and FFNs. Null gate disabled removes the null logit and renormalizes the six expert weights in both MoV blocks. The checkpoint, Bank, Focus selections, and decoding settings are identical across these interventions.

To Q5:

We have added a query-source ablation in orange text in Appendix C.5. We replaced Bank cross-attention queries with f(qc)+pkf(q_c)+p_k, where pkp_k is a learned slot embedding. Bank size, output length, visual residual, gate inputs, and auxiliary loss remain fixed, and Stage II is retrained from the same frozen Stage-I checkpoint.

Avg-9 decreases from 0.769 to 0.749 with question-generated queries and the visual residual retained. Initializing the full sequence from question-generated tokens gives 0.735. These results support using selected visual evidence to query the Bank at each evidence position.

Table 6. Ablation Study on Query Source

Query sourceAvg-9
Focus visual evidence0.769
Question-generated queries, visual residual retained0.749
Question-generated initial sequence0.735

To W2:

Our methodological contribution is the Preserve–Focus–Fuse design for the Relevance–Sufficiency Gap: selecting patches that match a question does not necessarily retain all morphological findings needed to answer it. A question specifies the decision to be made but may leave supporting morphology implicit. We study how a compact visual interface can preserve such morphology while still retrieving question-specific evidence.

Inspired by broad slide review, targeted examination, and evidence integration in pathological assessment, BeyondPath learns a question-agnostic Morphology Bank through answer supervision across multiple training questions about each slide. Sharing the Bank across these questions encourages it to retain reusable morphology beyond the explicit cues of any single question. For each new question, Focus independently searches all foreground patches, and MoV integrates the preserved morphology at the retrieved evidence positions. Established selection, attention, and gating mechanisms provide a concrete implementation of this learning and evidence-organization strategy.

Our controlled experiments test these design choices under the same backbone, data, and training schedule (manuscript Table 4). Cross-question supervision improves Avg-9 from 0.736 to 0.769, with a paired 95% confidence interval of [+0.020, +0.046]. Restricting Focus to the Bank gives 0.718, while concatenation followed by a learned resampler gives 0.727 at the same 96-token prefix. Direct concatenation gives 0.739 despite using 192 visual tokens. The pathologist study and targeted Bank-content interventions further support, within the annotated subset, that preserved morphology can supply evidence missing from Focus. Together, these results substantiate the proposed strategy for learning reusable morphology and integrating it with question-specific evidence.

Reviewer_cKXeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets whole-slide-image visual question answering (WSI-VQA), where question-guided patch selection can miss morphology needed to interpret the answer, while broad compression can dilute focal evidence. It proposes BEYONDPATH, a Preserve–Focus–Fuse framework. Preserve builds a question-agnostic Morphology Bank from frozen CONCH patch features using cross-question answer supervision: four semantic experts (tumor, stroma, immune infiltration, necrosis) and two free experts, each retaining 16 slots, yielding a 96-token Bank. Focus scores all foreground patches for the current question using pretrained image–text similarity plus a learned compatibility score, selecting 96 evidence tokens independent of the Bank. MoV Fusion uses these evidence tokens as queries to cross-attend the expert banks with position-specific gates, producing a 96-token visual prefix for Qwen2.5-7B-Instruct. Training is two-stage: Stage I learns Preserve/the Bank from multiple questions per slide; Stage II freezes Preserve and trains Focus, MoV, the projector, and Qwen LoRA for question-specific answering.

Critical strengths

1.The Preserve–Focus–Fuse design is pathologically intuitive and targets a genuine WSI-VQA issue: question-guided selection can omit supporting morphology. The decomposition into a question-agnostic Morphology Bank, full-foreground evidence retrieval, and evidence-position fusion is clean and well motivated. 2.The evaluation spans WSI-Bench VQA, report generation, and zero-shot SlideBench-BCNB, with controlled ablations for the two streams, cross-question supervision, and fusion. These comparisons support the design choices within the same CONCH-Qwen backbone.

Critical weaknesses

1.The empirical advantage is small and not statistically established. The headline gains are modest—0.769 Avg-9 versus 0.754 for WSI-LLaVA on WSI-Bench, and 0.639 versus 0.612 for MLLM-HWSI on SlideBench-BCNB—and the paper reports only a single seed (42), with Appendix F explicitly admitting no multi-seed variance or confidence intervals for predictive metrics. 2.Reported full-model and ablation numbers are internally inconsistent. The main text and Table 4 report the full model at 0.769 Avg-9, while Appendix Tables 9 and 10 report 0.791 for the full/default model. Similarly, the expert-count ablation reports 0.759 for “4/0” in Table 4 but 0.781 in Table 10, without explaining whether these are different checkpoints, metrics, or evaluation settings. 3.Inference overhead remains a concern: although the LLM receives only 96 visual tokens, Preserve and Focus still score all foreground patches per request, and the Morphology Bank is rebuilt for every request, including repeated questions on the same slide. This may limit interactive or multi-question use. 4.All experiments use a single seed (42), with no variance or confidence intervals for predictive metrics. Given the modest headline margins and mixed per-task results, this weakens confidence in the reported gains.

Questions for authors
  1. Since the Morphology Bank is rebuilt for each request, what is the inference-time breakdown between Bank construction and Focus/MoV as the patch count grows beyond 32,768? If Stage I/II training costs are intended, please report them separately.
  2. How robust are the anchored expert quotas and soft presence threshold ρ for rare or absent morphologies such as necrosis or immune infiltration? Does the model introduce noise when these morphologies are sparse or missing?
7 comments

We appreciate your valuable feedback. We hope the following clarifications and additional results address your concerns. W = Critical Weaknesses, Q = Questions For Authors.

To W1 & W4:

We have added the following results in blue text in Section 4.2 and Appendix A.6 of the revised manuscript. We repeated both training stages with seeds 42, 123, and 2026 using the same patient-level partition. The table below reports the mean and sample standard deviation of each metric. BeyondPath achieves 0.7693 ± 0.0025 Avg-9, compared with 0.7213 ± 0.0015 for Focus-only and 0.7390 ± 0.0010 for single-stream selection. Zero-shot BCNB Overall accuracy is 0.6390 ± 0.0030; per-task results are provided in manuscript Table 11.

Table 1. Three-Seed Results on WSI-Bench

ModelMorph. PMorph. RMorph.Acc.Diag. PDiag. RDiag. Acc.Treat. PTreat. RTreat. Acc.Avg-9
Full.5920 ± .0040.6453 ± .0035.9450 ± .0020.6213 ± .0035.6397 ± .0031.8753 ± .0025.7917 ± .0021.8143 ± .00211.0000 ± .0000.7693 ± .0025
Focus-only.5207 ± .0021.5743 ± .0015.9193 ± .0015.5833 ± .0015.5993 ± .0015.8420 ± .0010.7327 ± .0021.7623 ± .0025.9580 ± .0000.7213 ± .0015
Single-stream.5530 ± .0010.6133 ± .0015.9220 ± .0010.5860 ± .0010.6053 ± .0015.8510 ± .0010.7590 ± .0010.7823 ± .0012.9790 ± .0000.7390 ± .0010

We also computed paired 95% confidence intervals using matched predictions and 10,000 cluster-bootstrap resamples. Resampling groups all questions from the same patient, or from the same WSI when patient identifiers are unavailable. All five intervals are above zero, supporting the consistency of the improvements under test-set resampling.

For the paired baseline comparisons, we ran inference with the official released weights. Our WSI-LLaVA run achieves 0.753 Avg-9, compared with the published 0.754, giving a paired difference of +0.016 against our reported checkpoint. The MLLM-HWSI comparison uses predictions from our BCNB evaluation.

Table 2. Paired Differences and 95% Confidence Intervals

ComparisonMetricDifference95% CI
Full − WSI-LLaVAAvg-9+0.016[+0.007, +0.025]
Full − MLLM-HWSIBCNB Overall+0.027[+0.012, +0.041]
Full − Focus-onlyAvg-9+0.048[+0.034, +0.062]
Full − Single-streamAvg-9+0.030[+0.017, +0.044]
Full − Single-QA PreserveAvg-9+0.033[+0.020, +0.046]

The per-task results reflect different strengths. BeyondPath improves open-ended WSI-P and WSI-R for morphology and diagnosis, including morphology WSI-P of 0.592 versus 0.488 for WSI-LLaVA. MLLM-HWSI achieves higher closed-ended morphology and diagnosis accuracy, while WSI-LLaVA achieves higher treatment-planning WSI-P and WSI-R. On BCNB, BeyondPath leads on HER2 expression, HER2 status, and molecular subtype. We have clarified these comparisons in Section 4.2.

To W2:

The values 0.791 and 0.781 in the original Tables 9–10 are Treatment-Planning WSI-Precision for the default and four-semantic-expert configurations, respectively. Their Avg-9 values are 0.769 and 0.759, which match manuscript Table 4. We have separated the Avg-9 column with a vertical rule, bold type, and a shaded background to make this distinction clear. The original Tables 9–10 are now Tables 14–15 in the revised manuscript.

To Q1 & W3:

We have added module-level timing and repeated-question costs in blue text in Appendix D.5. Measurements use one A100-40GB GPU, batch size one, BF16, cached CONCH features, and at most 128 generated tokens. WSI reading and feature extraction are excluded from these inference timings.

Bank construction takes 0.53 s at 65,536 patches and 1.02 s at 131,072 patches, with total inference times of 3.23 s and 4.38 s. Decoding takes 1.82 s and is the largest individual module cost over this range. At 32,768 patches, BeyondPath takes 2.65 s, compared with 10.72 s for SlideChat under the same protocol (manuscript Table 25).

Table 3. Inference Time by Module (Seconds)

PatchesAdapterBankFocusMoVProjector + prefillDecodeTotal
8,1920.040.080.080.030.151.822.20
16,3840.080.150.130.030.151.822.36
32,7680.160.280.210.030.151.822.65
65,5360.320.530.380.030.151.823.23
131,0720.641.020.720.030.151.824.38

The submitted implementation constructs the Bank for each request. Preserve is frozen, deterministic at inference, and independent of the question, so a cached Bank produces identical predictions for the same slide and checkpoint. Caching both the Bank and adapted features reduces the computed cost of ten questions at 65,536 patches from 32.30 s to 24.65 s. The totals below are composed from module timings, including initial construction and excluding cache I/O. Training took 24 hours for Stage I and 16 hours for Stage II on three A100-40GB GPUs, totaling 120 GPU-hours.

Table 4. Repeated-Question Cost on One Slide (Seconds)

PatchesQuestionsNo cacheBank cacheBank + adapted-feature cache
32,768513.2512.1311.49
32,7681026.5023.9822.54
65,5361032.3027.5324.65
131,0721043.8034.6228.86

To Q2:

Each expert selects 16 patches, while the presence estimate ρ\rho interpolates semantic-expert tokens toward a learned null bank when the corresponding morphology is absent. MoV also provides a zero-output null path through which an evidence token can suppress Bank updates.

We have added a 90-question experiment on necrosis and immune infiltration in blue text in Appendix C.8. Each group contains 30 questions, classified by the target morphology's area fraction within valid tissue: absent, sparse (above 0% and at most 1%), or clearly present (above 1%). Two annotators independently agreed on 83/90 questions (92.2%; Cohen's κ=0.883\kappa=0.883). All variants use the same checkpoint.

Table 5. Accuracy under Morphology-Presence Interventions

Target morphologynFullForce ρ=1\rho=1Null path offBank updates off
Absent3086.7%80.0%76.7%83.3%
Sparse3083.3%73.3%73.3%76.7%
Clearly present3090.0%90.0%86.7%76.7%

Forcing ρ=1\rho=1 or removing the null path lowers accuracy by 6.7–10.0 percentage points in the absent and sparse groups. The corresponding changes are 0–3.3 points when morphology is clearly present. This pattern supports the role of presence weighting and the null path in handling absent or sparse morphology. Setting the presence threshold δ\delta to the training distribution's 25th, 50th, and 75th percentiles gives Avg-9 values of 0.767, 0.769, and 0.766.

Dear Reviewer cKXe,

I hope this message finds you well. As the discussion period is nearing its end with less than one days remaining, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Comments on the paper

3 comments

Dear ICLR 2027 AC, SAC, and PCs,

We sincerely thank the reviewers for their valuable feedback and constructive suggestions. We have carefully addressed their concerns through detailed rebuttal responses, additional experiments, and revisions to our manuscript.

However, we have not yet received follow-up responses from Reviewers. As the discussion period is approaching its end, we would greatly appreciate your assistance in encouraging these reviewers to revisit our rebuttal responses and updated manuscript.

We have made substantial efforts to address their concerns, including additional experimental validations, statistical analyses, and methodological clarifications. We sincerely hope to receive their feedback on whether our responses have adequately resolved their questions and concerns.

We understand that reviewers have demanding schedules and truly appreciate their time and efforts. Thank you very much for your consideration and support in facilitating a constructive discussion.

Best regards,

The Authors of Submission #10439

Dear Reviewers,

The authors have posted responses. As the discussion period is ending soon, could you please take a look and reply shortly, or at least acknowledge that you have read it?

Thank you!

AC

Dear ICLR 2027 AC, SAC, and PCs,

Following our rebuttal, PVFS raised their score from 2 to 3, acknowledging that the additional evidence substantially addressed their main empirical concerns. YBE2 maintained a score of 3 after reviewing our clarifications. The scores therefore changed from 3/2/2 to 3/3/2.

cKXe has not responded despite repeated follow-ups. Their inconsistency concern arose from misreading Treatment-Planning WSI-Precision (0.791/0.781) as Avg-9. The actual Avg-9 values, 0.769/0.759, were already consistent with Table 4; no numerical correction was needed. We also addressed their statistical and reproducibility concerns with three-seed evaluations and paired 95% confidence intervals, all five excluding zero, and supplied the requested cost and robustness analyses. These additions are included in the revised manuscript, but their original rating remains unchanged without follow-up. We respectfully ask the AC to independently assess whether the original weak-rejection grounds remain justified in light of the complete rebuttal and revised evidence.

Thank you for your time and consideration.

Best regards,

The Authors of Submission #10439

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.