acceptodds
Withdrawn from ICLR 2027

Cell2Case: A Unified Multi-Task Model from Cell-Level Perception to Patient-Level Diagnosis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Patient-level diagnosis in hematopathology requires integrating diverse cellular evidence distributed across patient-associated microscopic views to distinguish clinically and genetically distinct hematologic conditions. Existing unified hematopathology models recognize individual cells and images, but do not explicitly organize their predictions into patient-level evidence. To bridge this gap, we propose Cell2Case, a unified framework that converts local predictions into reusable cellular evidence and hierarchically aggregates it across microscopic views. Cell2Case adapts detector-anchored cellular predictions while preserving their initial semantics, then binds class, localization, morphology, and contextual features into structured, query-aligned cell representations. These representations encode what each detected cell is, where it occurs, and how it appears, while retaining contextual information for downstream reasoning. They support perception and language tasks and enable hierarchical aggregation from cell-level observations through view-level summaries to patient-level predictions. This design allows rare abnormal events, cellular composition, and morphological variation to jointly inform diagnosis instead of collapsing them into a generic case representation. We evaluate Cell2Case across seven tasks using 46 task–dataset configurations from 36 public hematopathology collections and three patient cohorts. Under a shared evaluation protocol, Cell2Case achieves the highest mean performance among the evaluated baselines on every supported task. For seven-class diagnosis on a patient-disjoint test set pooled from three cohorts, Cell2Case improves balanced accuracy by 10.0 percentage points over the strongest evaluated patient-level baseline.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
digital hematopathology, multi-task learning, structured cellular evidence, cellular representation learning, multimodal learning, medical vision-language learning, patient-level diagnosis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_A8rWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Instead of proposing a new hematology learning paradigm from scratch, it further makes a more complete and systematic connection between cell-level multi-task models such as Uni-Hema and case-level aggregation models such as CAREMIL. It is valuable in that it tries to preserve explicit information such as cell category, morphology, composition and context before case aggregation, rather than just passing a plain embedding.

Critical strengths
  1. The system provides relatively complete integration across tasks and scales. The main strength of Cell2Case is not a single new module, but that it brings several previously separate capabilities into one framework.

  2. The patient-level representation does not rely only on black-box pooling. Hematologic diagnosis may depend not only on a few highly abnormal cells, but also on cell-type proportions and patterns across multiple views. Therefore, the way Cell2Case organizes patient-level information is reasonably aligned with the nature of hematopathology.

  3. The experimental evaluation is relatively comprehensive. The paper evaluates multiple cell-level, language, and patient-level tasks, and also provides several component ablations.

Critical weaknesses
  1. The main novelty is more about system integration than a new modeling principle. Before Cell2Case, DinoBloom had already studied foundation representations for blood cells. Uni-Hema had already unified detection, classification, segmentation, morphology prediction, VQA, and MLM. CAREMIL had already used cell morphology, cell composition, and attention-based aggregation for case-level diagnosis. Cell2Case combines these directions in a more complete system, but it does not clearly introduce a new learning paradigm. Its novelty is therefore closer to system design and integration, which makes the methodological contribution somewhat weaker.

  2. The model is not as fully unified as the title may suggest. Not all seven tasks use the same CBME evidence. Different tasks still have separate adapters, prediction heads, losses, and training stages, and many modules are frozen during training. Segmentation also requires an additional segmentation ontology ID provided by dataset metadata. Therefore, it may be more accurate to describe Cell2Case as a multi-task system that shares part of its visual and cellular representations, rather than a model in which all tasks are unified through one common cellular evidence representation.

  3. The patient-level task has strong cohort–label confounding. The seven diagnosis classes are not fully shared across the three cohorts. JHU contains only PML–RARA and AML-unspecified, while C-NMC contains only Control and B-ALL. Several other classes, including NPM1, CBFB–MYH11, and RUNX1–RUNX1T1, mainly come from MLL. This means that cohort identity itself contains substantial information about the diagnosis. The source-only baseline reaches 41.18% accuracy, which directly shows this dependence. The source-adversarial loss reduces linearly recoverable cohort information, but a near-random source probe does not prove that the model no longer uses cohort-related shortcuts. Nonlinear source information, staining differences, scanner differences, and biological signals that are correlated with both cohort and diagnosis are still difficult to separate completely. Therefore, the reported 75.01% balanced accuracy is best interpreted as patient-disjoint classification within a mixture of three already seen cohorts, rather than evidence of generalization to an unseen cohort.

Questions for authors

see weakness

Reviewer_ECJnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Cell2Case, a unified multi-task framework for hematopathology that connects cell-level perception to patient-level diagnosis. It builds on a frozen, released unified model (Uni-Hema) as an "anchor": a residual refiner with a zero-initialized gate adjusts the anchor's class, box, and morphology predictions using multi-scale context from a frozen DINOv2 and a trainable convolutional stream. Sparse experts then produce per-cell event embeddings, which are bound with the refined predictions into class–box–morphology–event (CBME) records. These records support detection, morphology, segmentation, VQA, and masked language modeling. For patient diagnosis, a hierarchical patient evidence memory (HPEM) aggregates the records from cells to views (using attention, expected cell-type counts, and morphology statistics) and from views to the patient (using a View Transformer). The authors evaluate seven tasks on 46 dataset configurations from 36 public collections, and seven-class diagnosis on 396 patients from three cohorts (274/54/68 train/val/test, patient-disjoint). They report the best mean score on every supported task against task-specific, vision–language, and unified baselines, and 75.01% balanced accuracy for diagnosis, 17.84 points above the strongest patient-level baseline (CAREMIL), with ablations on the main components and on the number of views.

Critical strengths
  1. Clinically well-motivated problem with strong reported results. Patient-level diagnosis in hematopathology requires combining evidence scattered across many microscopic views, and the paper addresses the gap between cell-level models and case-level reasoning directly. The reported performance is strong: the method is best among the compared baselines on every supported task, and its patient-level diagnosis result is substantially above the strongest patient-level baseline. If the controls requested below confirm that these gains come from the proposed evidence representation, this would be a meaningful contribution.
  2. Careful and transparent patient-level evaluation. Validation and test patients are excluded from the training data of all tasks, and the authors openly acknowledge the main limitations (a small test set, label support that is not fully crossed across cohorts, and no unseen-cohort generalization). They also report a source-only baseline, a source probe, bootstrap confidence intervals, and a view-count analysis. This makes the results easier to interpret and the remaining issues easier to pin down.
Critical weaknesses
  1. The central claim and the headline diagnosis result are not adequately supported. The paper's main claim is that structured, query-aligned cellular evidence is better than generic cell embeddings for patient-level diagnosis, but I could not find a concrete analysis or controlled experiment that isolates this. The diagnosis gain over baselines (17.84 points in the PDF) is therefore hard to attribute to the proposed evidence representation. Three points make this more consequential:
  • The test set is small and the estimate is imprecise: the 95% CI of balanced accuracy is very wide ([60.92, 83.17]). Evaluation on more data is needed.
  • MILLIE, CAREMIL, and cAItomorph each come with their own feature extractor, but the authors replaced all of them with DINOv2-S and retrained them on 274 patients. For example, cAItomorph was originally trained on about 1,630 patients (per the arXiv version). A shared backbone is a reasonable control, but I would also like to see these baselines run as originally designed. CAREMIL, for instance, uses an encoder trained on cell classification (although its paper also reports other encoders), whereas here it gets a DINO backbone that is not cell-specific. The proposed method, in contrast, brings in cell-level information from another extractor (Uni-Hema).
  1. It is unclear where the gains on the other tasks come from. The paper is mainly about patient-level diagnosis, yet the gains in classification and detection appear to come from the backbone and the refiner rather than from HPEM, the core cell-to-case component.
  • Classification shows the largest gain over other methods, but Table 2 has no row showing which component produces it. A frozen-backbone comparison is needed, but the classification baselines in Table 1 include DinoBloom and no DINOv2, so the comparison is incomplete. Please add a DINOv2 baseline under the same input resolution and loss, including the matched input size.
  • It is unclear why multi-task training yields gains; the interpretation is uncertain and needs a more detailed, convincing analysis.
  • Several baselines weaken the "best on every task" claim. BLIP-2 is zero-shot but is compared with supervised models, which may make the proposed method look better. For RT-DETR, only the smallest variant (PResNet-18, 3 decoder layers) is reported, at 352×352 instead of the default 640×640, and it reaches only 26.8 mAP50 vs. 66.19 for Cell2Case; please state whether pretrained weights were used and fine-tuned. Most of the detection, classification, and segmentation baselines are also old, and recent methods are missing.

Minor.

  • The mixture-of-experts needs more analysis: the usage ratio of each expert and whether routing collapses to a few experts; whether routing correlates with cell type or task (direct evidence of specialization); and a capacity-matched control (a dense FFN with the same number of parameters) to check whether the gain simply comes from increased capacity.
  • "Field token" and "view token" should be one term.
Questions for authors
  1. The abstract on the OpenReview page differs from the abstract in the PDF: the OpenReview abstract reports a 10.0-point gain in balanced accuracy over the strongest patient-level baseline, while the PDF reports 17.84 points. Which one is correct, and which version should be used for evaluation?
  2. In Table 2, the "w/o residual updates" variant uses the anchor's attributes (C0, B0, M0) as is. Its detection score (62.43) matches Uni-Hema exactly, yet its morphology score is 89.81 vs. 82.25 for Uni-Hema, a 7.56-point difference that is not explained in the paper. Is this due to a difference in evaluation protocol, or are other modules affecting morphology in this variant?
  3. Was the released Uni-Hema, used as the frozen anchor, trained on images of the diagnosis validation or test patients? If its training data overlaps with these patients, how is leakage ruled out?
  4. The proposed model uses DINOv2 as its backbone, but the classification comparison uses DinoBloom. Why was DINOv2 chosen as the backbone, and why was DinoBloom chosen as the comparison? How would Cell2Case perform if DinoBloom replaced DINOv2?
Reviewer_TCV4OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Cell2Case presents a unified multi-task learning framework for digital hematopathology that bridges the gap between cell-level image analysis and patient-level diagnosis. Existing hematopathology models typically focus on separate tasks such as cell detection, classification, segmentation, morphology prediction, and language understanding, while patient-level diagnostic systems often aggregate generic cellular embeddings without explicitly preserving clinically relevant evidence. Cell2Case addresses this gap through an anchor-preserving residual refinement mechanism that enriches detector queries with multi-scale information and constructs structured class–box–morphology–event (CBME) representations. These records are integrated through a Hierarchical Patient Evidence Memory (HPEM), which combines salient cellular events, expected cell-type counts, morphology statistics, and cross-view relationships to generate patient-level diagnostic predictions. The authors evaluate seven tasks using 46 dataset configurations from 36 public hematopathology collections, together with three patient cohorts comprising 396 cases. Cell2Case outperforms evaluated baselines across the supported tasks and achieves 75.01% balanced accuracy for seven-class patient-level diagnosis.

Critical strengths
  • The paper tackles an important limitation in computational hematopathology: connecting local cell identification and morphological interpretation with patient-level diagnostic decisions. The structured evidence formulation is well motivated by the clinical task.
  • The authors evaluate seven tasks and provide component-level experiments. Removing HPEM reduces patient-level balanced accuracy from 75.01% to 59.40%, supporting the importance of the proposed aggregation mechanism. The reported patient-level improvement over CAREMIL also has a paired bootstrap 95% confidence interval of [8.64, 27.19] percentage points.
Critical weaknesses
  • The seven-class diagnostic taxonomy is not fully crossed across the three cohorts, meaning that source identity may provide information about diagnosis. Patient-disjoint splitting does not eliminate this confounding.
  • The patient-level test set contains only 68 patients, with some classes represented by very few individuals. This is insufficient to establish dependable clinical generalization, particularly for rare hematologic conditions.
  • The framework combines multiple neural components, but it is not yet clear whether the complete architecture offers sufficient advantages over simpler cellular foundation-model embeddings followed by strong hierarchical MIL methods. Furthermore, the clinical faithfulness of the structured evidence is not established solely by predictive accuracy.
Questions for authors

Cell2Case's main technical limitation is the possibility that highly flexible hierarchical aggregation learns correlations between diagnostic labels and dataset provenance rather than genuine disease-specific cellular morphology. Although the authors investigate source dependence and use source-adversarial learning, the pooled-cohort evaluation cannot fully determine whether the resulting representations generalize to new institutions or acquisition protocols. This concern is especially significant because the clinical label distribution differs across the three cohorts. Furthermore, the evaluation uses relatively few patient-level test cases, increasing sensitivity to class composition and random variation. The model's complexity introduces additional concerns involving computational cost, sensitivity to cell-detection errors, variable numbers of microscopic views, and the reliability of morphology predictions. The experiments would be strengthened by institution-held-out validation with overlapping diagnostic categories, comparisons against modern hematology foundation models with matched aggregation modules, calibration and uncertainty analysis, expert assessment of explanation fidelity, and detailed error analyses across rare diagnostic classes.

Here are the key questions to be addressed. Q1: Since the diagnostic labels are not fully crossed across cohorts, how can the authors distinguish disease-specific morphological learning from source-specific classification? Q2: Can the authors provide leave-one-cohort-out or institution-held-out experiments restricted to diagnostic classes shared across the relevant cohorts? Q3: What is the diagnostic accuracy of a model trained solely on cohort identity, acquisition metadata, or other non-disease characteristics? Q4: How stable are the seven-class balanced accuracy and macro-F1 estimates across bootstrap samples, alternative patient-level splits, and minority-class subsets? Q5: How does CBME+HPEM compare with strong hierarchical MIL, Set Transformer, attention pooling, and pathology foundation-model pipelines under matched backbones and training budgets? Q6: Can hematopathologists independently validate whether the cell-level evidence identified by HPEM corresponds to clinically relevant diagnostic features? Q7: How robust is patient-level diagnosis to false-positive cell detections, misclassification of rare cells, segmentation failures, missing microscopic views, and corrupted morphology attributes? Q8: What are the calibration, sensitivity, specificity, uncertainty, and subgroup-wise performance for each diagnostic category, particularly the clinically important rare classes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.