acceptodds
Withdrawn from ICLR 2027

MagicStain: High-Fidelity Pathology Image Virtual Staining via Guided Single-Step Diffusion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Virtual staining leverages computational methods to generate different stain styles from a pathology image of a chemically stained tissue section, offering a cost-effective alternative to chemical multiple staining. Despite extensive research based on generative adversarial networks (GANs) and diffusion models, achieving high-fidelity, high-quality, and computationally efficient virtual staining remains a significant challenge. While diffusion approaches typically produce more photorealistic images than GAN counterparts through multi-step sampling, this comes at the cost of high computational overhead and inference latency. This paper proposes MagicStain, a novel single-step diffusion model tailored for generating high-resolution virtual stains. Specifically, we adapt a single-step diffusion model to enable efficient virtual staining. By introducing pathology priors from a pathology foundation model and integrating pathology- and structure-consistency losses on both the original images and the hematoxylin channel, MagicStain achieves high-fidelity, high-quality generations. To address the limitations of single-step diffusion models in high-resolution virtual staining, we further propose a two-stage progressive training strategy that enables high-resolution adaptation at low training cost. Extensive experiments on eight virtual staining datasets, each involving translation between different staining dyes/biomarkers, demonstrate the superiority of MagicStain in fidelity, visual quality, and computational efficiency compared to existing methods. Our code and trained models will be publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Virtual staining, Single-step diffusion, Progressive training

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_y5aBOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The manuscript proposes a conditional image-generation method for translating between protein markers, using an H&E image as the conditioning input to generate images in other immunohistochemistry (IHC) marker domains. The method incorporates domain-specific information through the pretrained vision–language model PathCLIP, uses single-step generation with discriminators during training to reduce inference time, and introduces a domain-specific hematoxylin-channel regularization loss to preserve relevant morphological features. The authors further propose a two-stage training strategy that first trains at low resolution and then fine-tunes at high resolution to reduce computational cost. The method is evaluated extensively across multiple datasets.

Critical strengths
  1. The experimental evaluation is extensive, covering multiple datasets and metrics, with mostly relevant baselines. I particularly value the downstream evaluations in Table 2, which are highly relevant and provide an important result for the paper.

  2. The use of domain-specific encoders and the effort to address the inference-latency limitations of diffusion models are both relevant and well motivated.

Critical weaknesses

These are the Critical Weaknesses in my opinion:

  1. Novelty. In my view, this is a strong benchmarking paper, but it is not sufficiently novel as a methods paper. Most of the proposed components have either already been used in computational pathology or exist in closely related forms in other domains.

    • The hematoxylin-channel loss has previously been proposed and used for virtual staining, some of which have already been referenced in related works. I also have additional concerns regarding this component, which I discuss in the questions section.
    • Single-step diffusion with a discriminator may not previously have been applied to pathology, but it has been developed and used in other domains. I am willing to credit the authors for applying this idea in the pathology setting.
    • Domain-specific encoders have been used multiple times for virtual staining, including as feature encoders or through representation losses intended to provide more domain-specific information. This component therefore does not appear novel.
    • Training at lower resolution followed by fine-tuning at higher resolution is a well-established training paradigm across many domains and applications. It likewise should not be presented as a novel contribution.

Overall, I find the novelty argument of the paper weak. The work feels more like a combination of existing ideas that achieves modest improvements over prior methods, rather than a substantively new methodological contribution.

  1. Relevant ablations are missing.

    1. The claim in Line 228 does not appear to be sufficiently substantiated. Is it based on an empirical observation, and are there experimental results supporting it? The main relevant evidence seems to be Table 5, which shows that different pathology encoders—UNI-2, Virchow, and PathCLIP—have some effect, but the differences do not appear substantial. It is therefore unclear how strongly the claim can be made.

    2. To support the claim that these domain-specific embeddings provide information comparable to that of a seasoned pathologist, it would be valuable to compare them with non-domain-specific encoders, such as ResNet, ConvNeXt, or other general-purpose encoders. If those encoders provide similar benefits, then the claimed importance of domain-specific representations should be tempered and presented more neutrally.

  2. Missing relevant comparisons.

  • MCS-Stain [1] appears to be the closest GAN-based counterpart, as it uses pathology-pretrained encoders for cell segmentation, yet it is not included in the experimental comparisons.

  • M2PL-GAN [2] also appears to be a relevant baseline for comparison.

  • No Flow based models are included as baselines ? Flow based models tend to match generation capabilities of diffusion models with substantially less inference cost, so those seem as relevant baselines as well.

[1] Hu, Yihuang, et al. “MCS-Stain: Boosting FFPE-to-HE Virtual Staining With Multiple Cell Semantics.” IEEE Transactions on Medical Imaging, 2025.

[2] Liu, Zequn, et al. “M2PL-GAN: Multi-View Multi-Level Pathology Semantic Perception Learning for H&E-to-IHC Virtual Staining.” IEEE Transactions on Medical Imaging, 2026.

  1. Important results and analysis are confined to the supplementary material. Several central analyses, including the DAB-channel analysis, limitations, and future-work discussion, appear only in the supplementary material and are not referenced in the main paper. As a result, the paper can feel incomplete even when relevant answers exist in the supplement. Since reviewers are not required to review supplementary material, important observations and experiments may be missed. I therefore believe the paper requires substantial restructuring or rewriting to bring its key analyses, limitations, and conclusions into the main paper and sharpen its overall focus.
Questions for authors

Questions for the Authors:

  • Table 3 suggests that removing (L_{\mathrm{LPIPS}}) has the largest effect on performance. However, LPIPS is not pathology-specific; it primarily measures perceptual similarity between source and target images. In contrast, removing the prior or expert component does not appear to reduce performance by a comparably large margin. This raises the question of whether the reported gains are primarily attributable to the pathology-pretrained expert, or instead to the particular combination of losses. For example, would LPIPS combined with the expert component account for most of the improvement?

  • The model rankings in Table 2 do not appear to align with those in Table 1. Higher PSNR or SSIM, and lower KID or FID, do not seem consistently associated with better downstream IHC classification performance. This raises a question about how the two evaluations should be interpreted: if the trends are not correlated, which table should be considered more reliable for judging model quality? At a minimum, the paper should discuss why improved image-similarity metrics do not translate into improved downstream classification performance, identify the limitations of the current evaluation, and clarify how readers should interpret these results with appropriate caution.

  • In the comparison with VIM, which is presented as the closest prior method, is the model trained at (512 \times 512), as in the original paper, or retrained from scratch at (1024 \times 1024) for a fair comparison? Further, what would happen if VIMs were trained at (512 \times 512) and then fine-tuned at (1024 \times 1024), following the proposed two-stage strategy? Would this yield better results and reduce the apparent contribution of the proposed training procedure?

Minor Problems or Suggestions for Improvement:

  • Line 083 states, “The standard benchmark resolution on which GAN …”. This appears overly broad for pathology and conflates image size with image resolution. While many public benchmarks may use (1024 \times 1024)-pixel images, pathology resolution is generally specified in micrometers per pixel and varies with scanner magnification, such as (40\times), (20\times), or (10\times). The statement should be made more precise.

  • The first paragraph of the related-work section, beginning at Line 134, cites recent GAN-based methods from 2024–2026, but then uses a 2022 citation to support the claim that GAN-based generation produces low-quality images. Is this conclusion still justified in light of the newer models cited immediately before it? In addition, the cited study concerns natural images. It is unclear whether the same conclusion can be directly extended to other domains, including pathology, radiology, or MRI, without domain-specific evidence.

Reviewer_veYcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how to achieve high-quality, high-resolution virtual staining of pathology images using single-step diffusion models. The authors propose MagicStain, which incorporates pathology-specific priors from the PathCLIP foundation model, together with pathological and structural consistency losses to improve virtual staining fidelity. Experiments are conducted on eight virtual staining datasets, with comparisons against various GAN-based and diffusion-based methods. The results demonstrate that MagicStain achieves promising performance in image quality, structural similarity, and inference efficiency.

Critical strengths
  1. The main strength of this paper is its incorporation of pathology-specific knowledge to address the limitations of existing single-step diffusion models in understanding pathological content. The authors introduce a specialized pathology foundation model to provide feature guidance and semantic constraints, while employing structural consistency losses on the hematoxylin channel to enhance the preservation of cellular morphology. These designs are well motivated by the requirements of pathology image virtual staining.
  2. The paper provides a comprehensive experimental evaluation across eight datasets, covering different staining translation tasks and biomarkers. In addition to standard image quality metrics, including FID, KID, PSNR, and SSIM, the experiments include ablation studies, cross-dataset validation, a downstream HER2 classification task, and expert evaluations on a small number of whole-slide images.
Critical weaknesses

The proposed method largely builds upon the established single-step image-to-image translation framework. It incorporates a pretrained pathology foundation model to provide conditional features and employs a feature similarity loss to enhance the pathological consistency of generated images. Although the integration of these components yields strong empirical performance, similar strategies have already been extensively explored in the literature. Therefore, the main contribution appears to be a carefully engineered integration of existing techniques for a specific application, rather than a generative modeling approach that introduces a fundamentally new mechanism.

Questions for authors

N/A

Reviewer_ECkzOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MagicStain, a single-step diffusion framework built upon SD-Turbo and pix2pix-Turbo for 1024 by 1024 pathology virtual staining. To adapt single-step diffusion to histopathology, MagicStain uses a pretrained pathology foundation model, PathCLIP, to extract source-image embeddings for cross-attention conditioning in the denoising U-Net and applies a cosine similarity loss between PathCLIP embeddings of the generated and ground-truth target images. It further imposes mean squared error and perceptual losses on the hematoxylin channel separated via Ruifrok color deconvolution to preserve nuclear morphology. For 1024 by 1024 synthesis, the authors introduce a two-stage training protocol that fully fine-tunes the U-Net on 512 by 512 patches for 15 epochs and then updates only LoRA weights on 1024 by 1024 patches for 5 epochs. The model is evaluated across eight virtual staining datasets and a downstream HER2 membrane staining intensity classification task.

Critical strengths

Adapting adversarial single-step diffusion to 1024 by 1024 histopathology virtual staining achieves a strong practical balance between inference speed, taking 0.43 seconds per patch, and visual quality. The empirical evaluation is broad, covering eight paired datasets across multiple biomarkers as well as downstream HER2 scoring and pathology foundation model feature similarity metrics.

Critical weaknesses
  1. The narrative exhibits formulaic, over-assertive writing habits where speculative or factually incorrect explanations are repeatedly stated as established truths, undermining the readability and rigor of the paper. Despite the AI Use Statement claiming no generative AI was used for writing, the text frequently relies on repetitive, boilerplate assertions that directly contradict the paper's own technical setup and appendix numbers. For example, Section 3.5 on line 295 asserts that models trained on 512 by 512 images cannot generalize to 1024 by 1024 images due to incompatible latent spaces. Technically, the SD-Turbo variational autoencoder is a fully convolutional network that is kept completely frozen throughout both stages of MagicStain, meaning its latent space is strictly translation-equivariant and shares identical channel semantics at 512 and 1024 resolutions. Similarly, the Introduction and Figure 1 repeatedly assert that the baseline VIMs fails to generate brown cancerous regions because it lacks domain expertise, when the paper's own ablation tables prove that removing domain expertise from MagicStain barely changes its performance at all.

  2. The headline comparison against the primary baseline pix2pix-Turbo / VIMs in Figure 1, Table 1, and Table 12 is severely distorted by an unfair training configuration, which exaggerates the claimed contribution of the proposed modules. By construction, MagicStain is pix2pix-Turbo augmented with PathCLIP conditioning and loss, hematoxylin channel loss, and two-stage training. Looking at the ablation study on H&E2IHC-HER2 in Table 3 on page 9, removing the PathCLIP prior changes FID only slightly from 33.51 to 37.48, removing the expert loss gives 35.60, removing the hematoxylin MSE loss gives 35.85, and training directly at 1024 resolution without the 512 stage gives 35.01. Even removing any of the newly proposed components leaves the model with an FID around 35 to 44. Yet in Table 1 on page 8 and Table 12 on page 23, the baseline pix2pix-Turbo / VIMs is reported as collapsing completely, with an FID of 225.2 on H&E2IHC-HER2 and 303.1 on BCI. Table 16 on page 25 reveals the actual cause of this discrepancy: the baseline VIMs / pix2pix-Turbo was trained with only 9.505 million trainable LoRA parameters from scratch for 240.27 hours, whereas MagicStain performs full-parameter fine-tuning of all 866.4 million U-Net parameters in Stage 1 for 20 epochs, taking 24.90 hours in total. The authors' own Table 6 on page 20 proves that using LoRA in Stage 1 fails to learn virtual staining and degrades FID to 77.36. Blaming the 225.2 versus 33.51 FID gap in Figure 1 and Table 1 on a lack of pathology priors, when it is actually caused by training the baseline with LoRA-only updates while giving MagicStain full U-Net fine-tuning, is a major experimental flaw.

  3. The spatial resolution mismatch in PathCLIP supervision and the necessity of the two-stage progressive training are poorly substantiated. In Section 3.2, PathCLIP, which is a vision transformer pretrained at 224 by 224 resolution, is applied to 1024 by 1024 images to compute a single global cosine similarity scalar in Equation 1 and to extract source embeddings for U-Net cross-attention. The paper never explains whether the 1024 by 1024 images are downsampled by more than four times to 224 by 224 before entering PathCLIP, which would blur out fine cellular and membrane structures, nor how a single globally pooled cosine similarity scalar can supervise localized brown DAB staining patterns across a 1024 by 1024 image. That the global PathCLIP loss provides weak spatial guidance for diagnostic DAB expression is further confirmed by Table 18 on page 26, where explicitly supervising the diagnostic brown DAB channel yields no improvement over supervising only the nuclear hematoxylin channel. Furthermore, Table 3 shows that training directly at 1024 resolution without the 512 stage achieves 35.01 FID, 18.41 PSNR, and 0.408 SSIM, which is nearly identical to the two-stage model's 33.51 FID, 18.58 PSNR, and 0.411 SSIM, and full fine-tuning in Stage 2 without LoRA actually achieves a higher SSIM of 0.417 than using LoRA at 0.411.

Questions for authors
  1. Clarification request on the pix2pix-Turbo / VIMs baseline fairness: In Table 3, MagicStain without the PathCLIP prior achieves 37.48 FID, and without Stage 1 achieves 35.01 FID, whereas pix2pix-Turbo / VIMs in Table 1 has 225.2 FID and takes 240.27 hours to train with only 9.5M LoRA parameters in Table 16. What is the exact performance of vanilla pix2pix-Turbo, without PathCLIP and without hematoxylin channel losses, when trained under the exact same 20-epoch full-U-Net fine-tuning schedule as MagicStain?

  2. Clarification request on PathCLIP input processing and color deconvolution: How are 1024 by 1024 images resized or tiled when fed into the 224 by 224 PathCLIP image encoder at both the cross-attention input and the expert loss in Equation 1? In addition, how are numerical instabilities in the logarithmic optical density conversion of Ruifrok color deconvolution handled during backpropagation in Equation 3, and what stain matrix vectors are used for the FFPE-to-H&E task?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.