acceptodds
Under review as a conference paper at ICLR 2027

Structured Generative Modeling for Pixel-Level Multiplex Protein Prediction from H&E with ProDICE

Abstract

Multiplex spatial proteomics enables spatially resolved characterization of protein expression but remains costly and difficult to scale, whereas hematoxylin and eosin (H&E) imaging is routinely collected and widely available. Existing histology-to-spatial-omics methods predict gene expression at either the multi-cell spot level or the single-cell level, with single-cell prediction typically requiring cell segmentation to delineate cells before molecular profiles can be assigned. Pixel-level approaches avoid this requirement, but most produce deterministic predictions for individual protein channels rather than modeling the joint conditional distribution of the multiplex protein panel, limiting their ability to capture cross-protein dependencies. We introduce tein iffusion from mage-onditioned mbeddings), a conditional generative framework for pixel-level prediction of multiplex protein expression from H&E. performs diffusion by modeling the joint conditional protein distribution given histological features, enabling the simultaneous prediction of spatially organized protein maps while preserving cross-protein structure. We evaluate on held-out subjects from colorectal cancer and heterogeneous multi-tumor cohorts against four competing methods. achieves the best overall performance across Pearson and Spearman correlation, SSIM, Wasserstein distribution distance, and variance retention, while better preserving protein-protein correlation structure. We further demonstrate clinical relevance of predicted protein profiles in downstream prediction tasks on TCGA-CRC. These results demonstrate the potential of joint generative modeling for scalable virtual spatial proteomics. Code is available at https://anonymous.4open.science/r/prodice-iclr2027-review-25CD.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
spatial proteomics, diffusion, generative modeling, multimodal learning, computational pathology, histopathology, spatial omics, computer vision

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_a64oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the problem of predicting pixel-level multiplex spatial protein expression from routine H&E histology images and proposes a conditional diffusion framework that models the full multiplex protein panel jointly. The method performs diffusion directly in the normalized protein-pixel space on 384×384 tiles. At inference, full regions are reconstructed from overlapping tiles combined by Hann-weighted overlap-add. Experiments are conducted on a colorectal cancer CODEX cohort and a multi-tumor CODEX cohort against four baselines, with an additional exploratory downstream evaluation on TCGA-CRC whole-slide images.

Critical strengths
  1. The studied problem is meaningful. Multiplex spatial proteomics is costly and difficult to scale, whereas H&E is routinely available.
  2. Applying generative modeling to this domain is interesting. Treating the multiplex protein panel as a single structured generative target is a different perspective from existing deterministic methods such as ROSIE and HEX.
Critical weaknesses
  1. My main concern is the scalability of both the method and its experimental validation. In practice, H&E whole-slide images are gigapixel-scale, while the model operates on 384×384 tiles (~145 µm), and the reconstruction benchmark is limited to 720×960 regions. Inference is also expensive since each tile requires 50 denoising steps, and each region is covered by heavily overlapping tiles (stride 96, 75% overlap). No runtime or memory cost is reported. Moreover, the TCGA analysis does not demonstrate pixel-level whole-slide prediction.
  2. The choice of image encoders is not well justified. I wonder if we can directly apply a pathology foundation model (e.g., UNI, CONCH, Virchow). Given that the training cohorts are small (e.g., 24 training subjects for CRC), an encoder trained from scratch is unlikely to generalize well across tissues, stains, and institutions. At a minimum, I suggest an ablation study that replaces the encoders with a pathology foundation model, or a simple pathology-FM-based baseline.
  3. The ablation shows that removing the global ResNet-50 branch changes raw Pearson by only 0.001, which suggests the current global features contribute little.
  4. The intuition behind Hann-weighted overlap-add is not introduced. The paper does not explain why it chose this blending scheme over alternatives (e.g., uniform averaging or non-overlapping tiling), and it provides no ablation on stride or window size.
  5. It seems that the baselines are too weak. ROSIE achieves a variance ratio of 0.001 on CRC (Table 1), which is nearly equivalent to outputting a constant map. HEX obtains a negative Pearson correlation on Multi-Tumor.
Questions for authors

See Weaknesses.

3 comments
AuthorsOpenReview ↗

Scalability, Conditioning, Reconstruction, and Stronger Baselines

Response to Reviewer a64o

We thank the reviewer for the constructive feedback and for recognizing the importance of virtual spatial proteomics and joint generative modeling. To directly address the concerns, we conducted additional experiments on (W1) computational cost and scalability, (W2) pathology foundation model encoders, (W3) global morphology conditioning, (W4) reconstruction strategies and stride selection, and (W5) stronger deterministic and generative baselines. We present the results below. Tables are numbered consistently across our responses to all reviewers.

W1: Scalability of the method and its validation.

Response: We agree that the submission should have reported cost.

Table R7. Sampling time per region under the CRC validation reconstruction protocol (one NVIDIA L4, batch size 1; ±\pm SD where available).

ConfigurationDenoiser evaluationsTiles / regionRuntime / region
ProDICE, 50 steps, stride 96 (default)1,75035136.56±1.48136.56 \pm 1.48 s
ProDICE, 25 steps, stride 968753568.19 s
ProDICE, 50 steps, stride 1926001246.70±0.4546.70 \pm 0.45 s
ProDICE, 25 steps, stride 1923001223.42 s
Deterministic U-Net, stride 9635353.41 s

Runtime is proportional to the number of denoiser evaluations (tiles ×\times steps), so ProDICE's extra cost over a deterministic network comes from its sampling steps rather than from a heavier network. A separate repeated pure-inference timing of the default setting gave 137.51±0.04137.51 \pm 0.04 s per region, consistent with Table R7, and peak allocated GPU memory was about 779 MiB at 50 steps and batch size 1. The cheaper settings change the output: halving the steps lowers the variance ratio slightly (0.2807→0.26220.2807 \rightarrow 0.2622) with no detected change in pointwise metrics or SSIM. Doubling the stride reduces tile blending, which raises the variance ratio to about 0.46 and lowers Pearson to about 0.20 (W4). Dense pixel-level whole-slide inference therefore remains expensive with the current sampler, although tiles are independent and parallelize across GPUs, and step distillation could reduce the number of sampling steps further.

The TCGA analyses ran tiled inference at cohort scale, with the earlier mixing-enabled checkpoint in the paper (710,812 tiles from 388 slides; Sec. 4.5) and since with the final model (49,205 tiles), but in both each tile was reduced to its mean predicted intensities. They show that cohort-scale tiled inference and feature extraction are feasible. We have added the cost analysis to App. H ("Runtime, stride, and sampling steps") and the Limitations.

W2: Choice of image encoders and pathology foundation models.

Response: Thank you for the suggestion. We agree that pathology-specific pretrained representations may generalize better than generic ImageNet features, particularly with limited paired H&E–CODEX data.

One clarification: the global encoder is not trained from scratch. It is a frozen ImageNet-pretrained ResNet-50 with a trainable projection; the only image encoder trained from scratch is the small local convolutional encoder.

We tested the suggested substitution, replacing the frozen ResNet-50 with frozen UNI, CONCH, or Virchow while keeping the rest of the model fixed:

Table R8. Global-encoder substitutions on CRC validation (16 regions, 4 subjects; seed 42; same training schedule and epoch-69 EMA checkpoint for every row).

Global encoderPearson ↑\uparrowNative 2D SSIM ↑\uparrowRMSE ↓\downarrow
ImageNet ResNet-50 (submitted design)0.24990.24920.1589
CONCH v10.28040.25530.1563
UNI v10.27790.26650.1565
Virchow v10.27160.25080.1574

All three pathology foundation models improve validation Pearson over ResNet-50, with CONCH achieving the largest gain (+0.0305) and UNI the highest SSIM. We originally selected ResNet-50 as a simple, established pretrained encoder to focus on evaluating the proposed joint diffusion framework rather than optimizing the image backbone. The additional results show that pathology-specific encoders can further improve prediction accuracy without changing ProDICE's core architecture.

We also evaluated frozen pathology foundation model embeddings with PCA/Ridge regression on Multi-Tumor.

Table R9. Frozen pathology-FM linear probes on Multi-Tumor (66 cores, five outer folds).

ModelGlobal-raw Pearson ↑\uparrow
CONCH v1 + PCA/Ridge0.1507
UNI v1 + PCA/Ridge0.1676
Virchow v1 + PCA/Ridge0.1676
H&E Feature Ridge0.1680
ProDICE0.2255

These frozen pathology foundation model features perform similarly to H&E Feature Ridge under a linear readout and remain below ProDICE. Together, these experiments show that ProDICE can benefit from stronger pretrained image representations, while its performance is not explained solely by the choice of image encoder.

W3: Removing the global ResNet-50 branch changes raw Pearson by only 0.001.

Response: We agree that the global ResNet-50 branch has little effect on raw Pearson correlation. However, it provides broader morphological context that complements the spatially aligned features from the local encoder. As shown in Table R10, removing this branch reduces region-centered Pearson by 0.008±0.0020.008 \pm 0.002 and SSIM by 0.010±0.0030.010 \pm 0.003, and worsens Wasserstein distance across all three training seeds. These results indicate that the global branch contributes primarily to within-region spatial prediction rather than overall Pearson correlation.

Table R10. Conditioning ablations on CRC validation. Deltas are ablation minus the mixing-enabled reference (App. C) across three training seeds; negative Pearson/SSIM and positive Wasserstein deltas indicate degradation.

Removed componentΔ\Delta raw PearsonΔ\Delta centered PearsonΔ\Delta SSIMΔW1\Delta W_1
Global morphology tokens−0.0008-0.0008−0.0079-0.0079−0.0099-0.0099+0.0013
Morphology-conditioned protein queries−0.0028-0.0028−0.0024-0.0024+0.0015−0.0007-0.0007
Dense cross-marker mixing−0.0010-0.0010−0.0013-0.0013−0.0002-0.0002+0.0000

W4: Hann-weighted overlap-add, alternatives, and stride or window size.

Response: Thank you for identifying this missing explanation. We use Hann-weighted overlap-add to reduce the contribution of tile boundaries, where convolutional context and padding effects may differ from central regions.

Each tile's prediction is least reliable near its borders, where the denoiser has the least context, and neighboring tiles are sampled independently, so switching abruptly from one tile to the next would leave seams. A Hann window gives each tile's center the most weight and tapers toward its edges (Eq. 7, App. F.4), so each output pixel is dominated by tiles in which it lies near the center and transitions between tiles are smooth.

The alternatives trade off differently. Uniform averaging gives tile-border pixels the same weight as tile centers and averages more tiles per pixel, which suppresses more of the sampled variation. Non-overlapping tiling keeps one sample per pixel but places every tile border, where predictions are weakest, along fixed seams. Stride sets how much blending occurs:

Table R11. Reconstruction weighting and stride on CRC validation (16 regions from 4 subjects; seed 42; 50 DDIM steps; tissue-supported evaluation). All rows reuse the same 35 sampled tiles (the stride-192 rows use their 12-tile subset), so differences reflect reconstruction alone. NeffN_{\mathrm{eff}} is the Kish effective number of contributing tiles per pixel, (∑kwk)2/∑kwk2(\sum_k w_k)^2/\sum_k w_k^2, a property of the weights rather than a count of independent observations. Matrix rr and Frobenius error are donor-pooled and not comparable with the region-wise values in Tables R1, R3, R5, and R12; ProDICE's Val-16 values in Tables R1 and R8 come from separate sampling and evaluation runs.

ReconstructionStrideNeffN_{\mathrm{eff}}Pearson ↑\uparrowRMSE ↓\downarrowNative 2D SSIM ↑\uparrowVariance ratioW1W_1 ↓\downarrowMatrix rr ↑\uparrowMatrix Frobenius error ↓\downarrow
Disjoint assignment (one tile per pixel)961.000.17970.18190.22040.60860.02990.88150.0099
Hann overlap-add (default)964.180.25340.15830.23130.28070.04770.87500.0136
Uniform averaging968.020.28270.15340.23070.21690.05320.86340.0179
Hann overlap-add1921.630.20270.17150.22400.45620.03660.88210.0100
Uniform averaging1922.690.22720.16390.22310.35260.04220.88020.0111

At stride 96, uniform averaging gives the best pointwise scores (Pearson 0.2827, RMSE 0.1534) but the lowest variance ratio (0.2169) and the worst W1W_1 and correlation-matrix agreement. Disjoint assignment retains the most variance (0.6086) and has the best W1W_1 at the lowest Pearson (0.1797). Hann weighting lies between them, with SSIM nearly tied with uniform averaging and better W1W_1, matrix rr, and Frobenius error (0.0477 vs. 0.0532; 0.8750 vs. 0.8634; 0.0136 vs. 0.0179). Moving from stride 96 to 192 reduces blending, raises the Hann variance ratio to 0.456, lowers Pearson to 0.203, and cuts sampling time 2.9×2.9\times (Table R7). Independently resampled stride-192 tiles give nearly identical values (Hann: Pearson 0.202, variance ratio 0.458; uniform: 0.227 and 0.354). Stride therefore trades pointwise accuracy against variance and cost. Disjoint assignment still samples all 35 tiles, so it is not the cheaper non-overlapping tiling (six tiles at stride 384, about 23.6 s per region), whose output quality we did not evaluate. In the tested Hann configurations, we observed no MAE increase in an eight-pixel band around tile seams. The window size is tied to the 384×384384 \times 384 training tile, so we varied weighting, stride, and steps rather than retraining at other tile sizes. We have added this explanation and the stride study to App. F.4 (Table 8).

W5: Baselines appear too weak.

Response: We agree that the weak results warrant careful examination and that performance under one adaptation protocol should not be interpreted as each method's best attainable performance. ROSIE and HEX were both fine-tuned on our training splits and kept their native footprints and reconstruction (Sec. 4.3; App. G.2), and every method's reconstructed full-resolution maps were scored by the same evaluator. Checking these baselines, we found that the full-canvas ROSIE and HEX inference behind Table 1 used inputs that differ from their training preprocessing (response to Reviewer 1qce, Q4), which accounts for ROSIE's variance ratio of 0.001 and HEX's negative Multi-Tumor Pearson. With training-matched inference on a common tissue support, ROSIE is essentially tied with ProDICE on CRC Pearson (0.204 vs. 0.209) and Spearman, while ProDICE retains lower W1W_1 (0.045 vs. 0.083) and a higher variance ratio (0.276 vs. at most 0.082). The revised Table 1 reports both protocols (Table R6), and we also corrected an export inconsistency in the supplementary per-marker tables.

Because performance under one adaptation protocol should not be read as a method's best attainable performance, we also added three stronger comparisons, all on the same 28 held-out CRC test regions:

  • Matched deterministic U-Net (Table R1, response to Reviewer Qq5x), matched to ProDICE in conditioning, output channels, tile reconstruction, and evaluation pixels. It beats ProDICE on Pearson (0.3077 vs. 0.2262) and RMSE (0.1441 vs. 0.1558), while ProDICE retains more variance (0.3216 vs. 0.1833), has lower W1W_1 (0.0390 vs. 0.0539), and better reproduces the inter-marker correlation matrix (MAE 0.1200 vs. 0.2976, lower in all 28 regions). The same pattern holds across three training seeds and on Multi-Tumor (Table R12), and adding variance-matched noise to the U-Net does not close the gap without losing most of its spatial fidelity (Table R5, response to Reviewer 1qce). The two models also differ in training objective and number of updates (the U-Net received about half as many), but averaging eight ProDICE draws reaches the U-Net's pointwise scores on a six-region subset (Table R4, response to Reviewer 1qce), so the pointwise gap is consistent with sampling versus point estimation rather than a weaker network.
  • STFlow, a recent flow-matching method, has higher Pearson (0.2493 vs. 0.2262) and lower RMSE (0.1459 vs. 0.1558) than ProDICE, but compresses variance much more (variance ratio 0.0842 vs. 0.3216) and has higher W1W_1 (0.0699 vs. 0.0390; Table R3, response to Reviewer 1qce).
  • DiffVS (Oh et al., AAAI 2026), a marker-wise conditioned diffusion method for virtual multiplex staining and a more directly relevant generative comparator than ROSIE or HEX. ProDICE has higher Pearson (0.2262 vs. 0.1699), lower W1W_1 (0.0390 vs. 0.1100), a higher variance ratio (0.3216 vs. 0.1640), and closer agreement with the measured inter-marker correlation matrices (rr = 0.7476 vs. 0.4565; Table R3).

We have added these comparisons, the training-matched ROSIE and HEX results, and the matched U-Net results to the revision (Table 1; Apps. G.3, H, and I) and updated our claims accordingly.

We hope these additions address the reviewer's concerns and are happy to provide further details.

Reviewer_Qq5xOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ProDICE, a conditional diffusion framework for predicting pixel-level multiplex protein expression maps directly from H&E histology images. Unlike conventional deterministic approaches, ProDICE models the complete multiplex protein panel jointly to better capture protein-specific spatial distributions and inter-protein dependencies. The framework performs diffusion directly in normalized protein-pixel space using a multichannel denoising U-Net. It combines spatially aligned local H&E features with global morphology features extracted by a pretrained ResNet-50 encoder. Additionally, learnable protein-specific queries attend to histological features to provide marker-adaptive conditioning. During inference, the model generates protein maps through iterative denoising, and overlapping tile predictions are combined to reconstruct larger tissue regions. Experiments on two multiplex CODEX cohorts (a colorectal cancer dataset and a heterogeneous multi-tumor dataset) demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper models the full multiplex protein panel jointly rather than treating each protein as an independent prediction problem is meaningful even though its architectural elements are not fundamentally new.

  2. ProDICE achieves consistent improvements on most structural and distributional metrics, and the authors provide paired subject-level bootstrap confidence intervals to assess reliability. The analysis of protein–protein correlation matrices is particularly relevant to the paper's motivation, as it directly examines whether the predicted panel preserves dependencies between protein channels.

Critical weaknesses
  1. The paper claims that modeling the full conditional protein distribution with diffusion produces better predictions and preserves protein–protein dependencies. Therefore, my main concern is that insufficient evidence that diffusion is necessary or superior to a strong deterministic multichannel predictor. The main comparisons are against ROSIE, HEX, a simple ridge-regression model, and a mean-prior model. While ROSIE and HEX are relevant existing approaches, they differ from ProDICE in backbone, image representation, spatial prediction footprint, architecture, and reconstruction pipeline. Therefore, the reported improvements cannot be attributed specifically to generative modeling.

  2. Although ProDICE achieves improved correlations, spatial similarity, and distributional fidelity relative to the evaluated baselines, its absolute prediction performance remains limited. Pearson correlations are approximately 0.22 on both cohorts, and the predicted variance ratios of 0.307 and 0.266 remain substantially below the ideal value of 1. Moreover, a simple ridge-regression baseline achieves lower RMSE on both datasets. The exploratory downstream experiments also provide limited support for the clinical utility of the predicted proteins: virtual-protein representations generally do not outperform H&E-only features, fusion improvements are inconsistent, and correlations with directly matched measured RPPA proteins are very weak.

  3. My last concern is that the submission exceeds the 9-page limit, with two lines extending onto Page 10.

Questions for authors

Q1: What is the specific benefit of diffusion compared with a matched deterministic network?

Q2: Does ProDICE learn a meaningful conditional distribution?

Q3: How should the modest absolute accuracy and inconsistent downstream benefits be interpreted?

Q4: Is the gain in prediction quality worth the additional computational cost?

5 comments

Response to Reviewer Qq5x

We thank the reviewer for the constructive feedback and thoughtful assessment of our work, particularly for recognizing the value of joint multiplex protein modeling and the importance of evaluating cross-protein dependencies. We appreciate the opportunity to clarify our contributions and strengthen the empirical evidence. In response, we provide (Q1) a matched deterministic U-Net comparison to directly assess the contribution of diffusion, (Q2) analyses of whether ProDICE learns a meaningful conditional distribution, (Q3) further discussion of absolute prediction accuracy and downstream TCGA results, and (Q4) a quantitative comparison of computational cost. We also address the page-limit concern (W3).

Q1: What is the specific benefit of diffusion compared with a matched deterministic network? (W1)

Response: We agree with the reviewer that comparisons with ROSIE, HEX, and Ridge alone cannot isolate the contribution of diffusion, given differences in architecture, pretraining, prediction resolution, and reconstruction procedures. To directly address this concern, we trained a matched deterministic multichannel U-Net with the same H&E conditioning, output channels, tile reconstruction, and evaluation pixels as ProDICE. We evaluated both models on the CRC validation set and, importantly, on all 28 held-out CRC test regions.

Table R1. ProDICE vs. the matched deterministic U-Net (seed 42). Val-16: 16 validation regions from 4 subjects, with both models evaluated on the same tissue mask. Test-28: all 28 held-out test regions from 7 subjects, unmasked.

MetricVal-16 ProDICEVal-16 U-NetTest-28 ProDICETest-28 U-Net
Pearson rr (↑\uparrow)0.26120.34830.22620.3077
Spearman (↑\uparrow)——0.25590.3186
RMSE (↓\downarrow)0.15940.14900.15580.1441
Native 2D SSIM (↑\uparrow)0.24830.2752——
Variance ratio (ideal = 1)0.27440.14350.32160.1833
Wasserstein W1W_1 (↓\downarrow)0.04430.05920.03900.0539
Inter-marker correlation-matrix rr (↑\uparrow)0.79020.65020.74760.6433
Inter-marker correlation-matrix MAE (↓\downarrow)0.10140.27410.12000.2976
Mean absolute inter-marker correlation (measured: 0.1473)——0.21920.4125

The matched comparison reveals a clear trade-off between pointwise prediction accuracy and preservation of protein–protein dependencies:

  • Pointwise accuracy versus protein–protein dependencies: The deterministic U-Net achieves higher pixel-level Pearson correlation and lower RMSE, as well as higher SSIM on the validation set. However, ProDICE retains substantially more spatial variance, better matches the measured protein expression distributions, and more accurately preserves protein–protein correlations. On the 28 held-out test regions, ProDICE achieves lower inter-marker correlation-matrix MAE in all 28 regions and higher matrix correlation in 26 of 28 regions, demonstrating that these improvements extend beyond the validation data. The same pattern holds across three training seeds and on the Multi-Tumor cohort (Table R12 below).
  • Why deterministic prediction can distort protein–protein correlations: Under squared-error loss, a deterministic predictor estimates the conditional mean, averaging over protein expression patterns compatible with a given H&E image [1,2]. Although this minimizes expected pointwise error, it can suppress spatial variation not uniquely determined by morphology. This smoothing can also distort relationships among predicted protein channels. On the held-out test regions, the U-Net substantially overestimates the mean absolute protein–protein correlation (0.4125 versus 0.1473 measured), whereas ProDICE is considerably closer (0.2192). This trade-off is also observed within ProDICE: averaging eight independent samples improves Pearson correlation and RMSE to levels comparable to the deterministic U-Net (Pearson: 0.3265 vs. 0.3302; RMSE: 0.1460 vs. 0.1461), but increases the inter-marker correlation-matrix MAE from 0.1030 to 0.2195 (six validation regions; Table R4 in our response to Reviewer 1qce). These findings indicate that optimizing pointwise accuracy alone does not necessarily preserve protein–protein dependencies.

  • Why the improvement is not simply due to added noise: To examine whether ProDICE's advantage arises merely from greater output variance, we added noise to the deterministic U-Net predictions to restore their variance. Although these noise controls achieve inter-marker correlation-matrix MAE values comparable to ProDICE (0.0899–0.1182 vs. 0.1014; ranges across noise constructions, not one jointly selected control), they show lower agreement in correlation-matrix structure (matrix rr: 0.6580–0.7472 vs. 0.7902) and substantially worse spatial and pointwise accuracy (SSIM: 0.0272–0.1172 vs. 0.2483; Pearson: 0.1301–0.1370 vs. 0.2424 on the same subject-macro estimand; Val-16; Table R5 in our response to Reviewer 1qce). Thus, adding variance alone does not reproduce ProDICE's combination of spatial prediction accuracy and preservation of protein–protein dependencies.

To check that this trade-off does not depend on one training seed or one cohort, we repeated the matched comparison with three training seeds on CRC validation and with the five-fold design on Multi-Tumor (Table R12).

Table R12. Replication of the matched comparison. CRC: mean (±\pm SD where available) over three training seeds, all evaluated on the same 16 tissue-masked validation regions from 4 subjects, so the seeds are not additional biological replicates. Multi-Tumor: pooled out-of-fold maps for 66 cores (five outer folds, seed 42); Pearson is tissue-masked (unmasked: ProDICE 0.2255, U-Net 0.2725). Inter-marker matrix statistics are computed region-wise.

MetricCRC ProDICECRC U-NetMulti-Tumor ProDICEMulti-Tumor U-Net
Pearson rr (↑\uparrow)0.2563±0.00900.2563 \pm 0.00900.3458±0.00430.3458 \pm 0.00430.20010.2420
RMSE (↓\downarrow)0.15900.14920.16840.1601
Native 2D SSIM (↑\uparrow)0.24100.27000.14560.1816
Variance ratio (ideal = 1)0.2592±0.02070.2592 \pm 0.02070.1357±0.00700.1357 \pm 0.00700.26420.1499
Wasserstein W1W_1 (↓\downarrow)0.04610.06040.05390.0692
Inter-marker correlation-matrix rr (↑\uparrow)0.78730.64460.69040.5901
Inter-marker correlation-matrix MAE (↓\downarrow)0.1027±0.00160.1027 \pm 0.00160.2851±0.01220.2851 \pm 0.01220.15620.3366

Every metric moves in the same direction as in Table R1, in both cohorts. These results therefore establish a replicated trade-off between the two trained systems.

In summary, these results demonstrate that the principal benefit of ProDICE over the matched deterministic U-Net is not higher pixel-level prediction accuracy, but better preservation of measured protein expression distributions and protein–protein dependencies. This distinction is important for analyses involving multiple protein markers, such as co-expression and co-localization analyses, where distorted protein–protein relationships could lead to misleading conclusions. It also motivates evaluating protein predictions beyond Pearson correlation and RMSE alone [1,3,4]. We have added these comparisons to the main Results (Sec. 5.1) with full tables in App. H (Tables 15–17), and revised the Abstract, Introduction, and Discussion to state this trade-off.


Comment Q2: Does ProDICE learn a meaningful conditional distribution, rather than adding arbitrary noise or inducing correlation through stitching? (W1)

Response: We appreciate this important question. We provide five complementary analyses showing that ProDICE captures measured protein expression distributions and protein–protein dependencies, and that its stochastic predictions exhibit coupling across markers.

  1. Preservation of measured protein expression distributions: On the held-out unmasked benchmark, ProDICE achieves lower Wasserstein distance than Ridge on both CRC (W1W_1 = 0.041 vs. 0.076) and Multi-Tumor (0.050 vs. 0.087; Table 1), despite Ridge achieving lower RMSE (Table 10). This advantage also holds against the matched deterministic U-Net on both the CRC validation and held-out test regions (Table R1). These results demonstrate that ProDICE more closely reproduces the measured protein expression distributions rather than simply minimizing pointwise prediction error.

  2. Preservation of protein–protein dependencies: Because matching marginals does not guarantee preserved relationships between proteins, we compared full spatial protein–protein correlation matrices. ProDICE agrees with the measured matrices at rspatialr_{\mathrm{spatial}} = 0.794±0.0040.794 \pm 0.004 across three training seeds on the 16 validation regions (Sec. 5.3), and at rr = 0.7476 versus 0.6433 for the matched U-Net on the 28 test regions (Table R1), and at rr = 0.6904 versus 0.5901 on Multi-Tumor (Table R12). These findings provide evidence that ProDICE better preserves relationships among protein channels, beyond matching their individual expression distributions.

  3. Joint stochastic sampling across protein markers: To test whether markers are sampled jointly rather than with independent per-channel noise, we analyzed repeated samples across six validation regions. Independently shuffling sample labels across markers preserves each marker's sampled distribution while disrupting the correspondence between channels. This reduced the model-residual correlation statistic from 0.811 to −0.063-0.063, and decreased agreement with the measured protein–protein correlation matrices from rr = 0.7959 to 0.7608. These results indicate that the stochastic variation is coupled across protein channels and that this coupling contributes to reproducing the measured correlation structure. Notably, correlation-matrix MAE decreased after shuffling (0.1039 to 0.0870), reflecting reduced overall correlation magnitudes. We therefore interpret these findings as evidence of joint stochastic sampling.

  4. Correlation structure is not created by stitching: Using the same sampled tiles, we assigned each pixel to a single tile (disjoint assignment, no blending). Agreement with the measured correlation matrices is slightly higher without blending (donor-pooled matrix rr = 0.881 vs. 0.875 for the default Hann output; matrix Frobenius error 0.0099 vs. 0.0136), and the variance ratio more than doubles (0.609 vs. 0.281; Table R11 in our response to Reviewer a64o). Blending therefore moves the correlation structure slightly away from the measured one rather than producing the agreement.

  5. Uncertainty calibration: Consistent with variance ratios below 1, the sampled distribution is under-dispersed. Eight-draw min–max intervals cover 27.09% of measured pixels on the 16 validation regions, against 77.78% expected for exchangeable draws, and nominal 90% intervals from 32 draws cover 33.48% on six regions (details in our response to Reviewer 1qce, Q1(c)).

In sum, these results demonstrate that ProDICE not only better preserves measured protein expression distributions and protein–protein dependencies, but also captures joint stochastic variation across protein channels. Together with the noise-control experiments in Q1, these findings support that ProDICE learns meaningful structure in the conditional protein distribution rather than merely introducing arbitrary noise.


Reviewer_1qceOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper predicts pixel-level multiplex protein images (CODEX, 56–58 markers) from H&E. Its method, ProDICE, is a conditional diffusion model that generates all protein channels jointly in pixel space, guided by H&E features. On a colorectal cancer cohort and a multi-tumor cohort, ProDICE beats ridge regression, ROSIE and HEX on most metrics (Pearson about 0.22 vs at most 0.19). It also preserves protein–protein correlations better. An exploratory TCGA study uses the predicted proteins for MSI and stage prediction.

Critical strengths

S1. Careful and transparent evaluation.

  • Leakage controls. Splits are by subject, and the architecture was fixed on CRC validation data only. Multi-Tumor uses outer cross-validation, with model selection inside each fold.
  • Uncertainty. Uncertainty is estimated with a subject-level bootstrap with paired comparisons, plus a leave-one-test-subject-out sensitivity analysis (Table 10).
  • Rule-based qualitative examples. The examples in App. Fig. 3 are chosen by a pre-set percentile rule, and full per-marker tables are provided (but see Q4 on whether they match Table 1).

S2. Consistent gains and a useful panel-level view. ProDICE beats all comparators on macro-averaged Pearson, Spearman and SSIM in both cohorts, with paired confidence intervals that exclude zero. Checking whether the predicted panel keeps the measured protein–protein correlation pattern is also a useful diagnostic: deterministic predictors produce over-correlated channels (Fig. 2E–F). (See Q1 on how much of ProDICE’s advantage here may come from sampling noise.)

Critical weaknesses

W1. Limited novelty relative to prior diffusion-based work on predicting molecular data from H&E.

  • The paper claims ProDICE is “the first diffusion method in the area of pixel-level histology-to-spatial-CODEX prediction” (Sec. 1). However, diffusion-based H&E-to-multiplex virtual staining already exists: Oh et al., “Virtual Multiplex Staining for Histological Images Using a Marker-wise Conditioned Diffusion Model” (AAAI 2026). Restricting the claim to CODEX does not change the learning problem. Compared with Oh et al., two differences remain: pixel-space vs. latent diffusion, and joint vs. marker-by-marker generation. Generating the whole molecular panel jointly is already standard in diffusion and flow models for spatial transcriptomics. Examples are Stem (Zhu et al., ICLR 2025) and STFlow (Huang et al., “Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching”, ICML 2025). STFlow models the joint distribution of expression across an entire slide. There are also existing prior works that achieve similar "pixel-space" level prediction, such as "Magnet: Multi-level attention graph network for predicting high-resolution spatial transcriptomics." (MIDL 2025).
  • The training and sampling recipe (v-prediction, zero-terminal-SNR cosine schedule, DDIM) follows standard pipeline in flow-matching models. The conditioning components are also established designs: frozen ImageNet ResNet-50 tokens with cross-attention, learned per-marker queries, and FiLM-style output modulation.

W2. The downstream study does not establish a meaningful use case for ProDICE, yet the abstract claims to “demonstrate clinical relevance”.

  • ProDICE’s averaged protein predictions are worse than the H&E (ResNet-50) features in all eight MSI and stage comparisons (four classifiers × two endpoints; App. D, Table 5). For MSI, ROC-AUC is 0.52–0.63 vs 0.66–0.69; for stage, 0.49–0.53 vs 0.55–0.58. On nodal status and depth of invasion, both feature sets are near chance (0.47–0.58).
  • Combining the two does not meaningfully help. Early fusion changes AUC by at most 0.012 on MSI and stage (Table 5). The only nominally significant gain (N-status, elastic net, +0.039; Fig. 7D) does not hold across classifiers and is not corrected for multiple comparisons.
  • Predicted case-level protein levels correlate weakly with measured RPPA (macro-averaged r = 0.062 over 14 markers; App. D, Fig. 7C). This is partly expected by design. The model is trained on intensities normalized within each region, so case-level averages should not be expected to track absolute bulk protein levels. This makes the downstream setup a poor test of molecular relevance.
Questions for authors
  1. The output blends about 16 independent tile samples per pixel, so it is neither a single sample nor the model’s mean. Averaging samples pulls any generative model toward its conditional mean, so the advantages in variance ratio, Wasserstein distance and inter-marker correlation should shrink as more samples are averaged. The real question is whether the extra variance in a single sample is placed correctly. Please report:

    • Pearson, Spearman and RMSE for the average of N independent full-region runs (N = 4, 16, 32; N = 1 is the current output), compared with the deterministic baselines.
    • Variance ratio, W1 and inter-marker correlation structure for a single unblended sample (e.g., from non-overlapping tiles). Compare these with a deterministic baseline plus noise matched to the measured variance (e.g., Ridge + residual noise).
    • Calibration of the predicted uncertainty, e.g., coverage of per-pixel 90% intervals from multiple samples, or CRPS.
  2. Physical scale appears only in App. E (20×, 0.377 µm/pixel). Please also state it in Sec. 4: a region covers about 272 × 362 µm and a tile about 145 µm. For TCGA, were tiles resampled to the training resolution? TCGA slides were scanned at both 20× and 40×, and different imaging machine offer slightly different physical scale correspondence even though they are all called as "20×" or "40×". A fixed 512 to 384 resize of full-resolution tiles gives about 0.33 µm/pixel for 40× slides but about 0.67 µm/pixel for 20× slides.

  3. What is the source of the Multi-Tumor cohort, and under what access and consent terms is it used? App. E states that donor-level records are “not recoverable from the available artifacts.” The TCIA collection cited for CRC (Schürch et al., 2020a) also lists a multi-tumor tissue microarray, imaged on the same Keyence BZ-X710 at the same resolution. Is that the source?

  4. Per-marker tables vs. Table 1. The per-marker tables appear inconsistent with the main results.

    • Table 8 shows ROSIE and HEX above ProDICE on many CRC markers (e.g., CD11b: 0.355 and 0.323 vs 0.295; CD25: 0.291 and 0.247 vs 0.223). Table 9 shows the same for Multi-Tumor (e.g., CD117, CD163).
    • Averaging Table 8 gives a Pearson of about 0.17 for ROSIE and 0.13 for HEX, vs 0.104 and 0.021 in Table 1. ROSIE’s averaged per-marker variance ratio is about 0.16, vs 0.001 in Table 1.
    • This also conflicts with App. B.1, which says ProDICE beats ROSIE and HEX on 55 of 56 CRC markers and all 58 Multi-Tumor markers.
    • Which numbers are correct, and how does the baseline evaluation differ between the tables?
6 comments

Response to Reviewer 1qce

We thank the reviewer for the detailed and constructive assessment, particularly for recognizing the importance of subject-level evaluation, statistical uncertainty, and panel-level correlation analysis. We agree that the submission needs clearer positioning against prior generative methods, a more careful account of what the sampled outputs are, and three corrections: to the ROSIE and HEX results (both the supplementary per-marker tables and the Table 1 rows), to the stated physical pixel scale, and to how we described the Multi-Tumor cohort. We address each point below. Tables are numbered across our responses to all reviewers (Tables R1 and R12 are in our response to Reviewer Qq5x).

W1: Limited novelty relative to prior diffusion-based work.

Response: We agree that diffusion-based virtual staining, joint generation of molecular panels, and high-resolution histology-to-omics prediction each already exist, and that our "first diffusion method" wording in Sec. 1 was too broad (we have removed it).

We do not claim that the individual training components, including v-prediction, zero-terminal-SNR scheduling, DDIM sampling, cross-attention, learned protein queries, or FiLM-style modulation, are novel. Rather, ProDICE's contribution lies in jointly generating dense multiplex protein images directly in protein-pixel space, without a separately trained target encoder or decoder, while preserving spatial protein patterns and protein–protein dependencies.

ProDICE differs from prior work in three ways:

Table R2. Closest prior work.

MethodModelingOutputKey Distinction
DiffVS Oh et al. (AAAI 2026)Pre-trained then fine-tuned conditional latent diffusionUp to 18 mIF images (one per generation)ProDICE generates all 56–58 channels together in pixel space, so they share one stochastic state
Stem (ICLR 2025)Spot-wise conditional diffusionPer-spot gene-expression vectorsSpot-level vectors rather than dense multi-channel images
STFlow (ICML 2025)Whole-slide flow matchingWhole-slide spot gene expressionModels joint distribution over spots and not pixels
MagNet (MIDL 2025)Deterministic graph-attention regressionVisium HD bin-level gene expressionProDICE generates stochastic multichannel protein maps
ProDICEH&E-conditioned pixel-space diffusion56–58 mIF protein channels jointlyJoint pixel-level generation without target-space compression
  1. Pixel-level prediction without predefined cellular or spot boundaries. Spot-level expression conflates expression of multiple cells within the probe's range, so spot-level methods can only train on and predict an aggregate expression that is unstable and dependent on probe locations. Predicting every pixel of a region instead lets ProDICE represent extracellular and other non-cellular signal, avoids dependence on cell segmentation, uses context from all tissue pixels rather than only segmented cells, and outputs maps at the resolution of the protein images.

  2. Joint generation of multiplex protein channels. When each marker is produced by a separate generation, the channels of a predicted panel do not share a stochastic state. In ProDICE all channels are denoised in one state, and its draws are coupled across markers (Q1(b) below). This allows for better capture of protein–protein correlations and is consistent with experiments showing closer agreement with the measured correlation structure than the marker-wise DiffVS (Table R3). Stem and STFlow also generate jointly, but over spot-level vectors rather than images in which every channel shares pixel-level spatial structure.

  3. Direct diffusion in protein-pixel space. Latent diffusion typically requires a two-stage framework in which a target representation is learned before the cross-modal conditional model. Our pixel-space formulation is single-stage, which simplifies training and design. In our experiments, a latent-diffusion pipeline with a protein autoencoder that we evaluated on the same validation regions reproduced the measured inter-marker structure less well (three-seed means: matrix rr 0.678 vs. 0.787; MAE 0.213 vs. 0.103).

Comparisons with STFlow and DiffVS. We adapted both methods to the CRC CODEX benchmark and evaluated them on the same 28 held-out regions, 7 subjects, and 56 markers as ProDICE, with the same truth, metric definitions, and unmasked support (Table R3).

Table R3. ProDICE vs. DiffVS and STFlow on the 28 held-out CRC test regions (seed 42 for all three; the ProDICE column is the run in Table R1). The two SSIM rows are different estimators (global image moments vs. native 2D maps) and are not interchangeable. —: not available on matched support (STFlow's inter-marker statistics were computed on a tissue-masked support, so we omit them).

MetricProDICEDiffVSSTFlow
Pearson rr (↑\uparrow)0.22620.16990.2493
Spearman (↑\uparrow)0.25590.19880.2791
RMSE (↓\downarrow)0.15580.16780.1459
Global-moment SSIM (↑\uparrow)0.20090.10520.1621
Native 2D SSIM (↑\uparrow)0.23080.0876—
Variance ratio (ideal = 1)0.32160.16400.0842
Wasserstein W1W_1 (↓\downarrow)0.03900.11000.0699
Inter-marker correlation-matrix rr (↑\uparrow)0.74760.4565—
Inter-marker correlation-matrix normalized Frobenius error (↓\downarrow)0.02070.0458—
Mean absolute inter-marker correlation (measured: 0.1473)0.21920.3842—

ProDICE is better than DiffVS on every metric. STFlow has higher Pearson and Spearman and lower RMSE than ProDICE, while ProDICE has higher SSIM, lower W1W_1, and retains far more spatial variance (variance ratio 0.3216 vs. 0.0842), mirroring the trade-off with the matched U-Net (Table R1). Both DiffVS and ProDICE over-estimate the magnitude of inter-marker correlation, ProDICE much less so (0.2192 vs. 0.3842; measured 0.1473). This is consistent with the joint-generation argument above, since DiffVS generates each marker in a separate pass.

Comparison with Stem. Stem generates one expression vector per spatial-transcriptomics spot, so applying it directly at native pixel resolution requires a separate generative trajectory for every pixel, whereas ProDICE generates all channels of a 384×384384 \times 384 tile in one trajectory. We implemented and profiled such a native-pixel adaptation, keeping Stem's 12-block diffusion transformer (width 384, six attention heads), 56 marker tokens, and 1,000-step ancestral sampling. One 720×960720 \times 960 region then requires 691,200 pixel-level vectors, or about 1.35 million batched transformer forward passes at batch size 512. Profiling on an NVIDIA RTX A5500 measured 7.34 pixels/s, with 99.8% of the time in diffusion sampling (histology features were cached), which projects to about 26.2 GPU-hours per region; an optimized implementation on an NVIDIA H100 reached 34.3 pixels/s, or about 5.6 GPU-hours per region. By linear extrapolation, scoring the 28 test regions would take about 730 A5500 or 157 H100 GPU-hours per seed. A 100-step DDIM sampler was about 10×10\times faster in a sparse pilot but gave worse errors and many out-of-range outputs, so we did not substitute it for the original protocol. Stem is practical at the spot-level resolution it was designed for, and we do not claim it cannot be accelerated or redesigned for dense prediction. Doing so, however, would require changes we could not establish as equivalent to the published method, so we report this as a feasibility measurement rather than an accuracy comparison.

W2: The downstream study does not establish a meaningful use case.

Response: We agree, and the reviewer's summary of the results is accurate: virtual proteins alone were below the H&E features in all eight MSI and stage comparisons, and the nominal nodal-status gain was not corrected for multiple comparisons. The original phrasing of "demonstrates clinical relevance" overstated these results. We have revised the Abstract and Contributions to describe the TCGA analysis as an exploratory feasibility test of cohort-scale tiled inference, consistent with the main text and Limitations.

However, as the reviewer notes, the weak RPPA concordance is partly expected by design. More generally, the setup is a weak test of ProDICE's spatial output, for three reasons (detailed in our response to Reviewer Qq5x, Q3):

Interpretation and Evaluation Limitations:

  1. Relative rather than absolute targets. Each protein channel is percentile-normalized independently within each tissue region (1st–99.5th percentiles). Consequently, predictions represent relative spatial intensities, not calibrated between-patient protein concentrations. As such, per-region percentile normalization largely removes the between-patient abundance differences that bulk RPPA measures and that carry substantial signal for MSI.

  2. Spatial-to-case averaging. The TCGA analysis averages predicted maps into one protein vector per case. This discards within-tissue localization and co-variation, which are the primary properties motivating a dense spatial model. Thus, a case mean of predictions derived from the same H&E would not be expected to outperform the H&E features themselves.

  3. Bulk molecular reference mismatch. The RPPA comparison includes only 14 proteins with a direct counterpart in the RPPA panel and compares case-level bulk proteomics with virtual predictions from diagnostic H&E sections that were not exact tissue matches to the RPPA specimens. RPPA validly measures bulk protein abundance, but this particular comparison is not a spatial ground-truth validation, and we have removed it from the revision.

  4. Domain shift. TCGA differs from the paired CODEX training cohort in scanner, physical pixel size, preparation, and morphology (Q2 below). The CRC model was trained on paired H&E–CODEX data from 24 donors (96 training regions), whereas the exploratory transfer used 388 TCGA diagnostic whole-slide images without paired spatial-protein ground truth. The contribution of each domain-shift factor to prediction error remains unquantified. An outcome-blinded random sampling of slide images identified pathologist pen markings in 29 of 36 inspected slides with usable masks.

Due to time constraints, the TCGA features were also extracted with a previous version of ProDICE that preceded the final architecture (Sec. 4.5), so they do not test the submitted model. We will update the main manuscript with the latest results.

Q1: Averaging, single samples, noise-matched baselines, and calibration.

Response: We agree with the reviewer's reading of the output: Hann-weighted overlap-add of independently sampled tiles is neither a single draw nor the conditional mean. Under Hann weights at stride 96, the effective number of contributing tiles per pixel (Kish, Neff=(∑kwk)2/∑kwk2N_{\mathrm{eff}} = (\sum_k w_k)^2/\sum_k w_k^2) is 4.18, compared with 8.02 for uniform averaging and 1 for disjoint tiles (response to Reviewer a64o, W4). Although interior pixels are covered by many tiles, the Hann weights concentrate on tile centers, so in terms of weight concentration the submitted output corresponds to roughly four equally weighted tile samples per pixel (NeffN_{\mathrm{eff}} summarizes the weights).

We conducted additional analyses to examine the effects of sample averaging, tile blending, added noise, and uncertainty calibration.

(a) Effect of averaging NN full-region runs.

Table R4. Pixelwise means of NN independent full-region reconstructions vs. the matched deterministic U-Net, on 6 tissue-masked CRC validation regions from 4 subjects. —: not computed on this subset.

OutputPearson ↑\uparrowSpearman ↑\uparrowRMSE ↓\downarrowVariance ratio (ideal = 1)Inter-marker MAE ↓\downarrow
ProDICE, NN = 10.24490.25730.15660.31410.1030
ProDICE, mean of NN = 40.30760.30470.14770.18810.1854
ProDICE, mean of NN = 80.32650.31920.14600.16760.2195
Deterministic U-Net0.3302—0.14610.16040.2865

The results confirm the reviewer's hypothesis: averaging moves ProDICE onto the deterministic model. At NN = 8, Pearson (0.3265 vs. 0.3302) and RMSE (0.1460 vs. 0.1461) essentially match the U-Net, while the variance ratio falls from 0.3141 to 0.1676 (U-Net 0.1604) and the inter-marker MAE roughly doubles (0.2195, still below the U-Net's 0.2865). The advantages in variance and cross-marker structure therefore belong to individual draws, and the pointwise gap to deterministic models is what averaging recovers. One trained model thus provides both outputs: single draws for distributional and cross-marker analyses, and draw averages for pointwise accuracy, at proportionally higher computational cost (response to Reviewer a64o, W1). We report N≤8N \leq 8 only, since at NN = 8 the pointwise metrics already match the deterministic U-Net.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.