acceptodds
Under review as a conference paper at ICLR 2027

Post-Extraction Compression of Pathology Foundation-Model Embeddings: Storage, Fidelity, and Downstream Utility

Abstract

Whole-slide learning and retrieval often reuse large stores of patch embeddings extracted by a frozen pathology foundation model. We evaluate compression of 1,294,852 CONCH v1.5 embeddings from 1,701 TCGA glioma slides (877 patients) at magnification. We measure stored bytes, CPU decode time, reconstruction and retrieval distortion, attention stability, and three-class multiple-instance learning (MIL) under patient-disjoint evaluation. Per-channel INT4 achieves measured compression and a five-seed mean macro-AUROC of , compared with for FP32. The patient-clustered 95% interval for the difference between five-seed ensembles is , above the specified engineering margin. Adding one packed residual-sign bit per coordinate lowers mean attention total variation from to and increases top-5% attention-set Jaccard overlap from to , while reducing compression to . Normalization followed by Haar rotation reduces the largest recorded coordinate by , but does not reduce maximum empirical excess kurtosis; the measured per-channel rotated decoder takes seconds per bag, compared with ms for INT4. We derive a bag-size-independent bound on gated-attention perturbation. These results support low-bit storage for this encoder, cohort, and MIL model while showing that attention fidelity and decode cost require separate evaluation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Histopathology, Compression, Embeddings, Quantization, Retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_CxAfOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper evaluates the compression of pathology foundation model embeddings, examining storage efficiency, reconstruction fidelity, decoding latency, and downstream task performance. Experiments on TCGA glioma data demonstrate that channel-wise INT4 compression achieves an 8.95× compression ratio while maintaining Multiple Instance Learning (MIL) performance comparable to FP32; residual-sign correction further enhances attention mechanism fidelity but incurs additional storage and decoding overhead. Furthermore, the paper establishes theoretical bounds regarding attention mechanisms and prediction stability under compression.

Critical strengths
  1. The paper provides a comprehensive evaluation of compression efficiency, retrieval fidelity, attention stability, and downstream performance.
  2. The experiments are rigorous, using patient-disjoint splits, multiple seeds, and bootstrap confidence intervals.
  3. The paper demonstrates substantial storage savings with minimal performance degradation and provides theoretical stability bounds.
Critical weaknesses
  1. The methodological novelty is limited, as the paper primarily evaluates existing quantization and rotation techniques.
  2. The experiments are limited to a single foundation model, dataset, and MIL architecture, raising concerns about generalizability.
  3. The paper is difficult to follow, particularly in the abstract, which presents numerous technical details and experimental results without clearly motivating the problem or highlighting the main contributions.
Questions for authors

What is the main methodological contribution of this work beyond evaluating existing quantization and compression techniques?

Reviewer_DCgVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the quantization compression of embeddings of the whole slide pathology images extracted from the frozen foundation models. The quantization strategies explored include FP16, INT8, INT4, residual-sign correction, and rotation-based quantization. Experiments are conducted on CONCH v1.5 embeddings extracted from 1,701 TCGA glioma whole-slide images. The evaluation considers storage efficiency, reconstruction fidelity, retrieval consistency, attention stability, computational overhead, and multiple-instance learning (MIL) classification.

The main findings are:

  • INT4 quantization achieves an 8.95× compression ratio while maintaining comparable classification performance (macro-AUROC 0.8400 versus 0.8364 for FP32).
  • Residual-sign correction improves attention consistency, reducing attention total variation from 0.0305 to 0.0166, at the expense of compression efficiency.
  • Rotation-based quantization provides limited practical benefits.
Critical strengths

1:The paper demonstrates that low-bit quantization can reduce pathology embedding storage without significantly degrading downstream classification performance. 2:The paper provides complete understanding of compression trade-offs by evaluating the outcomes from different perspectives of retrieval consistency, attention stability, storage efficiency, and decoding latency. 3: The derivation of a bag-size-independent perturbation bound for gated attention provides a theoretical perspective on how embedding quantization may affect downstream MIL predictions.

Critical weaknesses

1: The compression methods primarily rely on quantization techniques. The contributions mainly rely on empirical studies, and there is no technique innovation presented. 2: The empirical studies were based on the CONCH foundation model and the TCGA glioma dataset. Whether the same conclusions can be found on other settings, such as the use of different foundation models and other datasets, is not clear. Different staining types of images such as the HE stained or the IHC stained have completely different image characters. Without thorough evaluations, the conclusion have limited reference value for being adopted in general practical applications. 3: In image analysis, there are different approaches for compression or dimension reduction. How woud be the results if different compression approaches are used?

Questions for authors

1: The experiments results were reported based on the overall TCGA data set. Please report the individual results conducted on different diseases and also on the different staining data sets. 2: In practical applications, the model may be used for classification, detection and segmentation. For the different types of tasks, how would be the results? 3: On some applications such as segmentation of cells with small objects, what would the results? Would you draw a same conclusion?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.