WEAVE: Adaptive Patchification for Gigapixel Whole-Slide Images

Transformer-based architectures for computer vision, such as Vision Transformers, require patchification, which is a process in which the input image is split into patches and mapped to tokens. In contrast to uniform patchification, adaptive patchification allocates more tokens to information-rich image regions. While adaptive patchification is effective for natural images, extending this approach to high-resolution images, such as gigapixel whole-slide images (WSIs) in histopathology, introduces new challenges. Uniform patchification can produce prohibitively long token sequences, making slide-level aggregation expensive, particularly for quadratic-cost architectures such as Transformers. Adaptive patchification can reduce this cost by allocating tokens according to image content, but highly redundant and long-tailed WSI feature statistics destabilize existing scoring mechanisms. We propose $\text{W}$SI $\text{E}\text{nergy}$-based $\text{A}$daptive $\text{V}$ision $\text{E}$ncoder ($\mathrm{WEAVE}$), an adaptive patchification method for gigapixel WSIs. We introduce a Hopfield energy-based scoring function that provides stable importance estimates under WSI feature statistics and a lightweight mLSTM encoder for subquadratic token aggregation. Together, these components enable content-adaptive tokenization at gigapixel scale and achieve competitive or improved performance across five WSI classification benchmarks and much higher efficiency.