acceptodds
Under review as a conference paper at ICLR 2027

Tokenize the Physics, Not the Pixels: TessTok and Fold-Flow Quantization for Material Microstructures

11 views

Abstract

Visual tokenizers compress images into the tokens that latent generative models learn from. Existing tokenizers are built for natural images: a fixed latent grid, a codebook or straight-through estimator, and a perceptual objective. A physical field is instead a set of objects that a simulation reads directly. *Can a discrete tokenizer compress a material microstructure field by two orders of magnitude and return a field a crystal-plasticity solver can run?* We present **TessTok**, which encodes a polycrystal as sparse quantized Laguerre grain seeds and regrows it deterministically. Its **Fold-Flow Quantizer (FFQ)** needs no codebook or straight-through estimator and learns where to place its levels. At kbit per field reconstruction, TessTok reaches dB and rFID and matches of the grains, against dB, and for ViT-VQGAN at kbit. At matched bits, FFQ beats every codebook and binary quantizer on all pixel and orientation metrics with no codebook to collapse. TessTok also keeps of grain area at kbit, below the rate of MAGVIT-v2. On ImageNet, with its grain-specific parts switched off, the same code gives the highest PSNR among the latest tokenizers, though not the best perceptual scores and compression rate. Tokenizing physical-object fields instead of pixels, lets our model work at many bit budgets, and lets a reconstruction be governed by the physical simulation needs.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
visual tokenizer, vector quantization, Laguerre tessellation, representation learning, inverse design, materials science

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NxZ6OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TessTok, a tokenizer for microstructure orientation maps that represents an image using quantized tessellation seeds containing position, size, orientation, and shape information. It introduces Fold-Flow Quantization (FFQ), which combines a learned monotone warp with an iterated smooth folding operation, and an existence gate that enables inference-time rate control. The evaluation covers reconstruction quality, grain geometry, and orientation statistics on synthetic magnesium-alloy microstructures, together with an ImageNet experiment. The central idea is to make the compressed representation reflect the objects underlying the physical field.

Critical strengths
  1. The structural prior is well motivated by the application. Tessellation seeds are a reasonable representation for microstructure maps with approximately constant orientation within each grain. The compressed variables have interpretable connections to grain geometry and crystallographic orientation.

  2. The method supports inference-time rate control. Adjusting the fraction of retained seeds allows a single trained model to operate at different compression budgets without retraining. The rate–fidelity sweep demonstrates this flexibility experimentally.

Critical weaknesses
  1. The experiments focus on compression and reconstruction, without demonstrating benefits for simulation or inverse design. Compared with fine-tuned ViT-VQGAN, TessTok improves PSNR by 0.75 dB and matched grain area from 0.912 to 0.927. The practical significance of these gains for the intended physical applications remains unclear.

  2. The main comparison uses natural-image tokenizers without domain adaptation, while TessTok is trained on microstructure data. The fine-tuned results in the appendix narrow the gap substantially and should be the primary comparison. Training budgets, trainable parameters, and objectives also need to be comparable.

  3. The ablations do not clearly isolate the contributions of individual components. In particular, FSQ and other quantizers receive 60 training epochs, while FFQ uses the full staged schedule. FFQ gains 0.07 dB in PSNR and improves orientation statistics, but performs worse on rFID and matched grain area. Controlled comparisons are needed to establish its advantage.

Questions for authors
  1. Can you compare FSQ and FFQ under the same training schedule, objective, and rate accounting? An identity-warp FFQ ablation would help separate the effects of smooth folding and learned quantization-level placement.

  2. Can you report orientation errors between spatially corresponding pixels or matched grains, with a clear definition of grain matching? Global orientation statistics may not capture whether orientations are preserved at the correct locations.

  3. Is the intended input to the physical solver the hard tessellation render or the final neural reconstruction? If it is the latter, what ensures constant orientation within each grain and prevents additional boundaries?

3 comments
AuthorsOpenReview ↗

Response to Reviewer NxZ6: solver test, fine-tuned baselines as the main comparison, and FSQ vs. FFQ under one schedule

Thank you for finding the tessellation seed prior well motivated.

W1 "benefits for simulation or inverse design..." / Q3 "..intended input to the physical solver..."

Intended and tested solver input: The decoded field is nearest-upsampled to 300² and segmented by SAM into grains, each grain gets the per-component median of its pixels' Euler angles, so this step gives one piecewise constant orientation per grain. Its grain boundaries agree with the original's exact ones (F1 = 0.926; ViT-VQGAN 0.906). The resulting grain map goes to the open-source crystal-plasticity solver DAMASK [1] (as in [2,3]), which turns it into a stress–strain curve. Material strength (from this curve) depends on each grain's orientation to the load, so decode errors become strength errors. We ran it on 60 test fields (AZ31, Mg-5Gd alloys under 25 % tension) (Sec. 4.4). Median % error vs the original (exact grains) of yield stress σy\sigma_y, tensile strength σu\sigma_u and hardening exponent nn; curve RMS (MPa). Bits: one index-entropy estimate for all rows. TessTok's decoder now reads only the transmitted seeds and all TessTok numbers here outside Tab. 3 come from this model, trained for 75 epochs.

bitsσy\sigma_y (%)σu\sigma_u (%)nn (%)RMS 0–25 % (MPa)
TessTok11,9060.590.590.671.26
ViT-VQGAN12,3970.740.740.831.76
WeTok7,6020.920.970.932.14
MAGVIT-v24,3001.221.241.152.80
floor: 256² original, same steps0.360.150.160.33

TessTok has the lowest error in all four columns, at 4 % fewer bits than ViT-VQGAN. The gaps to ViT-VQGAN are significant for σy\sigma_y, σu\sigma_u and RMS (paired Wilcoxon, p=0.008p = 0.008, 0.0450.045, 0.0310.031). This is the practical significance you asked about: TessTok's PSNR is now 0.17 dB below ViT-VQGAN's, yet its solver error is lower, because the solver reads grains and their orientations, where TessTok is more accurate (per-grain error 24.3° vs 26.8°, matched area 0.948 vs 0.912). No inverse-design loop is run in this study, but such a loop repeats this step [4].

W2 "fine-tuned results … should be the primary comparison"

Following your recommendation, we make it the main comparison (Tab. 1) in the revised PDF. With 4 % fewer bits than ViT-VQGAN, TessTok is better on rFID (8.87 vs 13.56), SSIM, MS-SSIM and every grain metric (per-grain error 24.32° vs 26.79°, matched area 0.948 vs 0.912, mean-orientation error 2.18° vs 2.33° for all 15 alloy families), and slightly behind on PSNR (23.43 vs 23.61 dB). WeTok and MAGVIT-v2 use fewer bits and trail on every metric. For the baselines we take the public pretrained weights and fine-tune them on the microstructure train data for 30 epochs. TessTok's seed bottleneck has no pretrained weights and is trained from scratch on the same pretrained ViT ends for 75 epochs, with added MSE, rate/seed-count and grain-seed loss terms; Tab. 8 lists budget, trained weights and losses per method.

W3/Q1 "same training schedule"

Yes (Tab. 3): every row is trained to 75 epochs with the same recipe; only the quantizer (and, where marked, commitment) differs. Its rows send all 1,024 seed slots, hence more bits; the last row is our main model, which sends only the active seeds.

bitsPSNRrFIDper-grain°
FSQ (round + STE)21,95024.476.1326.36
FFQ, identity warp21,90324.605.6925.19
FFQ, identity warp + commit.21,64424.555.7725.13
FFQ, learned warp + commit.21,40524.555.9025.12
(only active seeds sent)
Main model: FFQ, learned warp + commit.11,90623.438.8724.32

The FFQ fold beats FSQ with rounding + straight-through estimator (STE) on rFID, PSNR, per-grain error and matched area. Compared image by image on the test set, the last three gaps are statistically significant (95 % confidence). The deficit you noted vanishes; Δθˉ\Delta\bar\theta ties (2.56° vs 2.55°). Row 3 is your identity-warp ablation where learned and identity warp agree within 0.01 dB and 0.02°, so we withdraw the level-placement claim; the contribution is the STE-free fold.

Q2 "orientation errors … matched grains"

SAM segments both original and reconstructed image, each original grain pairs with the reconstructed grain that overlaps it most (IoU ≥ 0.3). On the original's exact grains, per-grain error is 27.6° for TessTok vs 30.0° for ViT-VQGAN, matched area 0.882 vs 0.838. Without segmentation, the median pixel error is 22.8° vs 26.8°. All six matching variants agree: TessTok beats all the codebook and binary tokenizers in all 15 families (Tab. 15, App. H.1).

A revised PDF is uploaded with an additional page, containing fine-tuned comparison at equal rate, isolated fold, defined grain matching and a solver test added. With added evidence corresponding to your specific concerns, we would be grateful if you would consider raising your rating.

[1] Roters et al., Comput. Mater. Sci. 2019 [2] Tasan et al., Acta Mater. 2014 [3] Diehl et al., JOM 2017 [4] Kusampudi & Diehl, Int. J. Plast. 2023

Reviewer_NxZ6OpenReview ↗

Official Comment by NxZ6

After go through the author's feedback, I believe some of my concerns are well addressed in the rebuttal and I am glad to rise my rating to 3.

Thank you for reading the rebuttal and for raising your rating. Your suggestions on controlled comparisons and rate accounting have made the paper stronger. Two further results have come up in our experimentation since our response, and both are also in the general comment. First, set to ViT-VQGAN's rate (12,397 bits) without retraining, TessTok leads on PSNR ↑ (23.65 vs. 23.61 dB), rFID ↓ (8.45 vs. 13.56) and per-grain orientation error ↓ (24.0° vs. 26.8°). Second, we tested the variable seed count strength of TessTok in isolation. The same model, again without retraining, sends a fixed 552 seeds per field at the same bits. Against the variable count of the main model, every metric gets worse when we send fixed seeds (test images): PSNR ↑ 23.22 vs. 23.43 dB, rFID ↓ 11.76 vs. 8.87, per-grain error ↓ 24.8° vs. 24.3°. This helped us isolate the contribution of the content-adaptive seed count as a source of the TessTok gains.

Reviewer_EiEgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The goal of this paper is to compress material microstructure maps while preserving grain boundaries and crystal orientations needed for physical modeling. TessTok encodes a polycrystal as sparse quantized Laguerre grain seeds and re-grows it deterministically. For material microstructure encoding, instead of encoding just image patches, it encodes grain seeds containing position, size, shape, and crystal orientation (physics based properties)

Critical strengths

(+) although I've never worked on material micro-structure, the encoder design seems to be well-motivated to me, i.e. encode geometry and crystal orientation explicitly, providing a useful inductive bias for mateiral microstructure compression

(+) the results are reasonably good for the experiments in Table 1

Critical weaknesses

(-) actual usefulness is unknown. The paper only tested on synthetic data of the same family

(-) the result on imagenet is weak to moderate, limiting the method's generalizability

(-) in image generation, better reconstruction does not mean better generation. The paper does not seem to be tested on downstream applications.

Overall, I feel the contribution of this paper is limited.

Questions for authors

None

2 comments
AuthorsOpenReview ↗

Response to Reviewer EiEg: downstream solver test, unseen alloy family, and scope of the ImageNet result

Thank you for finding the Tesstok design well motivated (a "useful inductive bias").

W1a: "actual usefulness is unknown"

We now showcase a downstream task that uses this approach in crystal-plasticity simulations (Sec. 4.4, Tab. 5) where TessTok's reconstructions give the smallest error in every measured physical property below of all the four tokenizers tested, at no more bits than the best baseline.

First, each tokenizer's decoded 256² orientation map is nearest-upsampled to 300² and segmented into grains by SAM. Then, each grain gets the per-component median orientation of its pixels. This grain map goes to the open-source solver DAMASK [1], which stretches it to 25 % strain and returns a stress–strain curve. This is a standard way in materials science to predict a metal's strength from its microstructure [7,8].

Material strength depends on each grain's orientation to the load, so decoding errors become strength errors. We compare each curve with that of the original (300², exact grains) on 60 test fields (30 AZ31, 30 Mg-5Gd). Floor: the 256² original through the same steps (the pipeline's own error). Fine-tuned (ft) on our data (Tab. 1). Bits: one entropy estimate for all rows. TessTok's decoder now reads only the transmitted seeds and all TessTok numbers in W1a come from this headline model which is trained for 75 epochs. Median relative error of yield stress σy\sigma_y, tensile strength σu\sigma_u, hardening exponent nn and coefficient KK; curve RMS over 0–25 % strain:

InputBitsσy\sigma_y (%) ↓σu\sigma_u (%) ↓nn (%) ↓KK (%) ↓RMS (MPa) ↓
Floor (original)–0.360.150.160.160.33
TessTok (ours)11,9060.590.590.670.941.26
ViT-VQGAN (ft)12,3970.740.740.830.991.76
WeTok (ft)7,6020.920.970.931.432.14
MAGVIT-v2 (ft)4,3001.221.241.151.292.80

Tesstok's gains over ViT-VQGAN are significant for σy\sigma_y, σu\sigma_u and RMS (paired Wilcoxon, p=0.008p = 0.008, 0.0450.045, 0.0310.031), but not for nn and KK; all gaps to WeTok and MAGVIT-v2 are significant (p≤0.013p \le 0.013). This is the downstream usefulness of Tesstok tokenizer, where the solver reads grains and their orientations, which it reconstructs more accurately.

W1b: "only tested on synthetic data of the same family"

The data set spans 15 magnesium alloys with ~115 processing conditions, taken from experimental measurements, then oversampled and augmented. Upon your request, we now trained TessTok with and without ME21, the finest-grained one (same recipe, 30 epochs, one run each). The model without ME21 in train data, loses same fidelity on it as on the families it saw (Tab. 9; 95 % CI):

Trained without ME21ME21 (750)Others (9,350)Difference [CI]
PSNR (dB) ↑−0.133−0.155+0.021 [+0.016, +0.027]
per-grain disorientation ↓+0.18°+0.34°−0.16° [−0.21, −0.11]

Matched area does not change. Oversampled & augmented fields from measurements we used gave exact ground truth. Pure measured maps (EBSD) will be tested in future work, now its in the limitations.

W2: "the result on imagenet is weak to moderate"

We agree and we do not compete here. The ImageNet test only checks how much natural-image ability the same code keeps with its grain parts switched off, as Sec. 1 and 4.3 now state. It stays competitive: at ViT-VQGAN's rate (12.8 kbit) our retrained ImageNet model is within 0.12 dB PSNR (24.15 vs. 24.27) and better on rFID (4.41 vs. 4.93), behind on LPIPS (0.123 vs. 0.098) and SSIM (0.68 vs. 0.74). WeTok leads on LPIPS, SSIM and rFID at 8.2 kbit. We claim no state of the art here (Sec. 1, App. J) and the revised abstract states the scope prominently.

W3: "better reconstruction does not mean better generation"

We agree for image generation [2]. But, in many domains tokenizers also feed prediction: language models forecast tokenized time series [3], world models predict the next state as tokens [4], and simulation surrogates evolve a physical state in latent space [5]. These predictions have to be decoded back, so reconstruction enters every prediction. Joint-embedding predictive architectures also skip this decoding [6]. But the downstream solver task shown in W1a reads the decoded field directly, so there reconstruction fidelity matters. The revised Sec.1 and 2 make this clearer.

We uploaded a revised PDF that uses the one additional page with added evidence. It shows that TessTok's reconstructions give the smallest errors of all the four tokenizers tested in a crystal-plasticity simulation at no more bits than the best baseline, shows that an unseen family costs no more than seen ones, and states the ImageNet scope plainly. We hope this resolves your concerns and help you recommend acceptance.

[1] Roters et al., Comput. Mater. Sci. 2019 [2] Yao et al., CVPR 2025 [3] Ansari et al., TMLR 2024 [4] Micheli et al., ICLR 2023 [5] Wu et al., NeurIPS 2022 [6] Assran et al., CVPR 2023 [7] Tasan et al., Acta Mater. 2014 [8] Diehl et al., JOM 2017

AuthorsOpenReview ↗

Follow up to Reviewer EiEg

Dear Reviewer, As the discussion closes today, we would like to check whether our response and the revised PDF resolve your three concerns:

  • Usefulness and downstream test (W1a, W3): in a crystal-plasticity simulation of 60 test fields, TessTok's reconstructions give the lowest median error in every measured property of the four tokenizers tested (Sec. 4.4, Tab. 5).
  • Same family (W1b): a model trained without the ME21 family loses no more fidelity on it than on the families it saw (Tab. 9).
  • ImageNet (W2): Natural Image table js states as a check of retained natural-image ability, with no state-of-the-art claim (Sec. 1, 4.3). If any point remains open, we would be glad to answer it promptly. Thank you for taking out time in reading this.
Reviewer_9TkBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

TessTok compresses synthetic polycrystalline orientation maps using sparse quantized grain seeds, geometry-aware decoding, and an inference-time seed-retention control. Its Fold-Flow Quantizer (FFQ) combines repeated sinusoidal folds with a learned monotone warp, using hard indices for transmission. On 10,100 test fields, the paper reports 24.36 dB PSNR, 8.93 rFID, and 0.927 matched grain area at 10,426 bits, with a single-checkpoint sweep reaching 0.905 matched area at 4,365 bits. Additional experiments compare quantizers and decoders and evaluate a modified natural-image branch on an ImageNet subset.

Critical strengths
  1. Object-based tokenization is well motivated and retains an advantage after baseline adaptation: Tables 1 and 7 give TessTok 24.36 dB, 1.85° mean-orientation disorientation, and 0.927 matched area versus fine-tuned ViT-VQGAN’s 23.61 dB, 2.33°, and 0.912, respectively.
  2. One checkpoint supports a useful range of operating points: Table 12 reduces the reported rate from 10,426 to 4,365 bits while matched area changes from 0.927 to 0.905, with intermediate budgets explicitly evaluated.
  3. The evaluation extends beyond natural-image perceptual metrics: Appendix E defines orientation and grain-geometry measures, Table 6 reports all 15 alloy families, and Appendix M explicitly acknowledges the absence of end-to-end solver validation.
Critical weaknesses
  1. FFQ’s mathematical guarantees overstate the implemented relaxation. Proposition 1 claims C∞ smoothness despite the piecewise-linear warp in Section 3.3, which generally introduces derivative discontinuities. Appendix D’s five-fold worst-case contraction is also false: under Eq. (3), α = 1 and u = 0.49 give approximately 0.21236 after five folds; half-integers remain fixed. Finite folds therefore do not equal hard rounding. Algorithm 1 additionally rounds after folding, so the exact derivative of the soft map must be distinguished from the hard transmission path. Correct the proposition and specify the training forward/backward computation and measured soft–hard discrepancy.
  2. The evidence does not isolate FFQ’s improvement or support the abstract’s blanket superiority claim. Tables 2 and 9 explicitly compare 60-epoch competitors with FFQ’s different full schedule; Section 4.2 acknowledges the comparison is not budget matched. FSQ also has better rFID (8.44 versus 8.93) and matched area (0.945 versus 0.927). Retrain with matched schedules and rates, separating learned level placement from the folding surrogate. Sections 2 and Appendix F should also compare with STanH, TIP 2025, which already learns differentiable parametric quantization, rather than treating learned levels and differentiability as unique to FFQ.
  3. Solver-ready fidelity remains insufficiently demonstrated. Section 3.4 asserts a piecewise-constant output by construction, but Figure 2 places a learned ViT decoder after tessellation/adaptation without specifying an output constraint. Appendix M confirms no crystal-plasticity simulation was run. Moreover, Appendix E’s matched-area measure is geometric, while global orientation statistics cannot establish correct orientation assignment to individual grains. Specify which output constitutes the solver input, verify its constraints, and compare matched-grain orientation errors and downstream solver responses. SAM-derived geometry should also be checked against synthetic reference labels.
  4. Cross-method compression claims mix different coding conventions. Section 4.1 entropy-prices TessTok, uses fixed-width indices for other discrete tokenizers, and charges continuous latents at 32 bits per scalar. Table 11 shows entropy modeling reduces TessTok’s reported rate from 12,828 to 10,426 bits, a substantial part of its advantage. Compare entropy-coded baselines and a label-map/orientation-table or fitted-tessellation codec, and distinguish estimated code lengths from actual serialized round-trip sizes.
Questions for authors

See the weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 9TkB: corrected Prop. 1, matched quantizer ablation, solver test, and measured bitstream sizes

Thank you for checking Eq. (3) closely and for finding object-based tokenization "well motivated".

W1: "Correct the proposition and specify the training forward/backward computation…"

We agree that both of these were overstatements in our text. The method, even without these remains sound and no result depends on either statement. Every reported number is decoded from hard-rounded symbols. The fold is monotone, so training and transmission pick the same symbol: We measured it on the test set, none of the 23M transmitted values differ (new table, App. D). Both statements are now tightened:

  • Prop. 1: for α≤1\alpha\le1 the five-fold map is C∞C^\infty and monotone, as stated; composed with the piecewise-linear warp, qq is continuous and piecewise C∞C^\infty.
  • App. D: half-integers are fixed points, so finite folds are not rounding (0.49→0.2120.49\to0.212).

Training: the decoder reads the unrounded g−1(Φ(5)(u)/(L−1))g^{-1}(\Phi^{(5)}(u)/(L-1)) and the backward pass is its exact gradient, no straight-through estimator (STE). Transmission: i=round(u)i=\mathrm{round}(u), decoded as g−1(i/(L−1))g^{-1}(i/(L-1)). Both are now listed in Alg. 1. Rounding the folded values costs only 0.04 dB and 0.3° (now reported).

W2: "Retrain with matched schedules and rates…"

Tab. 3 now trains every row to 75 epochs, budget matched and only the quantizer differs. Its rows send all 1,024 seed slots, hence more bits (21–22 kbit). The last row is our main model, which sends only the active seeds.

QuantizerBits ↓PSNR ↑rFID ↓Per-grain° ↓Mean-orient.° ↓
FSQ (round + STE)21,95024.476.1326.362.55
FSQ (learned warp)21,45824.446.2026.242.66
STanH [1]22,51824.835.5623.292.95
FFQ, identity warp21,90324.605.6925.192.56
FFQ, learned warp + commit.21,40524.555.9025.122.46
FFQ + learned thresholds22,53724.845.4423.222.92
(only active seeds sent)
FFQ, learned warp + commit.11,90623.438.8724.322.18

At equal rate the fold beats FSQ: by 0.11 dB, 0.30 rFID and 1.1° per grain with the learned warp, and by 0.13 dB, 0.44 rFID and 1.2° with uniform levels. The revision credits the fold and removes the level-placement and blanket abstract claims. STanH (now in Sec. 2, App. F [1]) gains 0.23 dB over the fold for 2.8% more bits through learned thresholds; with learned thresholds, FFQ also matches it at similar bits.

W3: "Specify which output constitutes the solver input…"

Solver input (Sec. 4.4): the decoded image is upsampled 256²→300² and segmented by SAM where each grain gets the median orientation of its pixels, hence constant by construction. Sec. 3.4 now claims piecewise constancy only for the tessellation. SAM was checked against the synthetic reference labels, as you asked: its grain boundaries reach F1 = 0.953, adding only a small error. We ran the DAMASK [2] solver on 60 test fields in tension to 25% strain. Median error against the original with exact grains (full table in paper):

Solver inputYield stress(%) ↓Tensile strength(%) ↓Hardening exp(%) ↓Curve RMS (MPa) ↓
TessTok0.590.590.671.26
ViT-VQGAN (ft)0.740.740.831.76
WeTok (ft)0.920.970.932.14
MAGVIT-v2 (ft)1.221.241.152.80
Floor: original0.360.150.160.33

We also pair each reference grain with the reconstructed grain that overlaps it most (IoU ≥ 0.3) and compare orientations (Tab. 15). TessTok has the lowest error of the four tokenizers (27.6° vs. 30.0° for finetuned ViT-VQGAN) and matches more grain area (0.882 vs. 0.838), in all 15 alloy families and under six variants.

W4: "distinguish estimated code lengths from actual serialized round-trip sizes…"

Tab. 2 now measures every method the same way: every signal the decoder reads, entropy-coded, next to the actual serialized size. This led us to tighten the codec: TessTok's decoder now reads only the transmitted seeds. All TessTok numbers here outside Tab. 3 come from this model (75 epochs).

CodecIndex-counted bits ↓Serialized bits ↓rFID ↓
TessTok11,90611,9508.87
Label map + orientation table–31,778lossless
ViT-VQGAN (ft)12,39712,40313.56
SD-VAE, 4-bit latents9,9939,9999.85
SD-VAE, 6-bit latents18,33318,3405.98

The label map is the lossless codec you suggested; TessTok needs 62% fewer bits. Every row is entropy-coded with one estimator; TessTok then needs 4% fewer bits than ViT-VQGAN. The arithmetic-coded streams decode bit-exactly, within 0.13% (TessTok) and 0.04% (ViT-VQGAN) of the estimates; TessTok's reproduces its image exactly. At this rate TessTok is better than ViT-VQGAN on rFID and per-grain error (24.3° vs. 26.8°) and behind on PSNR (−0.17 dB).

Your specific suggestions improved the paper in four places with one additional page in the revised PDF. We hope the added evidence you asked for helps you reconsider your soundness score and recommend acceptance.

[1] Presta et al., IEEE TIP 2025. [2] Roters et al., Comput. Mater. Sci. 2019.

I appreciate the author’s point-by-point response. My primary remaining concern relates to the independence of the 60 test samples used for the solver experiments. Appendix C mentions that mirrored fields were added to the AZ31 and Mg-5Gd alloy families, while the scripts in the original supplementary material partitioned the training and test sets based on individual image files. Please confirm whether the original images or their mirrored counterparts for these 60 test samples appear in the training set. While the current documentation does not explicitly indicate such an overlap, if it is found to exist, I recommend regrouping the data—keeping original images and their mirrors together—and re-reporting the results in Table 5. Additionally, since the scalar quantizers in Table 3 are fine-tuned starting from the same checkpoint trained with uniform-level FFQ for 60 epochs, they are best interpreted as demonstrating the effect of replacing the quantizer at that specific checkpoint. Regarding Figure 3 and Table 19, it should also be clarified whether the decoder utilizes any seed information that has not been accounted for in the reported bitrates. Based on the evidence currently provided, I decide to maintain my score currently.

Thank you for the follow-up. All three of your current points are now checked directly:

1. Independence of the 60 solver fields (Tab. 5). We compared each of the 60 fields, and its horizontal mirror, pixel by pixel against all 19,965 training images of its three families (AZ31 extruded, AZ31 extruded and heat-treated, Mg-5Gd extruded). None of the 60 has an identical or mirrored copy in the training set, and none is itself a mirrored copy. Tab. 5 therefore needs no regrouping. Thanks for pointing that out and we will make it more clear in the paper text.

You are right that a per-file split can alloy such pairs, so we had already ran this same check before-hand on the full test set: 144 of the 10,100 test fields (1.4%) have a mirrored counterpart in training. Removing them changes no result. Every tokenizer is trained or fine-tuned on the same training set, and on the remaining 9,956 fields TessTok moves from 23.435 to 23.432 dB with its per-grain error unchanged at 24.32°, and ViT-VQGAN from 23.606 to 23.603 dB and from 26.79° to 26.80°. This pre-check we did’nt mention before will be stated in App. C.

2. Tab. 3. We agree that the scalar rows (now the first block of Tab. 13 in the updated PDF) show the effect of replacing the quantizer at one checkpoint for the last 15 of 75 epochs. To test whether the conclusions depend on that checkpoint, we had also repeated the swap at a another checkpoint with the main model's epoch-63 checkpoint, retrained for 12 epochs with each quantizer. Here and everywhere else now in Tesstok and new scalar row runs (apart from previous scalar rows) the decoder reads only transmitted active seeds and with matched bits under 0.1%.

QuantizerBitsPSNR ↑rFID ↓Per-grain° ↓Matched area ↑
FSQ (round + STE)11,91023.489.0324.270.948
FSQ (learned warp)11,91823.459.1924.300.948
STanH11,90623.429.1224.430.947
FFQ, identity warp + commit.11,91023.488.9824.260.948
FFQ, learned warp + commit. (repeat)11,91023.459.1124.300.948
Main model11,90623.438.8724.320.948

The conclusions of Sec. 4.2 hold at both checkpoints: (1) The fold is never behind FSQ because with the same warp, it is ahead of rounding with a straight-through estimator by 0.08–0.13 dB in Tab. 13 and level with it here (within 0.01 dB and 0.01° per grain). (2) The learned warp is neutral (at most 0.05 dB at either checkpoint), therefore we already dropped the level-placement claim. (3) At equal bits the fold matches or is ahead of STanH. In Tab. 13 it matches STanH once its thresholds are learned (24.84 vs. 24.83 dB); here it is ahead by 0.02–0.07 dB and 0.11–0.17° per grain.

The two swaps also differ in rate (all 1,024 slots at about 22 kbit vs. active seeds at 11.9 kbit), so we do not attribute this to the checkpoint alone. Rows are single runs and 5th row above, repeats the main model's last 12 epochs and differs from it by 0.02 dB and 0.24 rFID.

The case for FFQ therefore stands at both checkpoints, with quality at or above FSQ's, with exact gradients and no straight-through estimator. These rows are now in Tab. 3 of the updated PDF, and Sec. 4.2 words the FSQ comparison accordingly.

3. Fig. 3 and Tab. 19. We have done this already along with previous response and revision where the decoder of the main model only read the transmitted active seeds everywhere else (except the ~21k matched bits scalar rows we shared in previous response). The bits count everything the decoder reads, so no seed information is unaccounted for. We also varied the bitrate for the Tesstok+FFQ, one set of weights, no retraining, with the rate moved by the seed-existence threshold.

BitsPSNR ↑rFID ↓Per-grain° ↓Matched area ↑
8,00020.1331.0330.400.891
10,00122.0013.9926.660.933
11,91023.459.1124.300.948
14,00024.028.0623.410.951

Fig. 3b and Tab. 19 are already replaced by these points. Also, with the same accounting, the main model set to ViT-VQGAN's rate (12,397 bits, no retraining) reaches 23.65 dB against 23.61 dB, rFID 8.45 against 13.56.

We thank you for the careful reading of the paper and we hope these added evidences and checks resolve your remaining main concerns, and helps recommending acceptance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

New evidences during the rebuttal and what was changed in the revised PDF

We thank all three reviewers and the area chair. The reviewers' specific suggestions made the paper stronger and increased soundness, and we appreciate them. Below are the new evidences that support TessTok's object-based tokenization, then what changed in the revised PDF and where to find it.

New evidence supporting TessTok

  • Downstream solver (Sec. 4.4, Tab. 5). In a crystal-plasticity simulation of 60 test fields, TessTok's reconstructions give the lowest median error in every property estimation simulation over the reconstructions of the four tokenizers tested. The gains over fine-tuned ViT-VQGAN are significant for yield stress, tensile strength and the stress-strain curve estimation over decoded fields. We report medians with a paired Wilcoxon test because the per-field errors are skewed with a few fields have large errors, so a mean ± standard deviation describes them poorly. The means give the same ranking, and TessTok has both the lowest mean and the lowest standard deviation in every property (yield-stress error ↓ 0.79 ± 0.63% vs. 1.16 ± 1.19% for ViT-VQGAN). We can provide the full mean ± standard deviation table if useful.
  • Rate against ViT-VQGAN (Tab. 1, 2). With 4% fewer bits than fine-tuned ViT-VQGAN, TessTok is better on rFID ↓ (8.87 vs. 13.56) and per-grain orientation error ↓ (24.3° vs. 26.8°), and 0.17 dB behind on PSNR. In an additional experiment we set TessTok to ViT-VQGAN's rate in inference mode without retraining. It then leads on all three: PSNR ↑ 23.65 vs. 23.61 dB, rFID ↓ 8.45 vs. 13.56, per-grain error ↓ 24.0° vs. 26.8°.
  • Variable seed count. In a second additional experiment the same model, without retraining, sending a fixed 552 seeds per field at the same bits is compared against the variable seed count which is main model capability. Quality drops in the fixed seed test mode: PSNR ↑ 23.22 vs. 23.43 dB, rFID ↓ 11.76 vs. 8.87. TessTok sets the seed count per field, about 400 to 730 (5th to 95th percentile), giving more seeds to fields with more grains. A fixed count costs the third of fields with the most grains 1.4 dB and gains the third with the fewest only 0.7 dB. Spending the bits where the grains are is therefore a source of TessTok's gain.
  • Grain-level accuracy (App. H.1, Tab. 15). We compare each original microstructure field with its reconstruction, grain by grain: every exact grain of the original is paired with the reconstructed grain that overlaps it most using IoU, and their orientations are compared. TessTok's reconstructions have a lower orientation error ↓ than those of the strongest baseline, ViT-VQGAN (27.6° vs. 30.0°), and match more of the grain area ↑ (0.882 vs. 0.838).
  • Held-out alloy family (Tab. 9). We trained a model that never saw one alloy family (ME21, the finest-grained). It handles this unseen family as well as the families that it was trained on: leaving ME21 out costs 0.13 dB on ME21 and 0.16 dB on the other 14 families.

What changed in the revised PDF, and where

  • Abstract. The headline numbers now compare TessTok with fine-tuned ViT-VQGAN, with every rate measured as in Tab. 2 (11.9 vs. 12.4 kbit). The solver result is added. The quantizer claim now refers to consistent 75-epoch budget and to the quantizers we tested. Three claims are removed: that FFQ learns where to place its levels and TessTok leading on ImageNet.

  • Baselines and rate. Fine-tuned baselines are the main comparison now (Tab. 1; training budgets in Tab. 8). Every method's rate is measured the same way, entropy-coded next to the actual serialized bitstream (Tab. 2).

  • Quantizer. Tab. 3 trains every scalar quantizer consistently for 75 epochs with one recipe. In the setting where all 1024 seed slots are sent, the FFQ leads FSQ by 0.13 dB and 1.2° per grain and STanH is added (Sec. 2, App. F, Tab. 3). The level-placement claim and the blanket claim in the abstract are removed.

  • Proposition and algorithm. Prop. 1 and App. D are tightened and Alg. 1 lists the training and transmission computations. The statement changes do not affect the result.

  • Scope. Sec. 1 and 4.3 state the ImageNet experiment as a check of retained natural-image ability with no state of the art claim here. Reconstructing pure Measured (EBSD) maps and testing the solver in an inverse-design loop are mentioned as future scope in the limitations.

Looking forward to answer any further questions during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.