acceptodds
Under review as a conference paper at ICLR 2027

RiCoTA: Riemannian Companding Token Autoencoder

Abstract

Fast generation on Riemannian manifolds must preserve the intrinsic structure of data distributions with low sampling cost. We introduce the Riemannian Companding Token Autoencoder (RiCoTA), which learns a joint distribution over discrete latent codes for continuous manifold generation. Its design addresses a geometric mismatch: equal-width latent intervals can induce very different geodesic displacements after decoding. RiCoTA therefore organizes latent coordinates by average decoder sensitivity and allocates finer resolution where local changes have larger intrinsic effects, keeping the number of intervals per coordinate fixed. Training and generation share a uniform within-cell reconstruction rule. At inference, an autoregressive prior selects cells, continuous values are sampled within them, and one decoder evaluation produces manifold samples without trajectory integration. For a fixed representation, our asymptotic analysis derives a coordinatewise bound on intrinsic distortion and identifies the sensitivity-based allocation that minimizes its leading term. A complementary population bound controls generation error in terms of quantized reconstruction error and joint code-distribution mismatch. Held-out diagnostics show that estimated sensitivity predicts geometric risk and adaptive allocation reduces coordinatewise intrinsic distortion at a fixed budget. Experiments across 14 manifold benchmarks demonstrate strong support fidelity at low end-to-end sampling cost.

Primary area:
generative models
Keywords:
Riemannian manifolds, generative modeling, latent quantization, companding, efficient sampling

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_ZeZeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces RiCoTA as a framework for generative modelling of data living on Riemannian manifolds. The idea is that a uniform discretization of latent space may be very geometrically inefficient -- equal changes in latent coordinates may correspond to very different changes in the data space. The method therefore uses pullback geometry to define canonical coordinates and uses decoder sensitivity to allocate finer cells in sensitive regions. It autoregressively models the joint distribution over these discrete cells. During generation, RiCoTA samples a sequence of cell indices, samples values within selected cells, and makes a single decoder pass.

Critical strengths

Strength 1. It is a conceptually appealing and well-motivated idea. Since there could be a mismatch between a uniform discretization of latent space and the non-uniform geodesics induced by the decoder, allocating resolution according to the decoder's sensitivty makes sense.

Strength 2. The paper tests the proposed mechanism with a good effort. For example, the full versus identity comparisons are better than simply comparing against unrelated generative models (as they keep the architecture, prior, optimization, and code budget fixed while changing the cell allocation).

Critical weaknesses

Weakness 1. The practical utility and scalability of the design -- and hence the significance -- are not very convincing. The DNA study partly addresses this, but it has substantial modifications to tackle the higher dimensionality. I therefore find it unclear whether the basic formulation may scale to problems beyond the low-dimensional manifolds considered here. While some scaling aspects are discussed in the text, the paper would benefit from empirical results showing efficiency and quality change with latent dimensions.

Weakness 2. Some parts are overly dense and difficult to follow. Several sections appear fully AI-written; I recommend the authors to rewrite or remove overly dense parts and ensure everything is very clear. Generally, the paper needs a better, easier and more clean presentation.

Questions for authors

Question 1. The main experiments use a very small latent dimensionality. How does efficiency and quality compare with other generators when m0m_0 becomes significantly larger? When might autoregressive generation become the dominant cost?

Question 2. The full-vs-identity results support the authors' mechanism, but the improvements of Table 5 are very modest compared to the larger gains over baselines. How much of the overall improvement in the quality--speed tradeoff comes from geometry-aware companding rather than the broader encoder–decoder architecture and AR prior? It would strengthen the paper to more clearly demonstrate whether the benefits of their main novelty justify its additional complexity.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ZeZe (1/2)

W1 · Q1 — Quality and speed versus other generators at larger latent dimensions, and the AR bottleneck

Thank you for taking the time to review our paper.

Increasing RiCoTA’s latent dimension m0m_0 slowed generation without a consistent gain in quality (Table A). On each dataset, we evaluated RiCoTA at the latent dimension used in the main experiments (1×) and at 2×, 4× and 8× that dimension, keeping the data manifold and the number of cells per coordinate fixed. The nominal bit budget therefore grew with m0m_0, and each additional coordinate required one more sequential token draw before decoding. Some quality metrics improved at 2× or 4×. At 8×, however, generating 100k samples took 3.19–6.11 times as long as at 1×, and all five datasets showed lower mean local F1(0.5)F_1(0.5) and higher mean HD95 and MMD.

RiCoTA first took longer to generate 100k samples than M-FFF at 4× the base dimension on Fire and Protein Glycine and at 8× on RNA and SO(3)-Line. At 8×, Fire and RNA were the only two of these four datasets on which RiCoTA still led M-FFF on all three quality metrics. On the hyperbolic dataset, 8× RiCoTA remained faster than G-LSD, with similar local F1(0.5)F_1(0.5) and lower HD95 and MMD.

Table A. Generation quality and time per 100k samples as RiCoTA’s latent dimension increases. For each dataset, we tested RiCoTA at 1×, 2×, 4× and 8× the latent dimension used in the main experiments. Quality values are five-seed means and times are medians.

DatasetGeneratorLocal F₁(0.5) ↑HD95 ↓MMD ↓Time per 100k (ms) ↓
FireRiCoTA (1×)0.61422.2100.023307.14
FireRiCoTA (2×)0.59622.3200.0207712.06
FireRiCoTA (4×)0.60752.5080.0250022.30
FireRiCoTA (8×)0.58002.6700.0250642.54
FireM-FFF0.54368.0030.0299818.98
Protein GlycineRiCoTA (1×)0.65958.7110.022736.95
Protein GlycineRiCoTA (2×)0.64759.1520.0257512.00
Protein GlycineRiCoTA (4×)0.63329.7390.0269721.90
Protein GlycineRiCoTA (8×)0.64809.4030.0257142.46
Protein GlycineM-FFF0.65328.7280.0377221.59
RNARiCoTA (1×)0.131364.5720.039506.75
RNARiCoTA (2×)0.112072.1400.0369410.16
RNARiCoTA (4×)0.079978.6500.0413917.25
RNARiCoTA (8×)0.061578.7960.0457731.72
RNAM-FFF0.0459182.8550.4248523.75
HyperbolicRiCoTA (1×)0.65930.1320.022837.31
HyperbolicRiCoTA (2×)0.65900.1270.024879.64
HyperbolicRiCoTA (4×)0.66570.1250.0252514.01
HyperbolicRiCoTA (8×)0.64950.1400.0252123.30
HyperbolicG-LSD0.64800.1460.0989331.21
SO(3)-LineRiCoTA (1×)0.460613.0950.0239519.09
SO(3)-LineRiCoTA (2×)0.396115.9570.0290232.40
SO(3)-LineRiCoTA (4×)0.317818.4060.0350858.83
SO(3)-LineRiCoTA (8×)0.237220.2380.04613111.69
SO(3)-LineM-FFF0.190123.8930.0248885.08

When autoregressive sampling dominates. At the base latent dimension (1×), autoregressive (AR) token sampling took more than half the end-to-end generation time on Fire, Protein Glycine, RNA, and SO(3)-Line. On the hyperbolic dataset, this ratio first exceeded 50% at 4× the base dimension. At 8×, the ratio ranged from 78.5% to 94.4% across the five datasets (Table B). We added these results to Appendix H.6 of the revised paper (Table 32).

Table B. AR token sampling time per 100k samples. For each dataset, 1× denotes the latent dimension used in the main experiments. Entries give median AR time (ms) and, in parentheses, its percentage of the median end-to-end generation time. Bold marks the first tested dimension at which this ratio exceeded 50%.

Dataset1×2×4×8×
Fire5.25 (73.6%)10.27 (85.2%)20.38 (91.4%)40.18 (94.4%)
Protein Glycine5.24 (75.4%)10.20 (85.0%)19.94 (91.1%)39.62 (93.3%)
RNA3.81 (56.5%)7.21 (71.0%)14.15 (82.0%)28.24 (89.0%)
Hyperbolic2.60 (35.6%)4.74 (49.2%)9.15 (65.3%)18.28 (78.5%)
SO(3)-Line13.42 (70.3%)26.63 (82.2%)52.56 (89.3%)104.90 (93.9%)
AuthorsOpenReview ↗

Response to Reviewer ZeZe (2/2)

Q2 — How much does geometry-aware companding contribute to the quality–speed tradeoff?

Replacing Identity’s uniform probability cells with RiCoTA’s geometry-aware cells improved all three mean quality metrics in every manifold family. Across families, the largest absolute difference in mean generation time between Identity and RiCoTA was 0.06 ms per 100k samples (Table C).

Manifold Free-Form Flows (M-FFF) was the fastest competing baseline on average across the 13 shared benchmarks in our main experiments. It uses an autoencoder (AE) and decodes a continuous prior sample in one pass. To test a conventional quantizer, we trained a model with finite scalar quantization (FSQ) [1] and matched its decoder backbone and autoregressive (AR) prior architecture to RiCoTA’s.

Table C. Generation quality and time per 100k samples. Entries give local F1(0.5)F_1(0.5) ↑ / HD95 ↓ / MMD ↓ · time (ms). For each dataset, we used mean quality scores across five training seeds and median generation time. We then weighted datasets equally within each family. The last row gives the relative F1F_1 increase and HD95 and MMD reductions from Identity to RiCoTA.

Model or comparisonSphereSO(3)Product toriHyperbolic
M-FFF0.6277 / 11.939 / 0.0719 · 18.98 ms0.1941 / 59.302 / 0.2171 · 84.78 ms0.5263 / 43.290 / 0.1067 · 22.65 ms—
FSQ0.5940 / 9.020 / 0.0628 · 8.84 ms0.4132 / 4.976 / 0.0216 · 30.05 ms0.4390 / 18.402 / 0.0299 · 5.85 ms0.5033 / 0.135 / 0.0262 · 7.25 ms
Identity (RiCoTA with uniform cells)0.6747 / 9.495 / 0.0516 · 9.05 ms0.4654 / 5.180 / 0.0198 · 30.22 ms0.5399 / 19.379 / 0.0313 · 6.01 ms0.6503 / 0.140 / 0.0294 · 7.27 ms
RiCoTA0.6999 / 7.678 / 0.0445 · 9.05 ms0.4780 / 4.893 / 0.0180 · 30.28 ms0.5501 / 17.236 / 0.0238 · 5.99 ms0.6593 / 0.132 / 0.0228 · 7.31 ms
RiCoTA vs. Identity (relative improvement)3.7% / 19.1% / 13.7%2.7% / 5.5% / 9.2%1.9% / 11.1% / 24.0%1.4% / 5.9% / 22.2%

M-FFF vs. FSQ. Even with conventional quantization, FSQ generated samples faster than M-FFF on the sphere, SO(3) and product tori. Its mean local F1(0.5)F_1(0.5) exceeded M-FFF’s only on SO(3).

FSQ vs. Identity. The two models use different quantization schemes but share the same AE–AR design and decode once per sample after drawing the tokens autoregressively. At nearly identical generation times, Identity had higher mean local F1(0.5)F_1(0.5) in all four manifold families, while FSQ had lower mean HD95 in all four.

Identity vs. RiCoTA. Across the four manifold families, replacing Identity’s uniform cells with RiCoTA’s geometry-aware cells increased mean local F1(0.5)F_1(0.5) by 1.4–3.7% and reduced mean HD95 by 5.5–19.1% and mean MMD by 9.2–24.0%. RiCoTA estimates decoder sensitivity during Phase 1 to set the boundaries. At generation time, both models use their stored boundaries, take the same number of autoregressive steps, and decode once. Their measured generation times were nearly identical (Table C).

References

[1] Mentzer, F., et al. Finite Scalar Quantization: VQ-VAE Made Simple. ICLR, 2024.

W2 — Clarifying the presentation of the methods and theoretical analysis

We revised the presentation of our method and theoretical analysis. In Section 2, we expanded the overview of the two training phases. The revised Figure 1 caption describes how output geometry guides the choice of latent axes and cell boundaries and how RiCoTA uses the learned prior to generate samples. Section 3.2 now opens by explaining how decoder sensitivity is measured and how it guides cell placement. Section 3.4 explains what the autoencoder and the prior each learn before presenting their training objectives.

Thanks for your effort. Looking at the pdf, I believe the manuscript is going toward a better state now, but I still believe it falls short of typical ICLR-quality, so I still lean weak reject.

I do think the overview figure could be substantially improved, I see this was also mentioned by another reviewer, and I agree. I also think the appendices have become quite large, and the readability of them could be substantially improved. Making a much more focused new version may be in your best interest.

Best of luck with the paper, I highlight that it's an interesting idea.

AuthorsOpenReview ↗

Thank you for your feedback

Thank you for taking the time to read the revised manuscript and share your feedback. In the final version, we will improve the overview figure and shorten the appendices to make them easier to read.

Reviewer_m2RmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces RiCoTA, a two-stage generative framework for manifold-valued data that adapts latent quantization to geometric reconstruction sensitivity. It constructs canonical coordinates from latent covariance and decoder sensitivity, followed by marginal probability normalization and sensitivity-aware scalar companding. The encoder/decoder is trained using clean and quantized reconstructions with uniform within cell sampling, after which an autoregressive prior learns the joint distribution of discrete codes. Generation combines code sampling, continuous within-cell sampling, and a single manifold valued decoder evaluation. This helps in avoiding trajectory integration.

The theoretical contributions include affine equivalence of the unregularized population coordinate construction, a fixed representation high rate distortion bound with a cube-root optimal scalar allocation for its leading coordinatewise term, and a Wasserstein-1 generation error bound involving reconstruction error and code-distribution mismatch. Experiments cover 14 geometric benchmarks, matched code budget ablations, sensitivity diagnostics, alphabet size sweeps, and conditional DNA promoter generation. Results indicate improved geometric and support fidelity over fast baselines at low measured sampling cost in the evaluated settings.

Critical strengths

The paper provides a clear explanation of the geometric motivation and the overall two-stage framework. It carefully distinguishes between adapting quantization resolution within each coordinate and increasing the number of quantization levels, while also clarifying the scope of the theoretical results relative to the learned model. Although some technical details require careful reading, the main text and appendices offer a well-structured explanation of the method and its underlying assumptions.

The paper offers an interesting perspective on latent quantization by showing that regions with equal probability mass may have different reconstruction errors, and that decoder sensitivity can be used to allocate quantization resolution more effectively. Its main contribution lies in formulating this idea in a geometric setting and incorporating it into a generative framework. The proposed cube-root allocation rule is also well connected to existing results in scalar and functional quantization.

The theoretical results appear sound, and I have not identified any specific mathematical errors. However, I have some concerns regarding the empirical validation of the proposed framework.

In particular, the theoretical analysis and the main quantization diagnostics focus on coordinatewise distortion, while actual reconstruction involves simultaneously resampling all active latent coordinates. Although the paper provides a theoretical extension to joint reconstruction under additional assumptions and reports end-to-end ablations, it remains unclear how well the coordinatewise diagnostics reflect the behavior of the full reconstruction process.

Furthermore, the main comparisons with baselines do not include uncertainty estimates, making it difficult to assess the consistency of some reported improvements. The relatively small latent dimensions used in the geometric benchmarks also leave questions about scalability. Overall, these concerns primarily affect the empirical support and generality of the results rather than their mathematical correctness.

  1. Geometry-aware quantization with clear theoretical justification

The paper addresses the mismatch between latent-space quantization resolution and the resulting geodesic reconstruction error. It combines canonical coordinates, probability normalization, and sensitivity-aware companding to allocate quantization resolution according to the geometric impact of latent perturbations. Moreover, using the same within-cell sampling procedure during training and generation ensures consistency between the reconstruction and generative processes. The high-rate analysis provides a principled justification for the proposed allocation strategy, while clearly specifying the scope of its optimality guarantees.

This is an important strength because it offers a theoretically motivated and practical way to allocate a fixed quantization budget more effectively. In particular, the method reduces measured coordinatewise distortion through adaptive cell placement without introducing additional sensitivity computations or decoder evaluations during generation.

  1. Promising empirical results and meaningful ablation studies

The paper provides an extensive empirical evaluation across different manifold benchmarks, considering distributional quality, support fidelity, and geodesic-tail metrics. The reported results are promising, with RiCoTA achieving a macro-averaged support F1 of 0.872 compared to 0.743 for M-FFF on the 13 shared benchmarks, while requiring an average of 14.67 ms to generate 100,000 samples. Another positive aspect is that the sampling-time measurements account for the complete generation pipeline, including output processing and data transfer.

The ablation studies further strengthen the evaluation. In particular, the comparisons with the Identity Compander and Clean-Only variants help isolate the contributions of sensitivity-aware quantization and training through the finite reconstruction channel. Together with the frozen-representation experiments, these results provide evidence that the proposed quantization strategy contributes to the observed improvements.

Critical weaknesses
  1. Insufficient statistical uncertainty reporting in baseline comparisons

While the empirical results are promising, I find the absence of uncertainty estimates in the main comparison tables concerning. Although the authors report evaluating methods over five independent training seeds, Tables 1–4 present only point estimates without standard deviations or confidence intervals. This makes it difficult to assess the robustness of the reported improvements over competing methods.

This is particularly relevant for benchmarks where the differences are relatively small. For instance, Table 3 reports identical rounded support F1 scores of 0.906 for RiCoTA and M-FFF on Protein General, while the difference on Glycine is only 0.002. Without uncertainty estimates, it is unclear whether such differences are consistent across independent training runs or whether the performance distributions overlap substantially.

Although the paper provides uncertainty estimates for its ablation studies, similar reporting is missing from the main baseline comparisons. Given that improved generation quality is a central empirical claim, I believe reporting variability across training seeds, as well as confidence intervals for the main performance differences and macro-averaged results, is important for establishing the reliability of the conclusions.

  1. Limited empirical validation of the joint reconstruction behavior

The proposed companding rule is derived from a coordinatewise sensitivity analysis, where one latent coordinate is perturbed while the remaining coordinates are kept in data-supported contexts. However, actual reconstruction involves simultaneously sampling all active latent coordinates within their quantization cells.

The theoretical extension from coordinatewise to joint reconstruction relies on the hybrid-context stability assumption in Appendix C.4 (Eq. 70). This distinction is important because independent within-cell sampling does not necessarily preserve the conditional dependence structure of the original latent distribution. When latent coordinates are strongly dependent or interact nonlinearly through the decoder, simultaneous perturbations could produce reconstruction errors that are not adequately captured by the individual coordinatewise measurements.

Although the paper provides end-to-end ablations demonstrating the benefits of the proposed method, the frozen-representation diagnostics in Appendices H.3–H.4 explicitly measure sums of one-coordinate-at-a-time distortions rather than the actual joint reconstruction error. Consequently, the extent to which the proposed scalar allocation criterion improves the deployed joint reconstruction channel remains insufficiently characterized.

A direct comparison between coordinatewise and simultaneous reconstruction errors would help establish whether the theoretical allocation principle translates into improvements under the actual sampling procedure. It would also be useful to distinguish errors introduced by within-cell reconstruction from those arising from learning the joint code distribution.

Questions for authors

Questions for Authors

1. Statistical reliability of the baseline comparisons

Could the authors provide standard deviations or confidence intervals across training seeds for the results in Tables 1–4, including the competing methods? In particular, are the improvements consistent across independent runs for benchmarks where the reported performance differences are small? It would also be helpful to quantify uncertainty in the macro-averaged improvements, as this could materially affect the interpretation of the claimed performance gains.

2. Joint reconstruction error and the role of the learned prior

Could the authors clarify how the coordinatewise distortion improvements relate to the actual reconstruction error when all latent coordinates are sampled simultaneously? In particular, how does the joint reconstruction distortion compare with the sum of individual coordinatewise distortions reported in Appendices H.3–H.4? Is the hybrid-context stability discrepancy in Eq. (70) empirically small at the evaluated quantization resolutions?

3. Dimensional scalability of the proposed quantization framework

Theorem 1 establishes a high-rate distortion bound with K0−2K_0^{-2} scaling for a fixed latent dimension m0m_0. However, expressing the nominal code length as B=m0log⁡2K0B=m_0\log_2 K_0 gives K0−2=2−2B/m0K_0^{-2}=2^{-2B/m_0}, raising questions about the quality–resolution trade-off as the latent dimensionality increases under a fixed total bit budget. While this observation does not establish a curse-of-dimensionality lower bound for the proposed generative model, it suggests that dimensional scaling deserves further investigation.

The geometric benchmarks use relatively small latent dimensions (m0=2m_0=2 to 66), while the DNA experiment employs a specialized conditional architecture.

Could the authors clarify how the method scales as m0m_0 increases, particularly in terms of reconstruction quality, quantization resolution, the complexity of learning the joint code distribution, and computational cost? Is there theoretical or empirical evidence that the favorable quality–cost trade-off persists in higher-dimensional latent spaces? Even a limited comparison across latent dimensions, ideally distinguishing fixed-total-bit from fixed-bits-per-coordinate settings, would help clarify this issue.

5 comments
AuthorsOpenReview ↗

Response to Reviewer m2Rm (1/3)

W1 · Q1 — Uncertainty across seeds in per-dataset and macro-averaged baseline comparisons

Thank you for taking the time to review our paper.

We added sample standard deviations across five training runs for RiCoTA and the competing methods to the per-dataset quality tables in Appendix G (Tables 12–17 and 19 of the revised manuscript). We estimated the approximate 95% confidence intervals in Tables A–B using a percentile bootstrap, resampling each method’s five runs independently within each dataset.

On Protein General and Protein Glycine, the approximate 95% confidence intervals for the sF1 difference between RiCoTA and M-FFF both included zero, so five training runs per method leave the ranking uncertain on these two datasets (Table A).

Table A. Differences in sF1 between RiCoTA and M-FFF on Protein General and Protein Glycine. Method columns report mean ± sample SD across five training runs.

DatasetRiCoTA sF1M-FFF sF1RiCoTA − M-FFFApprox. 95% interval
Protein General0.90649 ± 0.000920.90603 ± 0.00132+0.00046[−0.00083, 0.00167]
Protein Glycine0.90359 ± 0.003920.90204 ± 0.00122+0.00155[−0.00156, 0.00473]

With equal weight given to each of the 13 shared benchmarks, RiCoTA had higher mean sF1 and lower mean HD95 and MMD than each of the four baselines. For all three metrics, the approximate 95% confidence interval for each difference excluded zero (Table B).

Table B. Mean differences across the 13 shared datasets, RiCoTA minus each baseline (approximate 95% bootstrap intervals). Positive sF1 differences and negative HD95 and MMD differences favor RiCoTA.

BaselineΔ sF1 ↑Δ HD95 (°) ↓Δ MMD ↓
M-FFF+0.129 [0.109, 0.149]−28.07 [−29.51, −26.81]−0.1016 [−0.1055, −0.0979]
G-LSD+0.218 [0.212, 0.225]−23.83 [−24.65, −23.05]−0.0835 [−0.0942, −0.0757]
RCM/RCT+0.134 [0.130, 0.139]−26.15 [−27.45, −24.94]−0.1236 [−0.1278, −0.1195]
RMF-MT+0.260 [0.254, 0.266]−26.90 [−28.35, −25.53]−0.0634 [−0.0739, −0.0537]
AuthorsOpenReview ↗

Response to Reviewer m2Rm (2/3)

W2 · Q2 — Coordinatewise versus joint distortion and sensitivity after resampling preceding coordinates

We compared uniform and RiCoTA cells on the same frozen Phase 1 autoencoder for each training run. For each test observation, we decoded its continuous latent vector to obtain the reference output. To measure TT, we resampled one active coordinate at a time within its cell and summed the squared geodesic displacements from the reference output. For JJ, we applied the same draws to all active coordinates together and measured the squared geodesic displacement from that reference. RR measures the normalized squared geodesic distance from this joint reconstruction to the original observation. Each observation was reconstructed from its own encoded cell indices, so none of these measurements involved sampling from the learned prior.

With RiCoTA cells, the five-seed mean of JJ was within 3% of the mean of TT on all 14 datasets (Table C). Switching from uniform to RiCoTA cells yielded comparable improvements in coordinatewise and joint reconstruction, with the percentage reductions in TT and JJ differing by at most 1.1 percentage points on any dataset. The five-seed mean reconstruction error RR against the original observations also fell on all 14 datasets.

Table C. Coordinatewise distortion, joint distortion and reconstruction error. For RiCoTA cells, J/TJ/T is the five-seed mean of JJ divided by the five-seed mean of TT. The last three columns give the percentage decrease in each five-seed mean when switching from uniform to RiCoTA cells.

DatasetJ/T (RiCoTA cells)T reductionJ reductionR reduction
Earthquake1.008068.17%68.83%67.12%
Flood1.011551.21%51.23%50.85%
Fire0.996183.20%83.44%82.89%
Volcano1.023493.85%93.99%92.91%
SO(3)-Cone1.010975.91%76.24%40.33%
SO(3)-Fisher241.002149.19%49.24%45.55%
SO(3)-Line0.999254.66%53.75%5.45%
SO(3)-Swiss1.000932.56%32.67%32.58%
Protein General0.980696.60%96.57%86.30%
Protein Glycine0.997955.19%54.14%53.75%
Protein Proline1.005685.79%85.56%46.20%
Protein Pre-Pro1.000484.16%84.28%65.27%
RNA1.030088.65%88.58%46.41%
Hyperbolic0.999957.58%57.32%50.17%

When the preceding coordinates were resampled within their RiCoTA cells, the mean relative change in squared decoder sensitivity along the current axis was below 5% on 13 of 14 datasets (Table D). On Volcano, it was 6.06%, based on 21 eligible bins out of 360. Table D reports the observed discrepancies in eligible bins at one resolution per dataset. Eq. (70) assumes that the absolute difference between the two sensitivity profiles converges uniformly to zero as resolution increases. In the revised paper, we added the joint reconstruction results (Table 25) and the sensitivity measurements (Table 26) to Appendix H.3.

Table D. Change in finite-difference estimates of normalized squared decoder sensitivity after resampling preceding coordinates. In each bin with at least five distinct test observations, we compared mean sensitivity along the current axis with the preceding coordinates at their observed values and after resampling them within their respective RiCoTA cells. For each seed and active axis after the first, we divided the mean absolute bin change by the mean original sensitivity, weighting both by the number of observations per bin. Mean relative discrepancy averages these percentages equally across eligible seed–axis pairs. For the bin with the largest absolute change in mean sensitivity, the last column reports that change as a percentage of the bin’s mean sensitivity before resampling.

DatasetCells per axis (K₀)Mean relative discrepancyEligible / total binsRelative change at the largest absolute discrepancy (%)
Earthquake640.51%610/6400.38%
Flood960.66%558/9602.55%
Fire640.34%640/6400.57%
Volcano366.06%21/3605.93%
SO(3)-Cone961.47%1440/14403.42%
SO(3)-Fisher24640.42%960/9600.60%
SO(3)-Line961.00%2400/24004.45%
SO(3)-Swiss720.37%1080/10800.03%
Protein General1280.31%640/6400.61%
Protein Glycine600.53%900/9001.71%
Protein Proline641.37%318/3208.57%
Protein Pre-Pro641.17%627/6404.23%
RNA243.80%360/3605.65%
Hyperbolic480.57%240/2401.62%
AuthorsOpenReview ↗

Response to Reviewer m2Rm (3/3)

Q3 — Increasing latent dimension at fixed total bits or fixed bits per coordinate

Theorem 1 fixes the representation and latent dimension as the number of cells per coordinate increases. To examine the effect of increasing the dimension itself, we trained RiCoTA at latent dimensions of 4, 8, 12 and 24 on five datasets under two bit budgets. Both experiments used the four-dimensional model with 64 cells per coordinate as their reference. As we increased the dimension, we held the total budget at 24 bits per sample in one experiment and the resolution at 6 bits per coordinate in the other.

Table E. Generation sF1 ↑ across latent dimensions at 24 total bits or six bits per coordinate. Entries are means ± sample SD over five training seeds. The nominal budget is B=m0log⁡2K0B=m_0\log_2 K_0 bits per sample.

BudgetLatent dim.Cells/axisTotal bitsFireSO(3)-SwissProtein GlycineRNAHyperbolic
24 bits total, 6 bits/axis464240.8635 ± 0.00540.9042 ± 0.00070.9014 ± 0.00230.7250 ± 0.00940.9038 ± 0.0007
24 bits total88240.7126 ± 0.01720.8490 ± 0.01210.8929 ± 0.00400.7420 ± 0.00530.7253 ± 0.1101
24 bits total124240.6298 ± 0.00730.7403 ± 0.01920.8562 ± 0.01210.7156 ± 0.00740.8193 ± 0.0368
24 bits total242240.5350 ± 0.01370.6226 ± 0.03950.7552 ± 0.02240.6394 ± 0.00980.8016 ± 0.0201
6 bits/axis864480.8604 ± 0.00470.9051 ± 0.00270.8993 ± 0.00260.7209 ± 0.00440.9023 ± 0.0031
6 bits/axis1264720.8631 ± 0.00310.9048 ± 0.00240.9005 ± 0.00180.7012 ± 0.00420.9011 ± 0.0034
6 bits/axis24641440.8550 ± 0.00390.9041 ± 0.00300.9022 ± 0.00260.6895 ± 0.00370.9015 ± 0.0022

Table F. Reconstruction error, test Phase 2 loss and time to generate 100k samples at latent dimensions 4 and 24. Reconstruction error is the mean squared geodesic distance from a test sample to its joint within-cell reconstruction, divided by the squared training-data distance scale aM2a_{\mathcal M}^2. Test Phase 2 loss is the prior’s mean negative log-likelihood per test code divided by m0ln⁡K0m_0\ln K_0. Errors and losses are five-seed means. Times are medians.

MeasureBudget (latent dim., cells/axis)FireSO(3)-SwissProtein GlycineRNAHyperbolic
Reconstruction error ↓24 bits total, 6 bits/axis (4, 64)14.8413.1700.54316.1515.913
Reconstruction error ↓24 bits total (24, 2)25,441.4491,933.6831,418.187135.2191,114.691
Reconstruction error ↓144 bits total, 6 bits/axis (24, 64)13.4492.9180.5490.13612.652
Test Phase 2 loss24 bits total, 6 bits/axis (4, 64)0.4570.5800.6550.7590.736
Test Phase 2 loss24 bits total (24, 2)0.3260.4150.3940.6090.413
Test Phase 2 loss144 bits total, 6 bits/axis (24, 64)0.2970.2060.4930.6190.702
Time (ms/100k) ↓24 bits total, 6 bits/axis (4, 64)8.8160.767.028.3310.05
Time (ms/100k) ↓24 bits total (24, 2)30.24244.8919.9120.8523.02
Time (ms/100k) ↓144 bits total, 6 bits/axis (24, 64)42.54257.5832.3033.4635.83

1. Fixed total budget (24 bits per sample). Increasing the latent dimension from 4 to 24 reduced the number of cells per coordinate from 64 to 2 (Table E). Reconstruction error rose on all five datasets, and mean generation sF1 fell by 8.56–32.85 percentage points. Generating 100k samples took 2.29–4.03 times as long (Table F).

2. Fixed budget per coordinate (six bits). Keeping 64 cells per coordinate raised the total budget from 24 to 144 bits. At 24 dimensions, mean sF1 stayed within 0.85 percentage points of the four-dimensional result on four datasets and fell by 3.55 points on RNA (Table E). Reconstruction error fell on three datasets. As in the fixed-total-budget experiment, the number of sequential token draws per sample rose from 4 to 24. With 64 cells per coordinate, generating 100k samples took 3.56–4.83 times as long as with the four-dimensional model (Table F). In the revised paper, we added Section 6.5 and Table 6 to show how generation sF1 and relative sampling time change with latent dimension under both bit budgets.

To assess how well the prior models the joint code distribution as latent dimension increases, we compared its normalized Phase 2 loss on held-out token sequences (Table F). Under both bit budgets, this loss was lower at 24 dimensions than at four on all five datasets. Because the encoder is retrained at each dimension and produces different token sequences, the lower loss alone does not establish that the higher-dimensional joint token distribution is easier to learn.

Reviewer_2Wc1OpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes RiCoTA, a framework for efficient data generation on Riemannian manifolds. Instead of using diffusion or flow matching, the method uses an autoencoder to encode continuous data into discrete tokens and trains an autoregressive model to generate these tokens. The main idea is to adjust the size of each token's interval based on the decoder's sensitivity, using smaller intervals in more sensitive regions to reduce reconstruction errors. The paper also provides theoretical analysis to support this design. Experiments on multiple manifold benchmarks show promising generation quality and sampling efficiency compared with existing methods.

Critical strengths

The proposed method is reasonable and supported by theoretical analysis. The experiments are comprehensive, and the results show good generation quality and efficiency across different manifold benchmarks.

Critical weaknesses

The method is somewhat difficult to follow and could benefit from clearer explanations.

Questions for authors
  1. Have the authors considered comparing RiCoTA with pupular discrete latent generative models, such as VQ-VAE or FSQ combined with an autoregressive prior and the same manifold-valued decoder?

  2. Could the authors provide an ablation comparing the global geometric transformation in Section 3.1 with alternatives, such as whitening?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 2Wc1

W1 — Making the method easier to follow

Thank you for taking the time to review our paper.

We revised the overview in Section 2 and several passages in Section 3 to make the method easier to follow. For example, Section 3.2 now explains how decoder sensitivity guides cell placement before introducing Eq. (7). Section 3.4 opens with what the autoencoder learns in Phase 1 and what the autoregressive prior learns in Phase 2, before presenting their training objectives.

Q1 — Comparing RiCoTA with FSQ and VQ-VAE

For this comparison, we trained an autoencoder using finite scalar quantization (FSQ) [1] and a VQ-VAE [2]. For both models, we matched the decoder backbone and autoregressive prior architecture to those used in RiCoTA (Table A). Each model uses m0m_0 tokens with K0K_0 choices per token, giving the same nominal budget of m0log⁡2K0m_0\log_2 K_0 bits per sample. Each FSQ token selects one of K0K_0 fixed scalar levels, while each VQ-VAE token selects one of K0K_0 learned 16-dimensional codebook vectors. The VQ-VAE decoder therefore receives 16m016m_0 scalars, compared with m0m_0 for FSQ and RiCoTA. RiCoTA trains and generates with a continuous draw within each selected cell.

Across all four manifold families, RiCoTA had higher mean sF1 and lower mean HD95 and MMD than both baselines. Its sF1 advantage ranged from 2.76 to 5.77 percentage points over FSQ and from 17.08 to 26.08 points over VQ-VAE.

Table A. Generation quality of FSQ, VQ-VAE and RiCoTA at matched nominal bit budgets. Entries give sF1 ↑ / HD95 ↓ / MMD ↓. We averaged over five training seeds per dataset and weighted datasets equally within each family.

GeneratorSphereSO(3)Product toriHyperbolic
FSQ0.8476 / 9.020 / 0.06280.8214 / 4.976 / 0.02160.8122 / 18.402 / 0.02990.8565 / 0.135 / 0.0262
VQ-VAE0.7045 / 9.478 / 0.05800.6444 / 7.284 / 0.02120.6091 / 20.681 / 0.02760.7344 / 0.193 / 0.0268
Full RiCoTA0.8978 / 7.678 / 0.04450.8490 / 4.893 / 0.01800.8699 / 17.236 / 0.02380.9052 / 0.132 / 0.0228

References

[1] Mentzer, F., et al. Finite Scalar Quantization: VQ-VAE Made Simple. ICLR, 2024.

[2] van den Oord, A., et al. Neural Discrete Representation Learning. NeurIPS, 2017.

Q2 — Does the global geometric transformation improve on whitening?

We compared RiCoTA with two whitening controls at the same token budget (Table B). The whitening-only control uses Cholesky whitening of the regularized covariance of the training latents. After the same whitening step, RiCoTA uses the decoder’s mean pullback of the output metric to determine the orientation and order of the axes. PCA whitening orders the axes by the eigenvalues of the regularized training covariance before whitening, from largest to smallest. During training, all three models fit marginal CDFs in their own coordinate systems and then use decoder sensitivity to place cell boundaries.

Table B. Generation quality with canonical, Cholesky-whitened and PCA-whitened axes. Entries give local F1(0.5)F_1(0.5) ↑ / HD95 ↓ / MMD ↓. We averaged over five training seeds per dataset and weighted datasets equally within each family.

Global transformationSphereSO(3)Product toriHyperbolic
Whitening-only (Cholesky)0.6760 / 8.682 / 0.06100.4725 / 4.952 / 0.02230.5474 / 17.887 / 0.02940.6617 / 0.1413 / 0.0244
PCA whitening (variance order)0.6848 / 9.186 / 0.06260.4712 / 4.985 / 0.02240.5459 / 17.360 / 0.02910.6588 / 0.1320 / 0.0217
Full RiCoTA (mean decoder pullback)0.6999 / 7.678 / 0.04450.4780 / 4.893 / 0.01800.5501 / 17.236 / 0.02380.6593 / 0.1319 / 0.0228

On the sphere, SO(3) and product tori, RiCoTA’s geometry-based choice of axis directions and order gave higher mean local F1(0.5)F_1(0.5) and lower mean HD95 and MMD than either whitening control. Across these families, the gain in local F1(0.5)F_1(0.5) over the better whitening control ranged from 0.27 to 1.52 percentage points. On the hyperbolic dataset, the benefit of using output geometry to choose the axes depended on the metric. We added this ablation comparing RiCoTA’s geometric transformation with Cholesky and PCA whitening to Appendix H.1 (Table 21).

Dear Reviewer 2Wc1,

We appreciate your feedback on our paper. In our response, we compared RiCoTA with FSQ and VQ-VAE and evaluated Cholesky and PCA whitening as alternatives to RiCoTA’s method for choosing latent axes.

As the discussion draws to a close, we would be grateful if you could let us know whether our response has addressed your concerns.

Thank you again for your time and consideration.

Thanks. I keep my positive score.

Reviewer_7mbWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In the context of Riemannian manifolds, this paper proposes RiCoTA to explore another complementary direction for efficient manifold generation: how to model the latent distribution at different resolutions, and how this resolution should reflect the geometry after decoding. RiCoTA uses the decoder's geometric sensitivity to adaptively allocate resolution. It adopts a two-stage training process: the first stage trains a geometry-aware quantized autoencoder, while the second stage uses an autoregressive prior to learn the joint distribution of discrete tokens.

Critical strengths

1.The method is complete and supported by detailed mathematical analysis. The paper presents a relatively complete framework for generation on Riemannian manifolds. The authors also provide theoretical analysis of canonical coordinate transformations, quantization distortion, and the final generation error, which gives mathematical support for the proposed method.

2.The experiments are comprehensive. The authors evaluate the method on various Riemannian manifold datasets and provide a fairly thorough evaluation of generation quality, sampling efficiency, and component ablations. In particular, the fixed-representation experiments and companding exponent analysis provide good evidence for the effectiveness of geometry-adaptive binning.

Critical weaknesses
  1. The writing and organization could be clearer. It is sometimes difficult to understand which problem each component addresses and how it contributes to the overall method. In particular, the authors should better explain how the second-stage training procedure relates to the central research question of how latent-space resolution should reflect decoded geometry.

  2. The novelty of the method and the need for a Riemannian formulation remain unclear in comparison with existing non-uniform quantization approaches. The resolution-allocation problem is not specific to Riemannian manifolds. Although the paper cites classical quantization work and demonstrates improvements over uniform binning, these results do not establish an advantage over existing non-uniform methods. Direct comparisons would help clarify the added value of the proposed geometric design.

Questions for authors

Q1. Could the authors clarify how the second-stage training procedure addresses the paper’s central research question of how to choose the resolution for modeling a latent distribution and how that resolution should reflect decoded geometry?

Q2. Could the authors further clarify the main contributions and performance improvements of RiCoTA on Riemannian manifolds compared with existing non-uniform quantization methods?

Q3. The authors focus on Riemannian manifolds, but some of the proposed methods and theoretical results, such as CDF transformation, non-uniform companding, and the cube-root allocation rule, also have corresponding forms in Euclidean spaces. Which parts of the proposed method and theoretical results actually capture the unique properties of Riemannian geometry, rather than simply extending existing quantization and generative methods to a new setting? Could the authors make these points clearer in the paper?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 7mbW (1/2)

W1 · Q1 — How Phase 2 uses the latent resolution set by output geometry

Thank you for taking the time to review our paper.

Phase 2 models the latent distribution at the resolution chosen in Phase 1. Phase 1 uses the latent covariance and the decoder’s mean pullback metric to choose the axes. After mapping each axis to a probability scale with its marginal CDF, it allocates a fixed number of cells per axis, placing finer cells in regions of higher decoder sensitivity, measured using geodesic distance on the output manifold.

The partition determines which latent values share a code. Normalizing each axis separately can leave dependencies across axes, so knowing how often each cell occurs on its own does not tell us how often particular cells occur together. Phase 2 keeps the representation and partition fixed and trains the autoregressive prior to learn those joint probabilities.

Theorem 2 makes the role of the prior explicit. The generation-error bound combines quantized reconstruction error with a term for the mismatch between the data and prior code distributions (Eq. 17). For the fixed partition and representation, minimizing the Phase 2 population cross-entropy also minimizes the KL divergence in that term. We revised Section 2, the caption of Figure 1, and the opening of Section 3.4 to clarify this division of roles.

Q3 — Which parts of the method and theorems depend on Riemannian geometry?

The marginal CDF transform, non-uniform companding, and cube-root allocation rule all have Euclidean counterparts. RiCoTA builds on these tools. Its geometric design uses the Riemannian metric gg on the output manifold to determine the finite latent interface: both the choice of latent axes and the placement of cells along each axis.

Globally, the decoder pullback measures the squared Riemannian speed of the decoded output along each latent direction. RiCoTA combines its population average with the latent covariance to orient and order the whitened axes (Eqs. 3–4).

Locally, the squared geodesic displacement induced by a small change in probability coordinate jj has a leading quadratic coefficient given by the squared Riemannian speed along that coordinate (Lemma C.1). Normalizing this coefficient and taking its conditional expectation gives the distortion weight in Eq. (7):

γj(s)=E![aM−2,gD(U)!(∂jD(U),∂jD(U)),|,Uj=s].\gamma_j(s)=\mathbb E\\!\left[a_{\mathcal M}^{-2}\\,g_{\mathcal D(U)}\\!\left(\partial_j\mathcal D(U),\partial_j\mathcal D(U)\right)\\,\middle|\\,U_j=s\right].

These weights determine the cell boundaries, with finer cells in regions of greater intrinsic sensitivity. Thus, the output geometry enters the tokenization procedure through both the global frame and the local allocation.

Theorem 1. Sensitivity is measured one coordinate at a time, whereas reconstruction resamples all active coordinates jointly within their cells. In Theorem 1, we prove that, for a fixed representation and under the stated high-rate and hybrid-context stability assumptions, the single-coordinate sensitivity weights yield an asymptotic upper bound on the expected normalized squared geodesic displacement between decoder outputs before and after joint resampling (Eq. 16). Minimizing the leading term gives the classical cube-root rule λj(s)∝γj(s)1/3\lambda_j(s)\propto\gamma_j(s)^{1/3}, as in fixed-rate functional quantization [1]. The geometric content lies in the intrinsic distortion weights and their connection to joint decoder distortion; the allocation exponent remains classical.

Theorem 2. On a compact output manifold M\mathcal M, with the Phase 1 model and tokenization fixed, Eq. (17) bounds generation error in geodesic Wasserstein distance by quantized reconstruction error and joint code-distribution mismatch. This is a general metric-space coupling argument instantiated with dgd_g, and it also applies on a bounded Euclidean domain. Its role is to connect the two training stages to generation quality under the same intrinsic distance.

Our contribution is a decoder-aware finite-code interface in which the output manifold’s Riemannian metric guides latent-axis selection and cell placement. We do not claim a new curvature-dependent allocation exponent or that all theoretical results are exclusive to non-Euclidean manifolds. We revised Sections 3.1–3.3 to clarify how the output metric determines the axes and cell placement. The discussion after Eq. (16) now explicitly distinguishes the metric-dependent distortion weights from the classical cube-root allocation rule.

References

[1] Misra, V., et al. Distributed Scalar Quantization for Computing: High-Resolution Analysis and Extensions. IEEE Transactions on Information Theory, 2011.

AuthorsOpenReview ↗

Response to Reviewer 7mbW (2/2)

W2 · Q2 — How does geometry-aware cell placement compare with learned non-uniform quantization?

We adapted three learned non-uniform boundary rules [1–3] to RiCoTA’s generation pipeline and compared them with RiCoTA’s cell allocation at the same cell budget (Table A). The three alternatives learn their boundary parameters by minimizing reconstruction error. RiCoTA measures decoder sensitivity along each axis using geodesic distance on the output manifold and places narrower cells where sensitivity is higher.

We trained each variant through both stages using the same geometric coordinate construction, marginal CDF normalization, within-cell sampling rule, decoder and autoregressive prior architectures, and training budgets.

Table A. Generation quality under four boundary rules at the same cell budget. Entries give local F1(0.5)F_1(0.5) ↑ / HD95 ↓ / MMD ↓. We averaged over five training seeds per dataset and weighted datasets equally within each family.

MethodBoundary ruleSphereSO(3)Product toriHyperbolic
LCQ-style [1]Place narrower cells where the learned rescaling curve is steeper.0.6730 / 9.423 / 0.05700.4644 / 5.089 / 0.01950.5424 / 18.688 / 0.03190.6531 / 0.141 / 0.0269
Mentzer-style [2]Place each boundary halfway between two neighboring learned centers.0.6767 / 8.381 / 0.05400.4693 / 5.009 / 0.01970.5459 / 18.845 / 0.03240.6531 / 0.142 / 0.0275
NLIC-style [3]Learn where to split the next axis based on previously selected tokens.0.6770 / 8.188 / 0.05080.4641 / 5.110 / 0.02090.5446 / 18.891 / 0.03010.6644 / 0.136 / 0.0247
RiCoTAMeasure decoder sensitivity along each axis using geodesic distance. Place narrower cells where sensitivity is higher.0.6999 / 7.678 / 0.04450.4780 / 4.893 / 0.01800.5501 / 17.236 / 0.02380.6593 / 0.132 / 0.0228

On the sphere, SO(3) and product tori, RiCoTA’s mean local F1(0.5)F_1(0.5) was 0.42–2.30 percentage points higher than that of the best learned alternative in each family. NLIC-style scored highest on the hyperbolic dataset. The most consistent gains from sensitivity-based cell placement were in HD95 and MMD, where RiCoTA had the lowest means in all four families. We added this comparison to Appendix H.1 of the revised paper (Table 22).

References

[1] Yamamoto, K. Learnable Companding Quantization for Accurate Low-Bit Neural Networks. CVPR, 2021.

[2] Mentzer, F., et al. Conditional Probability Models for Deep Image Compression. CVPR, 2018.

[3] Ge, Z., et al. NLIC: Non-uniform Quantization-Based Learned Image Compression. IEEE Transactions on Circuits and Systems for Video Technology, 2024.

Dear Reviewer 7mbW,

We appreciate your feedback on our paper. In our response, we explained how Phase 2 learns the joint distribution of the codes defined in Phase 1 and where Riemannian geometry enters our analysis. We also compared RiCoTA’s cell placement with three learned boundary rules adapted from non-uniform quantization methods.

As the discussion draws to a close, we would be grateful if you could let us know whether our response has addressed your concerns.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their feedback. In response to their questions, we ran additional experiments during the rebuttal and incorporated the main findings into the revised manuscript.

Cell placement and latent-axis selection. Reviewer 7mbW asked how RiCoTA compares with existing non-uniform quantization methods. At the same cell budget, RiCoTA’s geometric rule for placing cell boundaries outperformed three learned boundary rules on mean HD95 and MMD in every manifold family. It also achieved the highest mean local F1 in three of the four families.

Reviewer 2Wc1 suggested whitening as an alternative for choosing latent axes. Keeping the sensitivity-based cell placement rule fixed, we compared RiCoTA’s axes with Cholesky- and PCA-whitened axes. On the sphere, SO(3), and product tori, RiCoTA outperformed both whitening controls on mean local F1, HD95, and MMD. We added the whitening comparison (Table 21) and the learned-boundary comparison (Table 22) to Appendix H.1.

Reviewer 2Wc1 also suggested comparing RiCoTA with FSQ and VQ-VAE. We matched both baselines’ decoder backbones and autoregressive prior architectures to RiCoTA’s. In each of the four manifold families, RiCoTA achieved better mean scores than both baselines on every quality metric used in this comparison.

From coordinatewise sensitivity to joint reconstruction. For a fixed representation, we derive a high-rate upper bound on expected joint decoder distortion in terms of coordinatewise sensitivities (Theorem 1). To address Reviewer m2Rm’s question, we extended our coordinatewise analysis by measuring how far the decoder output moved when all active coordinates were resampled together. On all 14 datasets, switching from uniform to RiCoTA cells reduced both joint distortion and summed coordinatewise distortion. The two reductions differed by at most 1.1 percentage points. Reconstruction error relative to the original observations also fell on every dataset. We added these results to Appendix H.3 (Table 25). They support using axiswise sensitivity to place cells for joint reconstruction.

Generation quality and speed across latent dimensions. Reviewer m2Rm suggested varying the latent dimension while holding either the total bit budget or the number of bits per coordinate fixed. We report the results in the new Section 6.5. With 24 bits in total, using 24 coordinates instead of four lowered mean sF1 on all four tasks. To address Reviewer ZeZe’s question about quality and speed at larger latent dimensions, we added a separate comparison, again holding the cell count per coordinate fixed (Appendix H.6, Table 32). On all five datasets, RiCoTA sampled faster and achieved better mean scores on all three quality metrics at the original latent dimension than at eight times that dimension. These results support our choice of short latent codes for fast, high-quality generation with fewer autoregressive steps.

We also revised Sections 2–3 and the Figure 1 caption to clarify how geometry guides cell placement in Phase 1, how the Phase 2 prior models cell combinations, and how RiCoTA generates samples from those codes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.