acceptodds
Under review as a conference paper at ICLR 2027

WTN: Low-Rank Invertible Adapters for Low-Bit Embedding Cache Migration

Abstract

Updating low-bit embedding caches for new tasks can disrupt services that depend on earlier representations. We introduce Wasserstein Transport Networks (WTN), a constrained family of low-rank convex-gradient residual maps designed to support both task adaptation and historical recovery. WTN learns a flexible set of transport directions under joint constraints that ensure a fixed residual Lipschitz bound, global bijectivity, and stable inversion. The non-expansive inverse allows the same learned maps used for adaptation to recover historical representations, with recovery errors bounded by accumulated quantization and numerical inversion errors. As a result, WTN requires only the latest cache rather than storing historical representations. Across four hierarchical vision datasets and multiple 3-bit migrations, WTN maintains strong latest-task performance while limiting degradation on historical tasks and preserving the neighborhood structure of older representations.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Representation learning, Invertible neural networks, Convex-gradient maps, Non-expansive inverses, Low-bit quantization, Embedding cache migration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WQmQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper proposes WTN, a low-rank, constrained convex-gradient residual map for repeatedly migrating a 3-bit embedding cache while storing only the latest version. Historical representations are recovered by fixed-point inversion. Strong monotonicity gives a global bijection with a 1-Lipschitz inverse, so recovery error is bounded by the summed quantization and solver errors. Experiments cover four hierarchical benchmarks, DISC21 copy detection, and DINOv2 encoder upgrades.

Critical strengths
  1. A practical method with theoretical guarantees that acts much better in practice than the theoretical lower bound.
  2. Despite my stated weaknesses below, I think there is a concrete discovery about the either the WTN adapter or the inverse-based recovery.
Critical weaknesses

[W1] The main experiments change two things at once relative to Raw-Q and Keep-All- the WTN adapter and the inverse-based recovery. The gains could come from the adapter’s added capacity rather than from migration and inversion. An ablation that gives Raw-Q and Keep-All with the same WTN adapter (e.g., applied at read time on the frozen cache) is needed to isolate the contribution of the inverse.

[W2] The paper attributes WTN’s limited historical degradation to its non-expansive inverse, but its own ablations contradict this. Variants that keep a 1-Lipschitz inverse (Unnormalized W, Identity base) still degrade by 8–14 pp, so the inverse property does not appear to be what drives performance.

[W3] In the encoder-upgrade experiments, WTN’s configuration was selected from a search over ranks, scales, learning rates, and codecs with up to 60 epochs, while the learned baselines use fixed architectures and 30 epochs. Under a fixed configuration, WTN falls below the nonlinear low-rank adapter on accuracy and loses its historical-drop advantage.

Questions for authors
  1. Can the authors address the weaknesses?
  2. Can the authors justify why they use "wasserstein" in the name of the method? I can not see a concrete enough relation there. Does the proof not hold for general convexity?
  3. [Relevant Paper] Does this paper give a direct baseline? I would recommend it for the final version if possible, but I think more of my decision depends on the experiments for [W1] and tuning in [W3]. Ricci, Simone, et al. "Backward-compatible aligned representations via an orthogonal transformation layer." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
  4. [Optional] How would a forward baseline work (instead of doing inverse)? Does it work better? Of course, it would induce latency since the most recent is (generally) the most used.
6 comments
AuthorsOpenReview ↗

Response to Reviewer WQmQ (Part 1/4: Q1 / W1)

Thank you for your constructive review and concrete experimental suggestions, particularly the read-time adapter control ([W1]/Q4), the matched-budget encoder-upgrade test ([W3]), and the OCA baseline (Q3). We have completed all of these experiments and integrated them into the revised manuscript, alongside clarified protocol tables in Appendix B.1 (Tables 6–8).

Q1 / [W1]. Following your suggestion, we evaluated three deployment modes sharing the exact same frozen WTN checkpoints, per-stage quantizers, and task heads across all four main benchmarks and five seeds (Section 4.2; Appendix D.4, Table 23):

  1. Write-time migration (WTN): retain only the latest cache cKc_K (144 B/vector); read cKc_K directly for current tasks and invert via T−1T^{-1} for historical tasks.
  2. Read-time forward replay (Raw-Q + read-time WTN): retain only c0c_0 (144 B/vector); apply the frozen forward chain ck=Qk(Tk(ck−1))c_k = Q_k(T_k(c_{k-1})) on demand up to requested generation jj.
  3. Shared-checkpoint Keep-All oracle: retain all K+1K+1 caches (720–1,008 B/vector) for direct reads at every version.

Under full-batch FP32 evaluation, all three modes yield sample-for-sample identical latest-task predictions across every dataset and seed, and read-time replay reproduces 100% of stored historical predictions across all 105 historical-task–seed pairs (0.000 pp drop). This confirms that forward accuracy gains over Repeated Raw-Q come from the learned WTN mapping, while clarifying the serving role of bijective inversion under a single 144 B cache:

  • Running adapters at read time avoids inversion error (0 pp), but passes every current-task query through all KK forward maps and quantizers, slowing latest-task serving by 6.3–9.2× at batch size 1 on an exclusive NVIDIA L20 (1.008–1.552 ms vs. 0.161–0.183 ms; Table 23).
  • Overwriting the cache at write time with a non-invertible adapter provides no structural inverse recovery of historical representations from the latest cache (n/a; Table 8), whereas retaining all caches (Keep-All) increases sample storage by 5–7× (80.0%–85.7% payload reduction for WTN).

WTN shifts forward computation to write time, giving frequent current-task queries direct O(1)O(1) cache reads, and uses inversion only on historical queries when retaining solely the latest cache.

AuthorsOpenReview ↗

Response to Reviewer WQmQ (Part 2/4: W2)

[W2]. There is no contradiction between these ablations and our theoretical claim. Our error-propagation analysis in Section 3.4 and Appendix A.4 (Equations 16–19 and 43) bounds historical representation-recovery error via

RMSEj(K)≤∑m=j+1KDm+∑m=j+1KHm(when Lk≤1).\mathrm{RMSE} _ j^{(K)} \le \sum _ {m=j+1}^{K} D_m + \sum _ {m=j+1}^{K} H_m \quad (\text{when } L_k \le 1).

With HmH_m denoting numerical solver error, this bound identifies two complementary mechanisms for controlling recovery error: (i) non-expansive inversion (Lk≤1L_k \le 1) prevents perturbation amplification across the inverse chain, and (ii) bounding the per-stage quantization noise DmD_m limits the perturbations injected at each forward write. Guaranteeing Lk≤1L_k \le 1 prevents backward error amplification; it does not imply that unnormalized forward maps will inject equally small quantization errors DmD_m:

  • Variants such as Unnormalized WW, Unnormalized aa, and Identity base all retain 1-Lipschitz exact inverses, so they do not isolate the effect of Lk≤1L_k \le 1; instead, they lack direction and/or simplex normalization. Specifically, Identity base shows 8.19 and 8.22 pp absolute historical drops on FungiTastic and BIOSCAN-5M (Table 16), while Unnormalized W increases historical drop on FungiTastic by +14.04 pp relative to full WTN (Figure 4). For the single-factor normalization ablations, our direct audit shows that removing direction or simplex normalization expands pre-quantization feature ranges and widens 3-bit quantization step sizes at a nearly constant ≈0.20\approx 0.20% clipping rate. We measured all 120 FungiTastic single-factor ablation checkpoints (r=32r=32; Appendix C.1, Table 15) using FP64 damped-Newton reference inverses whose mean per-seed maximum clean reconstruction RMSE is no greater than 2.41×10−112.41 \times 10^{-11}. Cumulative injected quantization RMSE (∑m=16Dm\sum _ {m=1}^{6} D_m) jumps 3.50× without direction normalization (0.676 ± 0.008 vs. 0.193 ± 0.009 for full WTN) and 2.71× without simplex weights (0.525 ± 0.022), driving exact-inverse oldest-cache RMSE from 0.161 to 0.408 and 0.302.
  • Conversely, when inverse non-expansiveness (Lk≤1L_k \le 1) is removed, even small quantization errors can amplify across stages: relaxing the unit base (Learnable α\alpha) pushes inverse-chain gain up to ∼3.0×\sim 3.0{\times} and degrades FungiTastic historical preservation (+0.89 pp drop; Figure 4, Figure 6a). On FungiTastic, CPL reaches a sampled inverse-chain gain of 64.08 (Table 2); with 200 inverse iterations per stage, its maximum clean reconstruction RMSE is only 1.38×10−61.38 \times 10^{-6}, yet generation-0 accuracy still drops by 48.477 pp after recovery from the quantized cache (Section 4.3; Appendix C.4). Meanwhile, removing intermediate quantization in our 20-generation WTN chains collapses 20-hop round-trip RMSE to 9.11×10−139.11 \times 10^{-13} (Table 19).

Thus, direction and simplex normalization bound nonlinear residual displacement and empirically curb injected quantization error under our calibrated codec, while the 1-strongly monotone base guarantees non-expansive inversion (Lk≤1L_k \le 1) and prevents backward amplification; see Sections 3.4 and 4.3 and Appendices C.1–C.2 (Tables 15–16).

AuthorsOpenReview ↗

Response to Reviewer WQmQ (Part 3/4: W3 & Q2)

[W3]. To compare all three learned adapters under a single fixed configuration per method with matched capacity, matched schedule, and no hyperparameter search, we evaluated five datasets: Aircraft, iNaturalist-Loc500, Shopify, FungiTastic (128-class subset), and iNaturalist 2019 (128-class subset) (Section 4.5, Table 4). All methods use matched ≈33.8K\approx 33.8\text{K} adapter parameters: 33,856 for WTN (r=32,τ=3.8r=32, \tau=3.8), 33,808 for iResNet (hidden=16,scale=0.99\mathrm{hidden}=16, \mathrm{scale}=0.99), and 33,808 for Nonlinear low-rank (r=16r=16). All models train for 60 epochs per stage across five seeds using AdamW (lr=0.002\mathrm{lr}=0.002, wd=10−4\mathrm{wd}=10^{-4}, cosine decay, batch=1024\mathrm{batch}=1024, alignment weight λ=1\lambda=1) and a common validation checkpoint rule:

DatasetWTN Accuracy (%)iResNet Accuracy (%)Nonlinear Low-Rank Accuracy (%)Max. Hist. Drop (pp): WTN / iResNet
Aircraft62.940 ± 0.44063.384 ± 0.39762.322 ± 0.3840.240 ± 0.060 / 0.312 ± 0.069
iNaturalist-Loc50075.645 ± 0.31275.597 ± 0.40775.195 ± 0.0970.056 ± 0.026 / 0.093 ± 0.058
Shopify41.424 ± 0.48941.688 ± 0.41241.137 ± 0.6900.012 ± 0.161 / 0.096 ± 0.172
FungiTastic$^\ast$67.952 ± 0.42867.891 ± 0.36066.515 ± 0.8330.151 ± 0.141 / 0.438 ± 0.247
iNaturalist 2019$^\ast$80.521 ± 0.53480.469 ± 0.61179.896 ± 1.3200.156 ± 0.143 / 0.156 ± 0.143

All results are mean ± sample SD over five seeds. ∗^\astFixed 128-fine-class subsets on FungiTastic and iNaturalist 2019. Nonlinear low-rank has no inverse for recovering historical representations from the latest cache (n/a).

Unlike the smaller 17.4K fixed-r=16r=16 reference under mismatched schedules in Table 25, fixed-configuration WTN in this budget-matched comparison achieves higher mean latest-task accuracy than Nonlinear low-rank across all 5/5 datasets, and lower mean historical degradation than iResNet on 4/5 datasets while matching the reported mean on the 5th (0.156 pp on iNaturalist 2019; 0.151 vs. 0.438 pp on FungiTastic).

Q2. Regarding the "Wasserstein" name (Section 3.2; Appendix A.7), each single-stage pre-quantization map is the gradient of a convex potential, Tk=∇ψkT_k = \nabla\psi_k, which is cyclically monotone (Rockafellar, 1970). By the linear-growth bound in Equation (35), pushing forward any absolutely continuous source feature distribution μ∈P2(Rd)\mu \in \mathcal{P} _ 2(\mathbb{R}^d) yields ν=Tk♯μ∈P2(Rd)\nu = T _ {k\sharp}\mu \in \mathcal{P} _ 2(\mathbb{R}^d). Brenier's theorem then identifies TkT_k as the unique (μ\mu-a.e.) quadratic-cost optimal transport map from μ\mu to ν\nu, minimizing expected squared displacement across all transport maps S♯μ=νS _ {\sharp}\mu = \nu:

W22(μ,ν)=inf⁡S:\mspace3muS♯μ=ν∫∥x−S(x)∥22\mspace3mudμ(x)=∫∥x−Tk(x)∥22\mspace3mudμ(x).W_2^2(\mu, \nu) = \inf _ {S:\mspace{3mu} S _ {\sharp}\mu = \nu} \int \lVert x - S(x)\rVert_2^2 \mspace{3mu} d\mu(x) = \int \lVert x - T_k(x)\rVert_2^2 \mspace{3mu} d\mu(x).

Thus, among all maps transporting μ\mu to its induced distribution ν=Tk♯μ\nu = T _ {k\sharp}\mu, each pre-quantization WTN stage achieves minimal expected squared L2L_2 displacement, motivating the Wasserstein Transport Network name; training itself does not minimize W2W_2 to a prescribed target distribution.

Regarding the proof conditions, our stability guarantees are established using convex analysis without optimizing an explicit W2W_2 loss or requiring quantized multi-stage compositions to remain a single Brenier map; however, they do not follow from general convexity (∇2ψk(x)⪰0\nabla^2\psi_k(x) \succeq 0) alone. Convexity alone guarantees neither global bijectivity nor a non-expansive inverse, and does not ensure convergence of the residual fixed-point iteration. Specifically, the lower bound ∇2ψk(x)⪰I\nabla^2\psi_k(x) \succeq I yields global bijectivity and a 1-Lipschitz inverse, while ∥JRk(x)∥2≤τk/4<1\lVert J _ {R_k}(x)\rVert_2 \le \tau_k/4 < 1 guarantees geometric fixed-point convergence (Equations 11–14 and 29–36). The constraint ladder (Table 16, r=16r=16) evaluates the practical effect of imposing these stronger conditions: tightening a general convex-gradient map (Learned base) to complete WTN raises FungiTastic accuracy by +10.76 pp (40.274% → 51.035%) and cuts historical drop from 6.782 to 1.389 pp. Separately, validation-selected WTN in the main benchmark serves the oldest version in 3.20–6.21 ms via fixed-point iteration (Table 22); for the constraint ladder, Table 17 reports Newton-based timings of 47.29–78.24 ms using damped Newton on the 5–6-stage Learned base models (6.74 ms with Newton on Shopify) versus 4.90–10.25 ms using Newton for complete WTN.

Reviewer_CJPJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

WTN achieves low-bit embedding cache through low-rank convex gradient residual maps and uses the same maps' inverses to recover older representations. Normalized directions, simplex-constrained positive coefficients, and a bounded residual scale can obtain a rank-independent residual Lipschitz bound, forming a global bijective map with a non-expansive inverse and contractive fixed-point inversion when the scale is below four. The obtained recovery bound accumulates quantization and numerical inversion errors additively. The experiments cover repeated 3-bit migration on four hierarchical vision datasets, historical retrieval, copy detection with a legacy verifier, and two successive encoder upgrades.

Critical strengths
  1. The paper provides clear guarantees of global bijectivity, non-expansive inversion and bounded error accumulation across repeated low-bit migrations.
  2. The shared-checkpoint Keep-All oracle isolates cache-retention effects, while the experiments quantify the trade-off between storage savings and historical recovery.
Critical weaknesses
  1. Compared with the one-cache Repeated Raw-Q baseline, the gains of WTN are small and encoder-upgrade comparison gives WTN broader tuning and up to 60 epochs versus 30 for learned baselines.
  2. The main-benchmark training settings, stem and baseline definitions, and reproducible Shopify filtering/split details are missing, which make the comparisons difficult to reproduce.
Questions for authors
  1. Does WTN retain its encoder-upgrade advantage when learned baselines receive comparable training and hyperparameter-search budgets?
  2. What do the five paired runs show about uncertainty in the gains over Raw-Q and the resulting accuracy/recovery-cost trade-off?
  3. Can the authors provide the complete training configurations, model definitions and reproducible Shopify filtering and split procedure?
2 comments
AuthorsOpenReview ↗

Response to Reviewer CJPJ (Part 1/2: Q1-Q2)

Thank you for your constructive review regarding matched capacity and tuning budgets, paired statistical uncertainty, and full reproducibility documentation. We have added the controlled five-dataset comparison, paired uncertainty analyses, and consolidated protocol specifications to the revised manuscript and expanded our supplementary code archive.

Q1. To compare all three learned adapters under matched capacity, schedule, and tuning budget, with no hyperparameter search, we evaluated one fixed configuration per method across five datasets: Aircraft, iNaturalist-Loc500, Shopify, FungiTastic (128-class subset), and iNaturalist 2019 (128-class subset) (Section 4.5, Table 4). All methods use matched ≈33.8K\approx 33.8\text{K} adapter capacity: 33,856 parameters for WTN (r=32,τ=3.8r=32, \tau=3.8), 33,808 for iResNet (hidden=16,scale=0.99\mathrm{hidden}=16, \mathrm{scale}=0.99), and 33,808 for Nonlinear low-rank (r=16r=16). All models train for 60 epochs per stage across five seeds using AdamW (lr=0.002\mathrm{lr}=0.002, wd=10−4\mathrm{wd}=10^{-4}, cosine decay, batch=1024\mathrm{batch}=1024, alignment weight λ=1\lambda=1) and a common validation checkpoint rule:

DatasetWTN Accuracy (%)iResNet Accuracy (%)Nonlinear Low-Rank Accuracy (%)Max. Hist. Drop (pp): WTN / iResNet
Aircraft62.940 ± 0.44063.384 ± 0.39762.322 ± 0.3840.240 ± 0.060 / 0.312 ± 0.069
iNaturalist-Loc50075.645 ± 0.31275.597 ± 0.40775.195 ± 0.0970.056 ± 0.026 / 0.093 ± 0.058
Shopify41.424 ± 0.48941.688 ± 0.41241.137 ± 0.6900.012 ± 0.161 / 0.096 ± 0.172
FungiTastic$^\ast$67.952 ± 0.42867.891 ± 0.36066.515 ± 0.8330.151 ± 0.141 / 0.438 ± 0.247
iNaturalist 2019$^\ast$80.521 ± 0.53480.469 ± 0.61179.896 ± 1.3200.156 ± 0.143 / 0.156 ± 0.143

All results are mean ± sample SD over five seeds. ∗^\astFixed 128-fine-class subsets. Nonlinear low-rank has no inverse for recovering historical representations from the latest cache (n/a).

Under matched capacity and training duration, WTN achieves higher mean latest-task accuracy than Nonlinear low-rank across all 5/5 datasets, whereas Nonlinear low-rank has no inverse to recover historical representations from the latest cache. Relative to iResNet, WTN achieves lower mean historical degradation on 4/5 datasets and matches the reported mean on the 5th (0.156 pp on iNaturalist 2019), while reaching higher mean latest-task accuracy on 3/5 datasets; on FungiTastic, mean historical drop decreases by 65.5% (0.438 → 0.151 pp). Both the validation-selected benchmark (Table 3; protocol in Appendix E.2) and this budget-matched comparison (Table 4) are reported side-by-side in Section 4.5.

Q2. Across all 20 dataset–seed pairs in the main benchmark (seeds 20–24), WTN outperforms Repeated Raw-Q in 17/20 paired runs: 5/5 on FungiTastic, 5/5 on BIOSCAN-5M, 4/5 on Shopify, and 3/5 on iNaturalist 2019 (Section 4.2; Appendix B.2, Table 10):

DatasetTest Samples / Fine ClassesPositive PairsPaired Gain ± SD (pp)95% Paired tt-Interval (pp)Two-sided ppHolm-adjusted pp
iNaturalist 20193,030 / 1,0103/5+0.1254 ± 0.3859[-0.3537, +0.6045]0.50760.5076
Shopify1,671 / 2924/5+0.3112 ± 0.2409[+0.0121, +0.6103]0.04460.0892
FungiTastic91,058 / 2,7565/5+0.3005 ± 0.0487[+0.2400, +0.3609]0.000160.00048
BIOSCAN-5M39,373 / 11,8465/5+0.4892 ± 0.0673[+0.4056, +0.5727]0.0000840.00034

WTN's gains remain statistically significant after Holm correction on the two largest, finest-grained benchmarks, FungiTastic and BIOSCAN-5M, with low variance (SD<0.07 pp\mathrm{SD} < 0.07\text{ pp}). Shopify has a positive unadjusted 95% confidence interval (p=0.0446p = 0.0446), though its Holm-adjusted pp-value is 0.08920.0892, and iNaturalist 2019 shows wider variance on its smaller official test split (3,030 samples, where 1 sample equals 0.033 pp), with the interval crossing zero. While gains on a single frozen DINOv2-S backbone are modest, WTN delivers up to +3.94 pp over static caches when encoders upgrade (Section 4.5, Table 3).

Regarding the cost trade-off: both WTN and Raw-Q retain a single 144 B/vector cache, with WTN trading 0.323–1.027 pp maximum historical drop and 3.198–6.206 ms oldest-version inverse latency for representation adaptation (Table 22). Holding the learned WTN chain fixed further shows that materializing updates at write time serves latest-task requests 6.3–9.2× faster at batch size 1 on an exclusive L20 than read-time forward replay (0.161–0.183 ms vs. 1.008–1.552 ms; Appendix D.4, Table 23), while reducing storage by 80.0%–85.7% compared with storing all cache versions.

AuthorsOpenReview ↗

Response to Reviewer CJPJ (Part 2/2: Q3)

Q3. Sections 3.1, 4.1, and Appendix B.1 (Tables 5–8) summarize the shared protocol, model definitions, and comparison methods in the manuscript, while the supplementary code archive expanded from our initial submission maps every reported table and figure to its runnable scripts, frozen per-stage settings (configs/), and split records (data/splits/) via README.md, paper_manifest.csv, and RUNBOOK.md:

  • Generation 0 maps frozen 384-D DINOv2-S CLS features through a shared stem (Linear-LayerNorm-GELU-Dropout); each stage k≥0k \ge 0 uses a LayerNorm-Linear head and a train-only 3-bit uniform quantizer ([0.001, 0.999] quantiles; Tables 5–6, 8). Appendix B.1 outlines the baseline setups and historical interfaces, including CPL (residual Lipschitz bound 0.9, hidden size 16) and the parameter-matched low-rank controls, while full model, baseline, and training-loss code is in wtn/, with OCA in experiments/oca.py.
  • In the four-dataset hierarchical benchmark (Table 6), each migration trains for 30 epochs with AdamW (wd=10−4\mathrm{wd}=10^{-4}, cosine decay), whereas the separate encoder-upgrade comparison in Table 4 uses 60 epochs per stage. Learning rates and batch sizes are selected on seed 20 validation data and replayed across seeds 20–24, while per-stage inverse budgets (m≤100m \le 100) are calibrated on ≤256\le 256 validation samples per checkpoint via Equation (48) without accessing test data.
  • For the Shopify split, retaining depth-5 taxonomy prefixes with ≥10\ge 10 official training examples defines 292 classes (14 → 27 → 57 → 131 → 292). A stratified 80/20 split (seed 20) yields 5,628 train and 1,407 validation samples, and applying the frozen class map to the official test split gives 1,671 test samples without test leakage.
  • Auditing all 210 main-benchmark WTN checkpoints (Table 7) confirmed a uniform rank rk=16r_k = 16 (6,560–6,561 parameters per stage). Validation selected learnable τk∈(0,3.8)\tau_k \in (0, 3.8) (initialized at 1.8) on iNaturalist 2019, FungiTastic, and BIOSCAN-5M, and fixed τk=3.8\tau_k = 3.8 on Shopify. Table 9 reports both fixed- and learnable-scale variants, with the learnable-scale variant on Shopify using cap τmax⁡,k=2.0\tau _ {\max,k} = 2.0. Six cells in the initial submission's rank table had inadvertently listed auxiliary-sweep ranks; all main-table results are unchanged and Table 7 has been corrected.
Reviewer_axEzOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Wasserstein Transport Networks (WTN), a low-rank invertible adapter for multi-generation low-bit embedding cache migration. WTN enables adaptation to new tasks and recovery of historical representations using the same invertible mapping, without retaining multiple cache versions. The authors establish theoretical guarantees for global invertibility, non-expansive inversion, and bounded recovery errors. Experiments on four hierarchical datasets demonstrate competitive latest-task accuracy, limited historical degradation, and efficient recovery across repeated 3-bit migrations. Additional experiments evaluate image retrieval and encoder upgrades.

Critical strengths
  • Practical and meaningful problem. The paper addresses multi-generation embedding cache migration, allowing historical representations to be recovered without storing multiple cache versions.

  • Theoretically grounded method. WTN provides guarantees on global invertibility, non-expansive inversion, and bounded recovery errors, which are important for stable low-bit cache migration.

  • Comprehensive experimental evaluation. The authors compare against various invertible and backward-compatible methods across multiple datasets, with extensive ablations supporting the effectiveness of the proposed constraints.

Critical weaknesses
  • Limited methodological novelty. WTN is a constrained instance of existing monotone gradient networks, as acknowledged in Appendix A.6. While the proposed constraints provide useful stability guarantees, the architectural novelty appears relatively limited.

  • Marginal improvements over simple baselines. WTN improves latest-task accuracy over Repeated Raw-Q by only 0.13–0.49 percentage points, while introducing additional computation and historical recovery errors. This raises questions about the practical necessity of learned embedding migration.

  • Restricted expressiveness. The positive semidefinite residual Jacobian prevents WTN from contracting distances, potentially limiting its ability to handle complex representation changes. The trade-off between expressiveness and inversion stability is not sufficiently investigated.

  • Limited retrieval benefits. Despite competitive classification accuracy, WTN does not achieve the best mixed-gallery retrieval performance. This weakens its practical motivation for embedding retrieval applications.

  • Questionable low-rank assumption. In the iNaturalist-Loc500 encoder-upgrade experiment, one WTN stage uses rank 1024, equal to the embedding dimension. This raises concerns about whether low-rank adaptation is sufficient for substantial representation changes.

  • Limited long-term validation. The main experiments cover only 4–6 migrations. Although WTN guarantees non-expansive inversion, quantization errors still accumulate across generations, and its performance over longer migration chains remains unclear.

Questions for authors
  1. Could the authors clarify which theoretical or architectural properties of WTN are fundamentally new compared with existing monotone gradient networks?

  2. Given the small improvements over Repeated Raw-Q, can the authors demonstrate scenarios where learned migration provides substantial benefits?

  3. How does the restriction against distance contraction affect WTN's representation learning capability?

  4. Why does WTN fail to outperform simpler baselines in mixed-gallery retrieval, despite improving classification accuracy?

  5. Why is a full-rank configuration (r = 1024) selected in the iNaturalist-Loc500 encoder-upgrade experiment? How does performance change with smaller ranks?

  6. Can the authors provide evidence of stable historical recovery over longer migration sequences, beyond the evaluated 4–6 generations?

5 comments
AuthorsOpenReview ↗

Response to Reviewer axEz (Part 1/3: Q1-Q2)

Thank you for your detailed review of WTN's parameterization, expressiveness, retrieval behavior, rank scaling, and long-horizon stability. Below we address each of your six questions with theoretical clarifications and controlled experiments integrated into the revised manuscript, alongside reorganized protocol summary tables in Appendix B.1 (Tables 6–8).

Q1. General Monotone Gradient Networks (G(x)=Ax+c+W⊤ϕ(Wx+b)\mathcal{G}(x) = Ax + c + W^\top \phi(Wx + b) with A⪰0A \succeq 0; Appendix A.6) guarantee monotonicity, but do not by themselves prevent inverse perturbation amplification, bound residual Jacobians independently of rank, or provide fast contractive fixed-point inversion. WTN introduces a tightly coupled low-rank parameterization tailored to repeated low-bit cache migration (Sections 2.2 and 3.2–3.3):

  • Unit-norm directions (∥wk,ℓ∥2≤1\lVert w_{k,\ell}\rVert_2 \le 1) and simplex weights (ak,ℓ>0,∑ℓ=1rkak,ℓ=1a_{k,\ell} > 0, \sum_{\ell=1}^{r_k} a_{k,\ell} = 1) confine sample-dependent displacements to an rkr_k-dimensional subspace (∥Tk(x)−x−tk∥2≤τk\lVert T_k(x) - x - t_k\rVert_2 \le \tau_k) while capping the residual Jacobian norm at ∥JRk(x)∥2≤τk/4\lVert J_{R_k}(x)\rVert_2 \le \tau_k / 4 regardless of rank rkr_k.
  • Fixing the identity base (A=IA = I) enforces 1-strong monotonicity (JTk(x)⪰IJ_{T_k}(x) \succeq I), guaranteeing global bijectivity and a 1-Lipschitz exact inverse (∥Tk−1(u)−Tk−1(v)∥2≤∥u−v∥2\lVert T_k^{-1}(u) - T_k^{-1}(v)\rVert_2 \le \lVert u - v\rVert_2) that does not amplify accumulated quantization noise.
  • Enforcing 0<τk<40 < \tau_k < 4 yields geometric fixed-point convergence at rate τk/4<1\tau_k/4 < 1 and cost O(rd)O(rd) per iteration using the forward parameters directly, avoiding Newton, L-BFGS, or operator-splitting solvers.

Our constraint ladder (Section 4.3; Appendix C.2, Tables 16–17, fixed scale, r=16r=16) isolates the empirical contribution of each constraint beyond generic convex-gradient structure. Progressing from an MGN-style parameterization (Learned base) to complete WTN raises FungiTastic latest-task accuracy by +10.76 pp (40.274% → 51.035%) and cuts final-cache maximum historical degradation by 79.5% (6.782 → 1.389 pp); historical drops likewise fall from 2.040 to 0.614 pp on iNaturalist 2019, 1.628 to 0.323 pp on Shopify, and 7.520 to 1.041 pp on BIOSCAN-5M. Table 17 reports Newton-based inversion timings for the constraint ladder: 47.29–78.24 ms for Learned base using damped Newton on the three 5–6-stage benchmarks (6.74 ms with Newton on Shopify) versus 4.90–10.25 ms for complete WTN using Newton across all four benchmarks. Separately, validation-selected WTN in the main benchmark serves the oldest version in 3.20–6.21 ms via fixed-point iteration (Table 22).

Q2. While gains over static coordinate reuse (Repeated Raw-Q / No migration) are modest when only label granularity changes on a single frozen backbone, learned migration provides larger improvements under encoder upgrades and domain shifts:

  • In same-backbone hierarchy refinement (Section 4.2, Table 1; Appendix B.2, Table 10), all generations share a frozen DINOv2-S encoder, so static features remain well aligned. Even in this conservative regime, WTN outperforms Raw-Q (+0.125 to +0.489 pp) in 17 of 20 paired seed runs across the four benchmarks, with 5/5 wins and statistical significance after Holm correction on the two largest, finest-grained tasks, FungiTastic (p=0.00048p = 0.00048) and BIOSCAN-5M (p=0.00034p = 0.00034), while keeping maximum historical drop within 0.323–1.027 pp.
  • Under cross-backbone encoder upgrades (DINOv2-S → B → L; Section 4.5, Table 3), WTN raises Aircraft latest-task accuracy from 61.128% (No migration, the direct analogue of Raw-Q) to 65.065% (+3.94 pp) with 0.144 pp historical degradation; holding the first-stage model fixed and setting only the second-stage rank to r=16r=16 still reaches 64.968% (+3.84 pp) with 0.210 pp degradation (Table 27). Under photometric and geometric domain perturbations on DISC21 (Section 4.4; Appendix E.1, Table 24), adapting cached features lifts candidate Recall@100 from 71.834% to 73.422% (+1.59 pp) while retaining a mean of 99.0% of a frozen legacy verifier's correct decisions.
  • Once representations are adapted, bijective write-time migration using identical learned WTN checkpoints produces sample-wise identical latest-task predictions to read-time forward replay while serving latest-task queries 6.3–9.2× faster at batch size 1 on an exclusive NVIDIA L20 (0.161–0.183 ms vs. 1.008–1.552 ms; Appendix D.4, Table 23), and reduces sample-cache storage by 80.0%–85.7% (144 B vs. 720–1,008 B/vector) relative to retaining all cache generations.
AuthorsOpenReview ↗

Response to Reviewer axEz (Part 2/3: Q3-Q4)

Q3. Preventing forward distance contraction (JTk(x)=I+JRk(x)⪰IJ_{T_k}(x) = I + J_{R_k}(x) \succeq I) is a geometric safeguard: if an invertible map contracts the Euclidean distance between a pair of inputs, its inverse expands the distance between the corresponding outputs (Lk>1L_k > 1), which can amplify low-bit quantization perturbations across migration hops.

To test directly whether forbidding distance contraction limits task adaptation, our matched Learnable α\alpha single-factor ablation relaxes the unit base to α=0.25+softplus⁡(u)\alpha = 0.25 + \operatorname{softplus}(u), permitting α<1\alpha < 1 at the dataset-specific protocol ranks specified in Appendix C.1 (Section 4.3, Figure 4; Table 14 and Figure 6). Across all four benchmarks, allowing α\alpha to vary pushes inverse-chain perturbation gain above 1 (up to ∼3.0×\sim 3.0{\times} on FungiTastic and ∼2.5×\sim 2.5{\times} on iNaturalist 2019) without yielding a statistically significant latest-accuracy gain on any dataset. On FungiTastic, enforcing α=1\alpha = 1 improves latest accuracy by +0.408 ± 0.093 pp (51.191% vs. 50.783%) and lowers maximum historical degradation by 0.892 ± 1.145 pp (1.281 vs. 2.173 pp). Together with our 1024-D rank study showing near-identical accuracy from r=16r=16 to r=1024r=1024 (Table 27), these results indicate that enforcing α=1\alpha = 1 preserves task adaptation accuracy while maintaining non-expansive inversion.

Q4. Classification and mixed-gallery retrieval impose distinct geometric criteria: classification optimizes class separability jointly with a learned task head, whereas mixed-gallery retrieval directly ranks migrated legacy vectors against native new-encoder vectors via raw similarity in a shared space.

Our controlled five-dataset comparison (≈33.8K\approx 33.8\text{K} adapter parameters, 60 epochs per stage, 5 seeds; Section 4.5, Table 4) evaluates classification, mixed-gallery retrieval, and historical preservation side-by-side:

  • Among invertible methods that support historical recovery, WTN closely tracks iResNet on mixed-gallery mAP@10 across all five datasets (within 0.0002–0.0012), while achieving lower mean historical degradation on 4/5 datasets and matching the reported mean on the 5th (Table 4). In main-benchmark historical exact retrieval, WTN also retains more neighbors at the oldest version: 0.9006–0.9675 five-seed mean Top-10 overlap in Table 11, and higher overlap than iResNet across all four benchmarks in the seed-20 comparison of Table 12 (Appendix B.3).
  • Compared with Nonlinear low-rank, the latter attains slightly higher mean mAP@10 in Table 4 (+0.0001 to +0.0089), but has no inverse to recover historical representations from the latest cache (n/a) and trails WTN in mean classification accuracy across all 5/5 datasets (by up to +1.44 pp on FungiTastic). Under the separate protocol in Appendix E.2 (Table 25), fixed-r=16r=16 WTN reaches 0.3842 mAP@10 on iNaturalist-Loc500, exceeding every baseline in Table 3 (0.3691–0.3833) and showing that WTN's parameterization does not impose an inherently low retrieval ceiling.
  • In targeted diagnostics on iNaturalist-Loc500 (Appendix E.2, Table 28), re-selecting second-stage checkpoints by validation mAP@10 changes test mAP@10 from 0.3691 to 0.3695 for WTN and from 0.3727 to 0.3757 for iResNet. The final 3-bit quantization write changes WTN mAP@10 from 0.369154 to 0.369133 (-0.000021). Neither check removes the observed mAP@10 gap between WTN and iResNet. All three metrics are reported jointly in Section 4.5 (Table 4) and Appendix E.2 (Table 28).
DatasetWTN: Acc (%) / mAP@10 / Drop (pp)iResNet: Acc (%) / mAP@10 / Drop (pp)Nonlinear LR: Acc (%) / mAP@10 / Drop
Aircraft62.94 ± 0.44 / 0.1491 / 0.24 ± 0.0663.38 ± 0.40 / 0.1487 / 0.31 ± 0.0762.32 ± 0.38 / 0.1492 / n/a
iNat-Loc50075.64 ± 0.31 / 0.3717 / 0.06 ± 0.0375.60 ± 0.41 / 0.3719 / 0.09 ± 0.0675.20 ± 0.10 / 0.3782 / n/a
Shopify41.42 ± 0.49 / 0.2044 / 0.01 ± 0.1641.69 ± 0.41 / 0.2050 / 0.10 ± 0.1741.14 ± 0.69 / 0.2079 / n/a
FungiTastic$^*$67.95 ± 0.43 / 0.1938 / 0.15 ± 0.1467.89 ± 0.36 / 0.1950 / 0.44 ± 0.2566.52 ± 0.83 / 0.2027 / n/a
iNat 2019$^*$80.52 ± 0.53 / 0.4504 / 0.16 ± 0.1480.47 ± 0.61 / 0.4502 / 0.16 ± 0.1479.90 ± 1.32 / 0.4561 / n/a

All results are mean ± sample SD over five seeds (mAP@10 SDs: 0.0002–0.0017). ∗^*Fixed 128-fine-class subsets on FungiTastic and iNaturalist 2019; ≈33.8K\approx 33.8\text{K} adapter parameters and 60 epochs per stage across all methods (Section 4.5, Table 4).

AuthorsOpenReview ↗

Response to Reviewer axEz (Part 3/3: Q5-Q6)

Q5. Full rank is not needed for encoder upgrades; r=1024r=1024 in Table 3 arose from unrestricted validation selection across nearly tied candidates. To isolate the effect of rank, we fixed the first-stage model, its 3-bit cache, and all optimization settings on both 1024-D encoder-upgrade benchmarks, varying only the second-stage rank across r∈{16,128,1024}r \in \{16, 128, 1024\} over five seeds (Section 4.5; Appendix E.2, Table 27):

DatasetSecond-stage Rank rrAdapter ParametersLatest Accuracy (%)Mixed mAP@10Max. Hist. Drop (pp)
iNaturalist-Loc500r=16r = 1617,440 (-98.34%)76.853 ± 0.0990.3706 ± 0.00040.010 ± 0.041
iNaturalist-Loc500r=128r = 128132,352 (-87.41%)76.808 ± 0.1550.3708 ± 0.00030.013 ± 0.034
iNaturalist-Loc500r=1024r = 10241,051,648 (baseline)76.819 ± 0.1460.3707 ± 0.00040.011 ± 0.031
Aircraftr=16r = 1617,440 (-98.34%)64.968 ± 0.3280.1483 ± 0.00040.210 ± 0.095
Aircraftr=128r = 128132,352 (-87.41%)65.203 ± 0.2220.1478 ± 0.00060.162 ± 0.089
Aircraftr=1024r = 10241,051,648 (baseline)65.191 ± 0.1480.1479 ± 0.00070.120 ± 0.037

Compact low-rank WTN (r=16r=16) cuts adapter parameters by 98.34% with proportional O(rd)O(rd) directional compute savings. It matches r=1024r=1024 accuracy (76.85% vs. 76.82% on iNat-Loc500; 64.97% vs. 65.19% on Aircraft, +3.84 pp above No migration) while maintaining small historical drop; both comparisons are included in Appendix E.2 (Table 27).

Q6. We extended FungiTastic and Shopify to 20 sequential 3-bit migrations by cycling through their task hierarchies across five training seeds (Section 4.3; Appendix C.5, Tables 18–19). At generation 20, WTN achieves higher mean latest-task accuracy than both iResNet and Raw-Q: 89.202 ± 0.231% versus 88.831 ± 0.128% and 89.098 ± 0.092% on FungiTastic, and 40.108 ± 0.559% versus 39.629 ± 0.470% and 39.437 ± 0.569% on Shopify. At the same time, WTN reduces mean maximum historical degradation relative to iResNet by 6.82 pp on FungiTastic (10.532 ± 1.162 vs. 17.347 ± 3.353 pp) and by half (-16.85 pp) on Shopify (16.697 ± 7.255 vs. 33.549 ± 3.689 pp).

Auditing all ten frozen 20-generation WTN chains on 512 paired samples per chain decomposes the error terms in Equation (43) (Table 19):

  • WTN's fixed-point deployment solver yields nearly identical oldest-cache recovery RMSEs to an FP64 reference inverse (2,000 max steps, 10−1110^{-11} tolerance): 0.486012 versus 0.486007 on FungiTastic and 0.619175 versus 0.619174 on Shopify, with cumulative solver error ∑m=120Hm\sum_{m=1}^{20} H_m of only 6.44×10−56.44 \times 10^{-5} and 1.54×10−41.54 \times 10^{-4}.
  • Removing intermediate 3-bit quantization reduces mean 20-stage round-trip RMSE to 9.11×10−139.11 \times 10^{-13} on FungiTastic and 1.03×10−121.03 \times 10^{-12} on Shopify.
  • Cumulative per-stage quantization noise (∑m=120Dm=0.6265\sum_{m=1}^{20} D_m = 0.6265 and 0.7685) exceeds cumulative solver error (∑m=120Hm\sum_{m=1}^{20} H_m) by a factor of ≈5.0×103\approx 5.0 \times 10^3–9.7×1039.7 \times 10^3. Together with these near-zero unquantized round-trip errors, the diagnostics identify repeated quantization as the dominant source of recovery error in the tested chains, while non-expansive inversion controls its propagation through the additive upper bound in Equation (43).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.