- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper proposes WTN, a low-rank, constrained convex-gradient residual map for repeatedly migrating a 3-bit embedding cache while storing only the latest version. Historical representations are recovered by fixed-point inversion. Strong monotonicity gives a global bijection with a 1-Lipschitz inverse, so recovery error is bounded by the summed quantization and solver errors. Experiments cover four hierarchical benchmarks, DISC21 copy detection, and DINOv2 encoder upgrades.
Critical strengths
- A practical method with theoretical guarantees that acts much better in practice than the theoretical lower bound.
- Despite my stated weaknesses below, I think there is a concrete discovery about the either the WTN adapter or the inverse-based recovery.
Critical weaknesses
[W1] The main experiments change two things at once relative to Raw-Q and Keep-All- the WTN adapter and the inverse-based recovery. The gains could come from the adapter’s added capacity rather than from migration and inversion. An ablation that gives Raw-Q and Keep-All with the same WTN adapter (e.g., applied at read time on the frozen cache) is needed to isolate the contribution of the inverse.
[W2] The paper attributes WTN’s limited historical degradation to its non-expansive inverse, but its own ablations contradict this. Variants that keep a 1-Lipschitz inverse (Unnormalized W, Identity base) still degrade by 8–14 pp, so the inverse property does not appear to be what drives performance.
[W3] In the encoder-upgrade experiments, WTN’s configuration was selected from a search over ranks, scales, learning rates, and codecs with up to 60 epochs, while the learned baselines use fixed architectures and 30 epochs. Under a fixed configuration, WTN falls below the nonlinear low-rank adapter on accuracy and loses its historical-drop advantage.
Questions for authors
- Can the authors address the weaknesses?
- Can the authors justify why they use "wasserstein" in the name of the method? I can not see a concrete enough relation there. Does the proof not hold for general convexity?
- [Relevant Paper] Does this paper give a direct baseline? I would recommend it for the final version if possible, but I think more of my decision depends on the experiments for [W1] and tuning in [W3]. Ricci, Simone, et al. "Backward-compatible aligned representations via an orthogonal transformation layer." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
- [Optional] How would a forward baseline work (instead of doing inverse)? Does it work better? Of course, it would induce latency since the most recent is (generally) the most used.
6 comments
Response to Reviewer WQmQ (Part 2/4: W2)
[W2]. There is no contradiction between these ablations and our theoretical claim. Our error-propagation analysis in Section 3.4 and Appendix A.4 (Equations 16–19 and 43) bounds historical representation-recovery error via
With denoting numerical solver error, this bound identifies two complementary mechanisms for controlling recovery error: (i) non-expansive inversion () prevents perturbation amplification across the inverse chain, and (ii) bounding the per-stage quantization noise limits the perturbations injected at each forward write. Guaranteeing prevents backward error amplification; it does not imply that unnormalized forward maps will inject equally small quantization errors :
- Variants such as Unnormalized , Unnormalized , and Identity base all retain 1-Lipschitz exact inverses, so they do not isolate the effect of ; instead, they lack direction and/or simplex normalization. Specifically, Identity base shows 8.19 and 8.22 pp absolute historical drops on FungiTastic and BIOSCAN-5M (Table 16), while Unnormalized W increases historical drop on FungiTastic by +14.04 pp relative to full WTN (Figure 4). For the single-factor normalization ablations, our direct audit shows that removing direction or simplex normalization expands pre-quantization feature ranges and widens 3-bit quantization step sizes at a nearly constant % clipping rate. We measured all 120 FungiTastic single-factor ablation checkpoints (; Appendix C.1, Table 15) using FP64 damped-Newton reference inverses whose mean per-seed maximum clean reconstruction RMSE is no greater than . Cumulative injected quantization RMSE () jumps 3.50× without direction normalization (0.676 ± 0.008 vs. 0.193 ± 0.009 for full WTN) and 2.71× without simplex weights (0.525 ± 0.022), driving exact-inverse oldest-cache RMSE from 0.161 to 0.408 and 0.302.
- Conversely, when inverse non-expansiveness () is removed, even small quantization errors can amplify across stages: relaxing the unit base (Learnable ) pushes inverse-chain gain up to and degrades FungiTastic historical preservation (+0.89 pp drop; Figure 4, Figure 6a). On FungiTastic, CPL reaches a sampled inverse-chain gain of 64.08 (Table 2); with 200 inverse iterations per stage, its maximum clean reconstruction RMSE is only , yet generation-0 accuracy still drops by 48.477 pp after recovery from the quantized cache (Section 4.3; Appendix C.4). Meanwhile, removing intermediate quantization in our 20-generation WTN chains collapses 20-hop round-trip RMSE to (Table 19).
Thus, direction and simplex normalization bound nonlinear residual displacement and empirically curb injected quantization error under our calibrated codec, while the 1-strongly monotone base guarantees non-expansive inversion () and prevents backward amplification; see Sections 3.4 and 4.3 and Appendices C.1–C.2 (Tables 15–16).
Response to Reviewer WQmQ (Part 3/4: W3 & Q2)
[W3]. To compare all three learned adapters under a single fixed configuration per method with matched capacity, matched schedule, and no hyperparameter search, we evaluated five datasets: Aircraft, iNaturalist-Loc500, Shopify, FungiTastic (128-class subset), and iNaturalist 2019 (128-class subset) (Section 4.5, Table 4). All methods use matched adapter parameters: 33,856 for WTN (), 33,808 for iResNet (), and 33,808 for Nonlinear low-rank (). All models train for 60 epochs per stage across five seeds using AdamW (, , cosine decay, , alignment weight ) and a common validation checkpoint rule:
| Dataset | WTN Accuracy (%) | iResNet Accuracy (%) | Nonlinear Low-Rank Accuracy (%) | Max. Hist. Drop (pp): WTN / iResNet |
|---|---|---|---|---|
| Aircraft | 62.940 ± 0.440 | 63.384 ± 0.397 | 62.322 ± 0.384 | 0.240 ± 0.060 / 0.312 ± 0.069 |
| iNaturalist-Loc500 | 75.645 ± 0.312 | 75.597 ± 0.407 | 75.195 ± 0.097 | 0.056 ± 0.026 / 0.093 ± 0.058 |
| Shopify | 41.424 ± 0.489 | 41.688 ± 0.412 | 41.137 ± 0.690 | 0.012 ± 0.161 / 0.096 ± 0.172 |
| FungiTastic$^\ast$ | 67.952 ± 0.428 | 67.891 ± 0.360 | 66.515 ± 0.833 | 0.151 ± 0.141 / 0.438 ± 0.247 |
| iNaturalist 2019$^\ast$ | 80.521 ± 0.534 | 80.469 ± 0.611 | 79.896 ± 1.320 | 0.156 ± 0.143 / 0.156 ± 0.143 |
All results are mean ± sample SD over five seeds. Fixed 128-fine-class subsets on FungiTastic and iNaturalist 2019. Nonlinear low-rank has no inverse for recovering historical representations from the latest cache (n/a).
Unlike the smaller 17.4K fixed- reference under mismatched schedules in Table 25, fixed-configuration WTN in this budget-matched comparison achieves higher mean latest-task accuracy than Nonlinear low-rank across all 5/5 datasets, and lower mean historical degradation than iResNet on 4/5 datasets while matching the reported mean on the 5th (0.156 pp on iNaturalist 2019; 0.151 vs. 0.438 pp on FungiTastic).
Q2. Regarding the "Wasserstein" name (Section 3.2; Appendix A.7), each single-stage pre-quantization map is the gradient of a convex potential, , which is cyclically monotone (Rockafellar, 1970). By the linear-growth bound in Equation (35), pushing forward any absolutely continuous source feature distribution yields . Brenier's theorem then identifies as the unique (-a.e.) quadratic-cost optimal transport map from to , minimizing expected squared displacement across all transport maps :
Thus, among all maps transporting to its induced distribution , each pre-quantization WTN stage achieves minimal expected squared displacement, motivating the Wasserstein Transport Network name; training itself does not minimize to a prescribed target distribution.
Regarding the proof conditions, our stability guarantees are established using convex analysis without optimizing an explicit loss or requiring quantized multi-stage compositions to remain a single Brenier map; however, they do not follow from general convexity () alone. Convexity alone guarantees neither global bijectivity nor a non-expansive inverse, and does not ensure convergence of the residual fixed-point iteration. Specifically, the lower bound yields global bijectivity and a 1-Lipschitz inverse, while guarantees geometric fixed-point convergence (Equations 11–14 and 29–36). The constraint ladder (Table 16, ) evaluates the practical effect of imposing these stronger conditions: tightening a general convex-gradient map (Learned base) to complete WTN raises FungiTastic accuracy by +10.76 pp (40.274% → 51.035%) and cuts historical drop from 6.782 to 1.389 pp. Separately, validation-selected WTN in the main benchmark serves the oldest version in 3.20–6.21 ms via fixed-point iteration (Table 22); for the constraint ladder, Table 17 reports Newton-based timings of 47.29–78.24 ms using damped Newton on the 5–6-stage Learned base models (6.74 ms with Newton on Shopify) versus 4.90–10.25 ms using Newton for complete WTN.
Response to Reviewer WQmQ (Part 1/4: Q1 / W1)
Thank you for your constructive review and concrete experimental suggestions, particularly the read-time adapter control ([W1]/Q4), the matched-budget encoder-upgrade test ([W3]), and the OCA baseline (Q3). We have completed all of these experiments and integrated them into the revised manuscript, alongside clarified protocol tables in Appendix B.1 (Tables 6–8).
Q1 / [W1]. Following your suggestion, we evaluated three deployment modes sharing the exact same frozen WTN checkpoints, per-stage quantizers, and task heads across all four main benchmarks and five seeds (Section 4.2; Appendix D.4, Table 23):
Under full-batch FP32 evaluation, all three modes yield sample-for-sample identical latest-task predictions across every dataset and seed, and read-time replay reproduces 100% of stored historical predictions across all 105 historical-task–seed pairs (0.000 pp drop). This confirms that forward accuracy gains over Repeated Raw-Q come from the learned WTN mapping, while clarifying the serving role of bijective inversion under a single 144 B cache:
WTN shifts forward computation to write time, giving frequent current-task queries direct O(1) cache reads, and uses inversion only on historical queries when retaining solely the latest cache.