- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents ibUMAP, a coherent field‑based synchronous variant of UMAP. Instead of in‑place updates from stochastic negative sampling in standard UMAP, ibUMAP computes attraction and degree‑weighted repulsion synchronously from a shared embedding snapshot, and efficiently evaluates the repulsive field via three‑scalar‑moment decomposition together with interpolation‑based FFT (ibFFT). It achieves considerable speedups over umap‑learn on CPU and cuML on million‑scale GPU datasets, and substantially improves geometric stability across unseeded reruns, at the cost of minor degradation in local and global embedding fidelity. Through fine‑grained ablation studies, the paper disentangles the individual impacts of synchrony, kernel capping, FFT approximation and safeguard mechanisms on embedding quality. A single‑cell analytics workflow further demonstrates that better geometric stability does not guarantee consistent downstream clustering outputs, offering a new optimization paradigm for large‑scale reproducible dimensionality‑reduction tasks.
Critical strengths
-
Propose a new synchronous field-driven UMAP optimization paradigm: Breaking away from the original UMAP framework of random negative sampling and local updates, it uses fixed embedded snapshots to synchronously calculate attraction and degree-weighted repulsion forces, using three-scalar moment decomposition + interpolation FFT to efficiently compute the repulsion field, addressing the root cause of poor reproducibility caused by sampling order in traditional UMAPs from the optimization mechanism level, rather than simply accelerating engineering processes.
-
Rigorous Design of Control Variable Ablation Experiments: Fixed graph structure and initialization, decoupling mechanisms such as synchronous updates, nuclear truncation, FFT approximation, and gradient protection, quantifying each change individually on local and global embedding fidelity; Distinguish between FFT approximate errors and dynamic changes caused by synchronous updates, clearly identifying the benefits and costs of each component.
-
Balancing Speed and Seedless Stability: The CPU achieves significant acceleration compared to umap-learn, and GPUs on millions of datasets also show speed improvements compared to cuML; In seedless mode, it can greatly improve the geometric consistency of embedded runs multiple times, eliminating the need for deterministic output by forcing serial execution like in the original UMAP; At the same time, it reveals that "geometric stability ≠ downstream clustering results are completely consistent," providing an objective and complete assessment without exaggerating the effect.
-
Comprehensive and fair multi-benchmark large-scale benchmarking: Compares mainstream implementations such as umap-learn, cuML, and TorchDR across 71 real datasets covering tables, images, and single-cell. It simultaneously evaluates both seeded and seedless operation modes, and reports multi-dimensional indicators such as runtime, local fidelity, global fidelity, and multi-run stability; Distinguish between dataset weighting and family weighting to avoid statistical bias caused by large homologous single-cell datasets.
-
Verification and reproducibility for real downstream scenarios: Using single-cell BRAQUE analysis pipelines as end-to-end cases to study how embedded fluctuations are transmitted to HDBSCAN clustering results; Complete open-source code, test scripts, demos; Distinguish between cross-device and same-device determinism, clearly state the boundaries of algorithm capability, fully disclose extreme cases of protection mechanism failure, and ensure all experimental conditions and hardware parameters are reproducible.
Critical weaknesses
- Embedding Fidelity Inherent Cost: Compared to the original UMAP, ibUMAP generally shows lower local neighborhood retention and global distance correlation indicators. The synchronous update changes the optimization dynamics of the original UMAP, not equivalent replacement. Users need to weigh between fidelity, speed, and reproducibility, and there is no situation where universality is superior to the original UMAP.
- Deterministic Device Limitations: The fully reproducible bit level implemented in the paper only ensures same hardware, same FFT library, and same compiler environment; It is impossible to achieve exact bit consistency across different GPU/CPU devices; it only improves geometric stability and cannot achieve true cross-platform reproducibility. GPU seed mode Under millions of large datasets, graph construction phase overhead skyrockets, resulting in huge runtime penalties.
- Lack of System Tuning for Protection Hyperparameters: To prevent point escape, thresholds such as repulsion force modulation and height node attractive damping are directly fixed during development, with no systematic parameter scanning or tuning; While extreme crashes can be avoided, these hyperparameters affect the final embedding effect and may not be optimal for different datasets.
- FFT Mesh Approximate Error: ibFFT uses 'p=1' piecewise constant interpolation for field solving, and the mesh size adaptively fits into the coordinate range. Although experiments show little effect on the average final embedding quality, near-field interactions have inherent approximate errors; There is no system ablation for mesh accuracy on different datasets and scales, and in extreme cases, it can approximate the computational accuracy of the data.
- Stability and returns have boundaries, downstream performance cannot be fully guaranteed: Even if embedding geometric similarity increases significantly, downstream clustering (HDBSCAN) results cannot be guaranteed to be stable and consistent; clustering allocation in seedless mode still shows considerable variation; Algorithms can only improve fluctuations at the embedding layer and cannot eliminate uncertainties brought by subsequent downstream analysis.
- Limitations in Applicable Output Dimensions: Currently, the implementation is only for 2D dimensionality reduction visualization scenarios and has not been extended to higher output dimensions (3D and above). In high-dimensional scenarios, the memory and computing power costs of FFT field computing surge, and the paper does not explore the feasibility and performance of high-dimensional fields.
- Performance Disadvantages of Small Datasets: On small datasets with less than 1000 sample sizes, ibUMAP's CPU runs slower than the original umap-learn. FFT and grid processing bring fixed overhead, making it more suitable for medium and large datasets, where small sample scenarios are not advantageous.
Questions for authors
- The article observes that ibUMAP sacrifices partial and global embedding fidelity in exchange for speed and stability. Are there future design improvement plans to minimize the fidelity gap with standard UMAPs while preserving the advantages of synchronous field optimization? Will you consider hybrid strategies, such as using synchronous fields early in training and switching back to random negative sampling later?
- Currently, ibUMAP only supports 2D dimensionality reduction. If expanded to 3D or higher output dimensions, the computing power and memory overhead of FFT field computation will increase significantly. What do you think is the biggest technical bottleneck in extending this field drive paradigm to high-dimensional dimensionality reduction, and is there a preliminary feasible technical route?
- The rejection norm clipping to prevent point escape and the attraction damping of hub nodes are two protective hyperparameters, which are fixed values and have not been adjusted by system scanning. Are there plans to develop adaptive hyperparameter strategies in the future that automatically determine clipping thresholds and damping coefficients based on dataset characteristics, to avoid poor performance of fixed parameters on certain datasets?
- This algorithm can only achieve bit-level reproducibility on the same device; complete consistency across hardware and FFT libraries cannot be achieved. Is there a way to further improve and achieve deterministic cross-platform embedded outputs? If not, what metrics can be used to quantify acceptable deviations between different device outputs?
- Experiments show: Even though ibUMAP greatly improves the geometric stability of embeddings, the downstream HDBSCAN clustering results still show significant variation when running seedless. Besides the residual fluctuations of embeddings themselves, what other factors dominate changes in cluster distribution? Is it possible to impose constraints at the optimizer level to further reduce downstream clustering instability?
2 comments
Thank you for your reply and answer my question, but I believe the quality of this article still needs improvement, and I maintain my score
We thank the reviewer for the thorough review. The questions prompted us to conduct some additional experiments and tests, which we have added to the revised paper (changes in blue).
Q1 / W1: Embedding Fidelity Inherent Cost and Hybrid Strategy
We compared the two CPU optimizers at several epoch budgets on 20 datasets, starting from the same graph and initialization for each dataset. We extended ibUMAP to 1,000 epochs to see whether longer runs would close the gap. On six datasets, ibUMAP's best tested neighborhood preservation (NP) was still 0.013–0.033 below unseeded umap-learn at 200 epochs. We report the extended results in Appendix C.6 under Matched fidelity.
We also tried a hybrid strategy in a 60-dataset pilot. We started with the field optimizer and switched to sampled UMAP for the final 40% of epochs. This raised NP by a median 0.021 over pure ibUMAP but took about 11× as long. We report the pilot in §7 and Appendix C.6. A next step would be to refine only poorly preserved neighborhoods and compare fidelity and repeatability at equal layout time.
Q2 / W6: Limitations in Applicable Output Dimensions
Grid memory is the main bottleneck we anticipate. In d output dimensions, with L padded nodes per axis, each of the d+1 fields needs Ld sites and O(Ldlog(Ld)) FFT work. We have outlined an approach for 3D in §7 that sizes the mesh per axis under a fixed memory budget and checks the field against direct summation.
Q3 / W3: Lack of System Tuning for Protection Hyperparameters
We have added a ten-dataset sweep in Appendix B.2. Without norm clipping, points escaped on three datasets, including 555 on CIFAR-10. Clip multipliers 1, 2, and 4 met our safety and quality criteria at a noise-adjusted tolerance with a 0.005 floor. Damping quantiles p95 and p99 and exponents 0.25 and 0.5 also met this criterion when varied separately. Turning off damping destabilized scDEED CART. The clip threshold already scales with the learning rate, and damping uses each graph's degree quantile. We have outlined choosing a bounded multiplier from early force norms as a further adaptation.
W4: FFT Mesh Approximate Error
We have added a direct-sum comparison with the same capped kernel in Appendix B.3. At random targets on 15 frozen layouts from five datasets (1.8k–11.4k points), the median of per-layout median relative vector errors was 22.7% at p=1, 0.80% at p=2, and 0.30% at p=3, with the default cap and box count. In those layouts, same-box pairs contributed zero p=1 model error. Most error came from pairs one to two cap radii apart. In final fits on 28 datasets, raising p or doubling the box count changed the five fidelity scores by at most 0.0023 in paired means but took 1.17–1.93× the layout time.
Q4 / W2: Deterministic Device Limitations
Agreed: we claim same-device repeatability; bitwise cross-platform equality would need fixed-order reductions and one FFT on every backend. We have added a matched-input CPU–CUDA test in Appendix C.7. On ten datasets, seeded reruns agreed bitwise on each backend, with 0 of 10 bitwise matches across backends. Across devices, median 15-NN overlap was 0.876 (range 0.646–0.946), median distance rank correlation was 0.9988, and median Procrustes disparity was 0.0009. For cross-device use, we would reuse the graph and initial coordinates and assess local and global deviations against task-specific tolerances. Portable bitwise agreement would require matching arithmetic, reduction order, sampling calendars, and FFT computation. The seeded GPU overhead at million-point scale comes chiefly from exact neighbor search during graph construction (Appendix C.2).
Q5 / W5: Stability and returns have boundaries, downstream performance cannot be fully guaranteed
We have added a cell-level analysis in §6.1 and Appendix D.1. Across ten unseeded run pairs per method, many-to-many cluster overlaps accounted for 74.2% of changed assignments for umap-learn and 83.5% for ibUMAP. The cells in the largest such component recurred across pairs (median Jaccard overlap 0.94 and 0.93, respectively). The two methods' core cell sets also overlapped (Jaccard 0.80). Within each method, the clusters involved had about a quarter to a third of the median HDBSCAN persistence of one-to-one matches. Density-preserving optimization is one hypothesis we discuss. For downstream consistency, we would reuse a fixed-seed embedding or check the partition against consensus across reruns.
W7: Performance Disadvantages of Small Datasets
We agree that the fixed overhead of FFT and grid operations can make ibUMAP slower than umap-learn on very small datasets. On the smallest datasets in our unseeded CPU tests, both methods completed layout optimization in a fraction of a second, so layout time is unlikely to be a practical bottleneck for a single fit.