- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CAT-PT to improve exchanges in parallel tempering. The idea is to use the simplest transport that gives a high enough acceptance rate. For each pair of neighboring distributions, the method fits increasingly flexible velocity fields by minimizing a Stein residual, then checks their acceptance rates on held-out samples. The chosen maps are fixed during sampling, with a Metropolis correction.
The theory bounds exchange rejection using the residual and gives conditions under which round-trip performance depends on the remaining variation rather than the full dimension. Synthetic experiments show that simple structured maps can give better sampling efficiency per second than the neural transport baseline, even when they achieve fewer round trips per sweep.
My reproducibility check covered a small-scale reconstruction of the Gaussian experiment. It recovered the qualitative trend, but not the reported CAT-PT acceptance rates. The main timing comparisons were not tested.
Critical strengths
-
Theorem 1 links the Stein residual used in fitting to rejection across a finite temperature gap. This gives the fitting objective a clear justification.
-
I also like the idea of trying cheap transports first and adding complexity only when held-out acceptance is too low. This could avoid unnecessary computation on edges where a simple map is enough. The idea is useful, though the claimed speed gains need stronger evidence.
Critical weaknesses
-
The speed comparison is hard to assess. Appendix J leaves out key NF-APT training settings and the hardware and timing setup. Reported runtimes also exclude calibration and fitting. Since efficiency is a main claim, these gaps limit how much weight I can put on the reported gains.
-
The experiments give limited evidence for adaptive selection itself. Figure 2 compares raw exchange with just one structured candidate, and the Max-Structured comparison covers only the quartic translation case. A fixed cheap Stein transport on the main benchmarks would be a useful comparison. Without it, it is hard to tell how much adaptation adds beyond simply using a cheaper map.
Questions for authors
-
Could you provide the full NF-APT configuration, hardware, and timing procedure? What are the calibration and fitting times for both methods, and do the speed gains remain when these costs are included?
-
How does CAT-PT compare with fixed-level Stein transports using the same fitting procedure on the main benchmarks? Please also report which levels CAT-PT selects across edges. This would help show how much adaptive selection adds beyond using a cheap map.
Thank you for recognizing our algorithm design and theoretical results. We have conducted additional experiments to address your concerns.
W1 & Q1: Runtime comparison and experimental protocol.
Our original large-scale experiments were conducted on rented servers. For these supplementary experiments, all methods were run serially on the same local machine (AMD Ryzen 5 7640H, 6 cores/12 threads, 32 GB RAM), using the CPU backend and PyTorch with 6 threads. Each run used 16 PT ensembles, 1,000 warm-up sweeps, and 5,000 production sweeps. For NF-APT, we used a 20-layer RealNVP with hidden width 128 and two hidden layers per subnet. Training used batch size 256, Adam (learning rate 0.001), with 500–5,000 steps per round and validation-based early stopping. Complete runtime is measured from process startup to termination. The results are:
We also evaluate sampling accuracy against analytical reference values, using two common metrics: mean RMSE and second-moment RMSE.
The reported mean and second-moment observables follow each benchmark's construction: coordinate averages for Gaussian, the full mean vector and mean-square statistic for GMM, and averages over the four active coordinates for Many-Well. CAT-PT achieves moment estimation accuracy comparable to NF-APT, while substantially reducing runtime.
W2 & Q2: The benefits of adaptive selection
We also added fixed-level ablations: L1 is translation plus global scaling; L2 adds coordinate-wise scaling.
CAT-PT selected L1 on all 15 Gaussian edges; L2 on 23/27 and L3 on 4/27 GMM edges; and L1/L2/L3 on 15/27, 8/27, and 4/27 Many-Well edges, respectively. These results illustrate different regimes. For Gaussian, L1 is sufficient, and CAT-PT behaves almost identically to Fixed L1. For GMM, Fixed L1 performs substantially worse, whereas CAT-PT selects richer candidates and approaches Fixed L2. For Many-Well, CAT-PT uses different complexity levels across edges. Adaptive selection doesn't always outperform the best fixed level chosen with hindsight. Its benefit is that the appropriate complexity need not be known in advance and can vary across neighboring temperature pairs.
We will incorporate these results into the paper. We hope these additional results address your concerns and appreciate your consideration of this new evidence.