- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ASAP replaces most iterations of a frozen Sinkhorn attention layer with a calibrated predictor of its source potential. The predictor uses one-dimensional sliced transport potentials and is fitted by ridge regression or a one-sided KL objective. The other potential is recovered analytically, followed by a fixed number of normalization passes. Experiments measure agreement with the teacher, classification accuracy, calibration cost and inference time.
Critical strengths
Predicting one potential and recovering the other analytically gives a compact replacement for a trained layer. The paper is careful about finite-step conventions: it distinguishes a change of potential coordinates from restarting Sinkhorn with a different kernel, and notes that the extra correction can move the reconstruction away from the finite teacher even with exact prediction.
The evaluation includes more than task accuracy. The reconstruction and predictor ablations help explain where the error comes from, and the shared-kernel comparison in Table E.3 addresses an important implementation concern. The reported full-model result on Cats and Dogs—1.53× faster with a 0.65-point accuracy loss—is also useful evidence of a deployment benefit.
Critical weaknesses
The short-Sinkhorn comparison changes the final constraint as well as the iteration budget. Appendix F.1 retains query rows and active key columns. A row-ending attention matrix has total mass , whereas a column-ending matrix has mass . Table H.1 states that end in rows, while the teacher and ASAP end in columns. For , these constraints are incompatible; a column-ending matrix necessarily has mean absolute row error at least . This follows from the marginal mass identity underlying the transport constraints in Cuturi (2013), §2. The authors disclose this convention, but Table 1 does not control for it. Column-ending on the existing frozen cases, preserving the teacher's score kernel and initialization, would determine how much of the fidelity gap remains after matching the terminal constraint.
The learned predictor's increment over inexpensive reconstruction is unresolved. Table H.3 includes useful fitted-feature controls, but lacks an unlearned source potential under the same column–row–column reconstruction. Besides zero potential in cost coordinates, use the known shift . With the paper's quadratic cost, this makes the first reconstructed attention exactly column-softmax of the scaled scores, without slicing, sorting or fitting. Appendix I.5 combines a learned query-norm coefficient with row log-sum-exp features; that ablation does not isolate the fixed analytic shift. These controls would distinguish the learned initialization from the normalization work, a relevant distinction given the established role of Sinkhorn initialization(https://proceedings.mlr.press/v206/thornton23a.html) and the credited precedent for sliced-potential regression(https://arxiv.org/html/2604.15114v2).
The shared-kernel timings in Table E.3 and the full-model result in Table 4 are strengths. However, the strongest long-sequence timings and the trained-model fidelity evidence use different settings. Testing the matched controls on one frozen trained model with the optimized backend would establish whether the learned predictor provides a useful practical improvement.
Questions for authors
How do column-ending , zero-cost-potential reconstruction and the analytic query-norm-source control compare on the Table 1 cases? Preserve each method's stated coordinates and report attention error, output RMSE and latency, including every normalization pass and grouping results by active-token fraction.
Eq. (F.2) includes every query, but feature construction and regression exclude padded positions. How are padded-query source potentials assigned and centered? Please specify which query positions enter the reported errors.
On one trained model, does ASAP retain an accuracy–latency advantage over these controls under the same optimized implementation? Report results by teacher seed; the 144 batch–layer cases are repeated measurements from three teachers.
2 comments
Q3. Trained Models, Common Implementations and Teacher Seeds
The 144 batch-layer cases are repeated observations from three trained teachers and should not be treated as independent model runs. We now report the teacher-level results explicitly in Table R9, with full-test accuracy evaluated after each replacement propagates its own hidden states through the model. Table R3 supplies the common-MPS timing comparison on teacher 1, including the predictor and every normalization pass, so we measure deployment cost on the trained model itself.
Table R9. Teacher-level IMDb results. Each accuracy uses all 10,000 test examples. Output RMSE averages the 48 frozen batch-layer cases from each teacher.
| Teacher | S20 accuracy (%) | S4 accuracy (%) | S6 accuracy (%) | ASAP accuracy (%) | S6 output RMSE | ASAP output RMSE |
|---|---|---|---|---|---|---|
| 1 | 87.31 | 87.31 | 87.34 | 87.36 | 0.004952 | 0.003818 |
| 2 | 87.63 | 87.56 | 87.56 | 87.57 | 0.002057 | 0.002009 |
| 3 | 87.09 | 87.03 | 87.14 | 87.16 | 0.002398 | 0.002140 |
We find lower output error than S6 for each teacher and 99.7733% mean prediction agreement with S20, while reducing teacher-1 full-model latency by 2.32 times in Table R3. The revised main table includes S4, S6 and both fixed sources with their lower latency and larger layer-output errors, so the comparison evaluates the additional fidelity from prediction as well as the saving over the teacher.
We have added controls for finite normalization and source initialization on the original cases, made the padded-position treatment explicit, and reported the trained teachers individually.
Q1. Marginal Mass and the Learned Source
The original S3/S5 comparison did not control for the effect of unequal total mass. We now use column-ending S2/S4/S6 from the teacher's score kernel and initialization, so the final target and total mass agree with the teacher and ASAP. We also evaluate the analytic norm source alongside zero cost potential, applying the same KQK reconstruction to both. Zero cost source corresponds to score source −ρ, while cost source ρ corresponds to zero score source. The latter therefore tests the inexpensive column-softmax initialization, without slicing or fitting.
On the original cases, we measure output RMSE 0.002656 for ASAP, against 0.007806 for zero cost source, 0.005793 for the analytic source and 0.003135 for S6. The fixed-source controls show a fitted-source benefit beyond the shared transforms, while S6 remains slightly better on aggregate attention error. We grouped inputs by active-token fraction and evaluated every group on the same sequence IDs across teachers. The output-fidelity improvement over the analytic source persists throughout, while the comparison with S6's attention error depends on the group.
Table R8. IMDb by active fraction. Entries are output RMSE followed by attention error, using all retained source rows. Each teacher sees the same 256 sequences. Within a group, squared output errors are pooled over its examples and six layers before taking the root, then averaged over teachers. Attention error averages example-head relative ratios. These are group estimators, not the case-averaged RMSE of Table R1.
In the fully occupied group, we find lower output and attention error for ASAP than for every listed control, so this result does not depend on unequal support sizes. We also repeated calibration and full-model evaluation with compatible row targets of one and active-column targets of N/m in Table R5, where the learned source again improves output fidelity over both fixed initializers. Table R3 reports the full cost of the original controls, including their initialization and all normalization passes.
Q2. Padded Positions and Centering
The active-only sentence did not describe the IMDb implementation, and we have corrected Appendix F.1 to state the support used by each experiment. For IMDb, we construct sliced features, fit the regression and center the source over all N retained positions, with the key mask applied during attention reconstruction. We compute a feature-based prediction at each padded source position just as at every other retained position. The active-only construction belongs to the N128 text experiments, where padding is removed from the transport support and excluded from pooling.
Writing u(T) for the stored score-coordinate source, ρ for the quadratic row shift and ΠN for centering over all retained source positions, we fit y=ΠN(u(T)+ρ) and reconstruct from u^=ΠN(ΠN(Xω)−ρ). This applies one common gauge shift to all source positions. Centering only active positions and then inserting zero padded potentials would alter their relative column weights, so we do not use that construction. The original fidelity metric includes every source row, and the classifier retains its original max pooling. To address which positions contribute to the error, we also evaluated valid source rows alone and obtained output RMSE 0.003533 for ASAP, 0.004177 for S6 and 0.007730 for the analytic source. The fitted-source benefit remains on the positions containing text.