- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2(+1)
Read review
Paper summary
This paper characterizes row amplitude as a downstream optimization channel through RMT, an exact gradient covariance, and the boundary of scalar-rescaling equivalence. To measure what that correction does to the update, Row-Amplitude Audit (RAA) applies the actual optimizer at the same model and optimizer state. External fixed-intervention gains establish task value.
Critical strengths
RMT provides a clean intervention for separating sample weighting from assignment changes. The mass–gradient covariance identity explains when this intervention changes gradient direction rather than merely its magnitude.
The optimizer audit accounts for incoming optimizer state and other loss terms, making it more informative than gradient-norm comparisons alone. Its selected coefficients are tested prospectively on subsequent training trajectories.
Fixed interventions produce meaningful ACC and ARI gains in both PROTOCOL and P2OT across paired seeds. The paper also reports negative results and competing metrics, clearly separating optimizer control from task improvement.
Critical weaknesses
The practical benefit of RAA remains unclear: the positive task results use separately chosen fixed coefficients, while RAA provides no observed accuracy gain in JUMBOT. The chosen update-deviation band therefore lacks a demonstrated connection to useful learning outcomes.
The evidence that gradient-direction changes explain the downstream gains is incomplete. Two P2OT seeds fail the numerical criterion for the update-scale control, leaving the full-training comparison beyond scalar rescaling unresolved.
The P2OT improvements come with a substantial metric tradeoff: class-balanced accuracy drops by 3.93 points and purity by 2.73 points in Table 3. This limits the case for amplitude equalization when performance across rare and frequent classes matters.
Questions for authors
How sensitive are RAA’s selections and future coverage to the deviation band, candidate grid, and prefix length? What guidance can users follow when choosing these settings for a new task?
Can the P2OT update-scale comparison be repeated with all seeds passing the numerical realization criterion? This would clarify whether the task gains persist after controlling for update magnitude.
Can an intermediate tempering coefficient retain the ACC and ARI gains while reducing the losses in class-balanced accuracy and purity? Please report whether this tradeoff changes with the imbalance ratio.
3 comments
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts in addressing my concerns. While some of my concerns have been partially addressed, I am not sufficiently familiar with this specific research area to confidently assess the broader significance and insights of this work for the community. From my perspective, the paper is borderline. I have slightly improved my assessment 'Presentation and Soundness score', and since a score of 2.5 is not available, I have tentatively raised my rating to 3.
Good luck!
Thank you for your thoughtful reassessment and for taking the time to consider our response and additional experiments. We sincerely appreciate your constructive feedback and updated evaluation.
Response to Reviewer b6Lt: Optimizer Controllability, Numerical Validation, and Task Tradeoffs
Response to Reviewer b6Lt
We appreciate the reviewer's recognition of the amplitude intervention and optimizer-aware analysis. We add prefix-length ablations and clarify practical value, numerical control, and metric tradeoffs.
1. What RAA adds in practice.
Transport masses can encode confidence or curriculum, but mass contrast alone does not determine the optimizer update. Under the stopped-target linear-loss interface, the immediate correction is gτ = g₀ + τΔ, with Δ = c_B m Cov_A(s,h), and Adam further transforms that gradient. RAA therefore evaluates the actual same-state proposal deviation Dτ = ‖uτ − u₁‖₂ / ‖u₁‖₂, rather than inferring its size from τ or gradient norms. RAA selects τ from a training prefix, freezes it, and validates the budget on the resulting future trajectory. Its objective is controlled interventions, not direct accuracy optimization.
2. New prefix-length ablation and fixed-policy context.
On P2OT seeds 86–95, with future success defined as at least 18/20 probes in the guarded deviation band [.021, .099], the observed results are:
Seed-level success rises from 20% (RAA-1) to 100% (RAA-10): +80 pp (two-sided exact McNemar p = .0078). RAA-3/5 show intermediate success rates, directly addressing single- versus multi-probe selection. RAA-10's reported audit overhead is 0.20% of training wall-clock time. Fixed-.875 is the strongest tested fixed policy (7/10); its difference from RAA-10 is descriptive rather than statistically conclusive (paired p = .25). These are observed policy comparisons, not coefficient-only causal effects.
On additional JUMBOT seeds 501–505, both RAA-10 and Fixed-0 pass 5/5, whereas Fixed-.875 passes 0/5. Thus the P2OT coefficient does not transfer; RAA does not outperform Fixed-0 on JUMBOT.
3. Band, grid, and prefix sensitivity.
The nominal 2%–10% band excludes negligible changes while limiting deviation from raw training; [.021, .099] provides a guard margin. It is an operational update budget, not an accuracy optimum. The candidate grid is {0, .25, .5, .75, .875, 1}. For RAA-10, eligibility requires at least 9/10 prefix probes in band and τ ≤ .875; the smallest eligible τ is frozen, with abstention if none qualifies. Since CV_A(s(τ)) = τ CV_A(s), this guarantees at least a 12.5% reduction in active-row mass contrast. The ablation tests prefix length, not band/grid sensitivity; we will specify each RAA-k eligibility rule in the revision.
4. Full-training update-scale control.
The P2OT scale-matched control tests a separate causal question: whether downstream gains can be reproduced by matching update magnitude alone. The predeclared acceptance rule requires at least 99% of 1,050 updates per seed to agree with the intended update within 1% relative vector error. Three seeds pass; seeds 76 and 80 achieve 1030/1050 and 1029/1050, respectively. All 41 exceedances occur in the last epoch. At the fixed endpoint, matched RMT exceeds the implemented scale control by +5.07 ± 4.73 ARI points and +4.49 ± 3.59 ACC points (population SD). The two failures limit the long-run scale-isolation conclusion, but not the separate local gradient identities or RAA control results.
5. Task value, class tradeoffs, and intermediate coefficients.
In the separate P2OT/CIFAR100-LT task study (imbalance ratio .01), fixed matching improves ACC by 5.60 pp and ARI by 6.53 pp across five paired seeds, while mACC decreases by 3.93 pp and Purity by 2.73 pp. Head classes account for 81.35% of samples; their recall rises by 5.90 pp, while middle- and tail-class recall changes are −10.73 and −7.45 pp. Thus the sample-level gains are meaningful but do not imply uniform class-balanced improvement. Intermediate sᵢ(τ) = s̄ + τ(sᵢ − s̄) allows partial matching; its Pareto frontier and dependence on imbalance ratio require task-level evaluation and cannot be inferred from proposal coverage.
We will foreground this distinction between amplitude mechanism, prospective optimizer control, and task utility, and incorporate the new ablation and per-seed numerical results into the revision.