acceptodds
Under review as a conference paper at ICLR 2027

Row Amplitude as an Optimization Mechanism in Transport-Based Learning

Abstract

A transport target determines both a sample's assignment and its weight in learning. We study the latter—its row amplitude—as a downstream optimization variable, separating its effect from changes in conditional assignment. For losses linear in a stopped target, Row-Mass Tempering (RMT) isolates amplitude while preserving assignments, the set of nonzero rows, and total mass. An exact mass–force covariance explains the resulting gradient correction and when scalar rescaling cannot reproduce it. This motivates Row-Amplitude Audit (RAA), which measures the actual optimizer effect and selects a coefficient from initial training observations for prospective validation. Frozen rules meet the declared update-deviation criterion in all three POT seeds and all five JUMBOT seeds. Separate fixed-intervention studies establish task value: development-selected RMT improves PROTOCOL/Hdigit clustering accuracy by 5.09 percentage points and adjusted Rand index by 6.56 points over 10 paired seeds; equalizing active row amplitudes improves these metrics by 5.60 and 6.53 points in POT/CIFAR100-LT over five paired seeds, with lower class-balanced accuracy and purity. JUMBOT shows the boundary: successful optimizer control yields no observed accuracy gain over the fixed reference. The contribution is a systematic account of an amplitude channel that can be isolated, explained, measured, and controlled, with optimizer controllability and downstream utility evaluated as distinct objectives.

Primary area:
optimization
Keywords:
optimal transport, row amplitude, optimization dynamics, sample reweighting, clustering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_b6LtOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

This paper characterizes row amplitude as a downstream optimization channel through RMT, an exact gradient covariance, and the boundary of scalar-rescaling equivalence. To measure what that correction does to the update, Row-Amplitude Audit (RAA) applies the actual optimizer at the same model and optimizer state. External fixed-intervention gains establish task value.

Critical strengths

RMT provides a clean intervention for separating sample weighting from assignment changes. The mass–gradient covariance identity explains when this intervention changes gradient direction rather than merely its magnitude.

The optimizer audit accounts for incoming optimizer state and other loss terms, making it more informative than gradient-norm comparisons alone. Its selected coefficients are tested prospectively on subsequent training trajectories.

Fixed interventions produce meaningful ACC and ARI gains in both PROTOCOL and P2OT across paired seeds. The paper also reports negative results and competing metrics, clearly separating optimizer control from task improvement.

Critical weaknesses

The practical benefit of RAA remains unclear: the positive task results use separately chosen fixed coefficients, while RAA provides no observed accuracy gain in JUMBOT. The chosen update-deviation band therefore lacks a demonstrated connection to useful learning outcomes.

The evidence that gradient-direction changes explain the downstream gains is incomplete. Two P2OT seeds fail the numerical criterion for the update-scale control, leaving the full-training comparison beyond scalar rescaling unresolved.

The P2OT improvements come with a substantial metric tradeoff: class-balanced accuracy drops by 3.93 points and purity by 2.73 points in Table 3. This limits the case for amplitude equalization when performance across rare and frequent classes matters.

Questions for authors

How sensitive are RAA’s selections and future coverage to the deviation band, candidate grid, and prefix length? What guidance can users follow when choosing these settings for a new task?

Can the P2OT update-scale comparison be repeated with all seeds passing the numerical realization criterion? This would clarify whether the task gains persist after controlling for update magnitude.

Can an intermediate tempering coefficient retain the ACC and ARI gains while reducing the losses in class-balanced accuracy and purity? Please report whether this tradeoff changes with the imbalance ratio.

3 comments
AuthorsOpenReview ↗

Response to Reviewer b6Lt: Optimizer Controllability, Numerical Validation, and Task Tradeoffs

Response to Reviewer b6Lt

We appreciate the reviewer's recognition of the amplitude intervention and optimizer-aware analysis. We add prefix-length ablations and clarify practical value, numerical control, and metric tradeoffs.

1. What RAA adds in practice.

Transport masses can encode confidence or curriculum, but mass contrast alone does not determine the optimizer update. Under the stopped-target linear-loss interface, the immediate correction is gτ = g₀ + τΔ, with Δ = c_B m Cov_A(s,h), and Adam further transforms that gradient. RAA therefore evaluates the actual same-state proposal deviation Dτ = ‖uτ − u₁‖₂ / ‖u₁‖₂, rather than inferring its size from τ or gradient norms. RAA selects τ from a training prefix, freezes it, and validates the budget on the resulting future trajectory. Its objective is controlled interventions, not direct accuracy optimization.

2. New prefix-length ablation and fixed-policy context.

On P2OT seeds 86–95, with future success defined as at least 18/20 probes in the guarded deviation band [.021, .099], the observed results are:

PolicyPassing seedsIn-band future probes
RAA-1010/10196/200
RAA-59/10187/200
RAA-37/10178/200
RAA-12/10133/180 (one abstention)
Fixed-00/10127/200
Fixed-.53/10168/200
Fixed-.8757/10179/200

Seed-level success rises from 20% (RAA-1) to 100% (RAA-10): +80 pp (two-sided exact McNemar p = .0078). RAA-3/5 show intermediate success rates, directly addressing single- versus multi-probe selection. RAA-10's reported audit overhead is 0.20% of training wall-clock time. Fixed-.875 is the strongest tested fixed policy (7/10); its difference from RAA-10 is descriptive rather than statistically conclusive (paired p = .25). These are observed policy comparisons, not coefficient-only causal effects.

On additional JUMBOT seeds 501–505, both RAA-10 and Fixed-0 pass 5/5, whereas Fixed-.875 passes 0/5. Thus the P2OT coefficient does not transfer; RAA does not outperform Fixed-0 on JUMBOT.

3. Band, grid, and prefix sensitivity.

The nominal 2%–10% band excludes negligible changes while limiting deviation from raw training; [.021, .099] provides a guard margin. It is an operational update budget, not an accuracy optimum. The candidate grid is {0, .25, .5, .75, .875, 1}. For RAA-10, eligibility requires at least 9/10 prefix probes in band and τ ≤ .875; the smallest eligible τ is frozen, with abstention if none qualifies. Since CV_A(s(τ)) = τ CV_A(s), this guarantees at least a 12.5% reduction in active-row mass contrast. The ablation tests prefix length, not band/grid sensitivity; we will specify each RAA-k eligibility rule in the revision.

4. Full-training update-scale control.

The P2OT scale-matched control tests a separate causal question: whether downstream gains can be reproduced by matching update magnitude alone. The predeclared acceptance rule requires at least 99% of 1,050 updates per seed to agree with the intended update within 1% relative vector error. Three seeds pass; seeds 76 and 80 achieve 1030/1050 and 1029/1050, respectively. All 41 exceedances occur in the last epoch. At the fixed endpoint, matched RMT exceeds the implemented scale control by +5.07 ± 4.73 ARI points and +4.49 ± 3.59 ACC points (population SD). The two failures limit the long-run scale-isolation conclusion, but not the separate local gradient identities or RAA control results.

5. Task value, class tradeoffs, and intermediate coefficients.

In the separate P2OT/CIFAR100-LT task study (imbalance ratio .01), fixed matching improves ACC by 5.60 pp and ARI by 6.53 pp across five paired seeds, while mACC decreases by 3.93 pp and Purity by 2.73 pp. Head classes account for 81.35% of samples; their recall rises by 5.90 pp, while middle- and tail-class recall changes are −10.73 and −7.45 pp. Thus the sample-level gains are meaningful but do not imply uniform class-balanced improvement. Intermediate sᵢ(τ) = s̄ + τ(sᵢ − s̄) allows partial matching; its Pareto frontier and dependence on imbalance ratio require task-level evaluation and cannot be inferred from proposal coverage.

We will foreground this distinction between amplitude mechanism, prospective optimizer control, and task utility, and incorporate the new ablation and per-seed numerical results into the revision.

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts in addressing my concerns. While some of my concerns have been partially addressed, I am not sufficiently familiar with this specific research area to confidently assess the broader significance and insights of this work for the community. From my perspective, the paper is borderline. I have slightly improved my assessment 'Presentation and Soundness score', and since a score of 2.5 is not available, I have tentatively raised my rating to 3.

Good luck!

Thank you for your thoughtful reassessment and for taking the time to consider our response and additional experiments. We sincerely appreciate your constructive feedback and updated evaluation.

Reviewer_KQnsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how transport row masses affect optimization independently of conditional assignments. Row-Mass Tempering (RMT) interpolates active row masses toward their mean while preserving conditional assignments, active support, and total mass. For losses linear in stopped transport targets, the paper expresses the resulting gradient correction as a mass–gradient covariance and characterizes when scalar rescaling cannot reproduce it. Row-Amplitude Audit (RAA) measures the corresponding optimizer proposals, selects an intervention from initial training observations, and tests its update-deviation criterion during continuation.

Validation combines a controlled clustering testbed with PROTOCOL, P²OT, and JUMBOT. Fixed interventions improve ACC/ARI in PROTOCOL/Hdigit and P²OT/CIFAR100-LT, although P²OT loses class-average accuracy and purity. Separate prospective studies achieve the specified optimizer-control criterion in P²OT and JUMBOT; JUMBOT shows no corresponding accuracy gain.

Critical strengths
  1. I liked that RMT allows the authors to study the effect of sample weights while keeping the conditional assignments fixed at the same model state. The covariance analysis explains how changing these weights affects the gradient, including its direction. The optimizer comparisons also use the same model parameters and Adam state, making the local comparison controlled. Appendix A.4 further clarifies when these weight changes can also be obtained by changing the transport solver’s source prior.

  2. The fixed interventions show useful empirical improvements. On PROTOCOL/Hdigit, the development-selected coefficient improves mean ACC by 5.09 percentage points and ARI by 6.56 points over ten paired seeds. On P²OT/CIFAR100-LT, equalizing the row masses improves ACC by 5.60 points and ARI by 6.53 points, with improvements in both metrics across all five paired seeds. I also appreciated that the authors report the limitations: class-average accuracy and purity decrease in P²OT, and JUMBOT shows no observed accuracy gain despite successful optimizer control. These results help clarify both the potential benefit and the limits of changing the row masses.

Critical weaknesses
  1. My main concern is that the additional contribution beyond sample reweighting remains limited. The paper acknowledges that RMT is sample reweighting, and the affine gradient path, covariance identity, and projection test appear to be straightforward consequences of this formulation. P²OT already uses transport row sums as sample weights. RAA measures the optimizer effect and tests whether a selected coefficient keeps future update deviations within a chosen range. However, the evidence comes from two previously studied settings: P²OT selects a shared τ = .875 across the confirmation seeds, while all five JUMBOT seeds select τ = 0, giving the same intervention as simply fixing τ = 0. These results demonstrate control in the tested settings, but I am not yet convinced that they establish a substantial benefit over simpler coefficient selection or provide enough guidance for choosing interventions in other settings.

I also found the writing harder to follow than necessary. The distinction between optimizer control and task improvement is repeated throughout, often without adding new information. The heavy terminology and repeated qualifications make it difficult to identify the main contribution. For example, phrases such as “dataset-level prospectively frozen non-inferiority” could be explained more directly. Reducing the repetition and simplifying the language would make the paper much clearer.

  1. The empirical benefits also need stronger support. PROTOCOL/Hdigit is one of the main positive results, but Table 2 reports no standard deviations or confidence intervals. Appendix B.3 also leaves unclear how the development sweep relates to the ten reported seeds and provides limited implementation and training details. This makes the robustness of the improvement difficult to assess. In P²OT, the ACC and ARI gains come with decreases of 3.93 percentage points in class-average accuracy and 2.73 points in purity. Appendix C.2 shows that the gains mainly benefit the more frequent classes, which is an important tradeoff for imbalanced clustering. Finally, two of the five P²OT scale-control seeds fail the declared numerical criterion, with all exceedances occurring in the final epoch (Table 9). This leaves unresolved whether the full-training gains go beyond changes in update magnitude. The authors acknowledge these limitations, but they still limit the conclusions about practical benefit and the explanation of the final training gains.
Questions for authors
  1. How was τ = .5 selected for PROTOCOL? Which coefficients, selection metric, and development seeds were used, and were labels used during selection? Were the ten reported paired seeds independent of this selection? Please also report the per-seed differences with standard deviations or confidence intervals and provide the implementation and training details needed to reproduce Table 2.

  2. Can you provide a full-training scale-control comparison in which all seeds meet the numerical criterion, evaluated at the same fixed endpoint used in Table 8? Does the advantage of equalizing row masses remain? This would help establish whether the final gains go beyond changes in update magnitude.

  3. How were the update-deviation range and the requirement of nine passing probes out of ten chosen? What evidence shows that using several initial probes improves future control compared with selecting a coefficient from a single probe or fixing it using development data? The results show that different coefficients are needed in the two settings, but I would like to understand whether using several initial probes gives more reliable future control than these simpler selection rules.

4 comments
AuthorsOpenReview ↗

Response to Reviewer KQns: Novelty, Experimental Robustness, and Optimizer Control

Response to Reviewer KQns

We thank the reviewer for highlighting originality, experimental robustness, and the value of RAA. We now provide a direct multi-probe/fixed-policy ablation.

1. Novelty beyond sample reweighting.

RMT is algebraically sample reweighting. Our contribution isolates a transport-specific intervention preserving conditional assignments, active support, and total mass, then characterizes solver feasibility and learning effects. For a stopped target entering the loss linearly, the exact correction is Δ = c_B m Cov_A(s,h), with gτ = g₀ + τΔ. A nonzero transverse component Δ⊥ cannot arise from scalar gradient scaling.

More importantly, Proposition A.1 states that, for the specified positive entropic KL-UOT problem with fixed cost/other inputs and a freely adjustable positive source prior, a fixed-conditional amplitude change is solver-realizable if and only if Pᵀs′ = Pᵀs; the amplitude freedom has dimension m − rank(P). Corollary A.2 identifies the gradient-visible part: 𝒢 = c_B H ker(Pᵀ), with dim(𝒢) = rank([Pᵀ; H]) − rank(P), and complete invisibility exactly when H = LPᵀ for some L. These results identify which mass interventions the solver can realize and which can change learning gradients. RMT as a downstream intervention need not itself remain optimal for the original solver. RAA then evaluates the actual candidate optimizer proposals at the same incoming optimizer state, rather than equating gradient change with parameter-update change.

2. PROTOCOL coefficient selection and paired robustness.

Development used seeds 201–210 and the grid τ = {0, .25, .5, .75, 1} (50 runs); ARI was the primary selection metric, with ACC checked for consistency. Mean development ARIs (%) were 85.94, 88.10, 89.02, 88.36, 82.71, respectively. τ = .5 was frozen before evaluation on disjoint seeds 1–10; this is label-informed development selection, not evaluation-seed tuning. Both ACC and ARI improve in 10/10 paired report seeds. The ACC gain is +5.090 pp (sample SD 2.102; 95% paired t-CI [3.586, 6.594]); ARI gains +6.560 pp (SD 1.825; CI [5.254, 7.866]). Two-sided paired t-test p-values are 3.13×10⁻⁵ and 1.22×10⁻⁶. We will add these selection, training, and dispersion details to Appendix B.3/Table 2.

3. New direct RAA ablation: is multi-probe selection useful?

We compare RAA-1/3/5/10 with fixed τ = 0, .5, .875 on P2OT seeds 86–95, using the declared future criterion of at least 18/20 in-band proposals per seed (guarded band [.021, .099]). Seed-level pass counts are:

RAA-10: 10/10 (196/200 future probes); RAA-5: 9/10 (187/200); RAA-3: 7/10 (178/200); RAA-1: 2/10 (133/180 valid probes; one abstention).

Fixed-0: 0/10 (127/200); Fixed-.5: 3/10 (168/200); Fixed-.875: 7/10 (179/200).

Thus, the observed success rate increases with prefix length; compared with one probe, ten probes improve seed-level success by 80 percentage points (two-sided exact McNemar p = .0078). RAA-10 also exceeds the strongest tested fixed coefficient by 30 points (10/10 vs. 7/10); this smaller paired comparison is not statistically significant (p = .25). The reported proposal-audit overhead for RAA-10 is 0.20% of training wall-clock time. These are comparisons at the defined optimizer-control endpoint, not task-accuracy comparisons or a claim of universal policy dominance.

On additional JUMBOT seeds 501–505, RAA-10 and fixed-0 both pass 5/5, while fixed-.875 passes 0/5. RAA does not outperform fixed-0 there. These results support multi-probe selection in the tested P2OT setting and caution against transferring a fixed τ across settings.

4. Update-scale control and metric tradeoffs.

For the separate P2OT full-training update-scale comparison, three of five seeds satisfy the prespecified ≥99%-within-1% realization criterion; seeds 76 and 80 reach 1030/1050 and 1029/1050, respectively, and fail. All 41 excursions occur in the last epoch. The observed matched-minus-control endpoint difference is +5.07 ARI pp, but this comparison does not conclusively isolate a long-run effect beyond update magnitude. The exact local results and separate RAA coverage remain intact.

The P2OT fixed-intervention task study likewise reports a metric tradeoff, not universal improvement: sample-level ACC +5.60 pp, ARI +6.53 pp, alongside mACC −3.93 pp and Purity −2.73 pp. Head-class prevalence helps explain why sample-level and class-balanced assessments diverge.

We will foreground the structural results in Section 3, add the ablation and paired statistics, and streamline the exposition.

Reviewer_KQnsOpenReview ↗

Response

Thanks for the response and the additional experiments. The new statistics and ablation address some of my concerns. However, I still find the contribution beyond sample reweighting limited. The full-training scale-control issue remains unresolved, and the benefit over a well-chosen fixed coefficient is still not clear. Given this, I will keep my score.

AuthorsOpenReview ↗

Follow-up to Reviewer KQns: Structural Contribution and Development-Selected Fixed-Policy Control

Follow-up to Reviewer KQns: Structural Novelty and Development-Selected Fixed-Policy Control

Thank you for your follow-up and for acknowledging the added evidence. We address the three remaining issues, including a new test against a development-selected fixed policy.

1. Contribution beyond sample reweighting

RMT has a sample-reweighting form at the stopped-loss interface; that algebra is not our novelty claim. We characterize which amplitude changes a transport solver admits, which are learning-visible, and how their optimizer effect can be prospectively controlled.

Under the positive entropic KL-UOT model with fixed costs and target-side inputs and an unrestricted positive source prior, Proposition A.1 gives an if-and-only-if condition for a fixed-conditional plan diag(s')P to be solver-realizable: Pᵀs'=Pᵀs. It constructs the unique realizing source prior, and the feasible family has dimension m−rank(P).

Corollary A.2 characterizes its learning-visible component: gradient changes span c_B H ker(Pᵀ), with dimension rank([Pᵀ;H])−rank(P). Consequently, the solver-realizable but gradient-invisible directions have dimension m−rank([Pᵀ;H]); all realizable changes are invisible exactly when H=LPᵀ. Thus we characterize the exact gap between solver-feasible and learning-effective amplitude freedom, beyond the generic reweighting identity.

For a concrete example, let P have rows (.8,.2), (.5,.5), (.2,.8). Then v=(1,−2,1) satisfies Pᵀv=0. Sufficiently small amplitude changes along v preserve the target marginal and are realizable through the source prior, but alter the learning gradient only when Hv≠0. The result is specific to this KL-UOT model; downstream RMT need not itself be solver-optimal.

RAA separately measures the actual same-state optimizer proposal, selects τ from prefix probes, freezes it, and tests future effects.

2. New evidence against a well-chosen fixed policy

Fixed-Dev uses ten disjoint P2OT development seeds (201–210) to choose from all five eligible coefficients τ∈{0,.25,.5,.75,.875}, maximizing mean future in-band coverage. The best fixed policy, τ=.875, achieved 14.3/20 development probes in band on average (6/10 seeds passing). Thus the baseline is development-optimized, not arbitrary.

We then compared this frozen policy with the frozen RAA-10 prefix-selection rule on 20 separate P2OT evaluation seeds (96–115), using band [.021,.099] and criterion ≥18/20 future probes in band:

Held-out P2OT evaluationRAA-10Fixed-Dev (τ=.875)
Successful seeds20/206/20
Future probes in band400/400234/400

The seed-level gain is +70 percentage points: 14 RAA-only versus 0 Fixed-Dev-only successes (two-sided exact McNemar p=1.22×10⁻⁴; seeds, not probes, are the units).

RAA-10 selected τ=.25/.5/.75/.875 on 1/5/8/6 evaluation seeds, respectively. On the six seeds in which the policies selected the same τ=.875, their recorded future-coverage counts agreed. The difference is consistent with prefix-conditioned selection against a strong fixed baseline, for the optimizer-deviation control endpoint—not task accuracy.

3. Full-training scale control and the scope of the claim

The 50-epoch P2OT scale-matched task comparison retains the disclosed numerical limitation: two of five seeds do not meet the prespecified requirement that ≥99% of updates be within 1% relative vector error. Accordingly, we do not use that comparison to claim that the full-training task gain has been causally isolated from update magnitude. This question remains open within that control.

This does not resolve long-horizon causal attribution, but it is distinct from the exact structural results and the prospective RAA-versus-Fixed-Dev control endpoint. The revision will clarify these distinct claims.

Summary. The contribution is not the generic effect of reweighting on gradients: it is an exact characterization of solver-realizable versus learning-visible amplitude freedom, coupled with a prospective optimizer-level selection and validation procedure. The new held-out comparison shows 20/20 versus 6/20 successful seeds against a genuinely development-selected fixed coefficient (p=1.22×10⁻⁴). We believe this directly addresses the two contribution-related concerns while preserving the disclosed scale-control limitation. Thank you for reconsidering the evidence.

AuthorsOpenReview ↗

Clarifying the Main Contributions: Beyond Sample Reweighting

Thank you for your continued consideration. We also appreciate your concern that the terminology and numerous technical qualifications may obscure the paper's main contributions. We recognize that the presentation can make the central message easier to identify.

We would therefore like to explain the contribution through one central question and three connected results.

Central question: What role does transport row-mass heterogeneity play in learning, and can its effect on optimization be understood and controlled?

An OT plan contains two different kinds of information: where each sample is assigned and how much weight that sample receives. Our work isolates the second factor and studies its consequences, from the transport solver to the learning gradient and the actual optimizer update.

1. Isolating a previously entangled optimization mechanism (Sec. 3).

RMT provides a controlled way to modify row-mass heterogeneity while keeping conditional assignments, active support, and total mass fixed.

The key observation is not simply that changing sample weights changes gradients. Our analysis establishes when such changes matter and how they affect optimization.

Specifically, the gradient correction depends on the covariance between row masses and conditional sample gradients. Thus, unequal masses alone do not imply a meaningful correction. Moreover, this correction can change the gradient's direction, not merely its magnitude.

These predictions are examined through controlled regularization experiments and P2OT gradient diagnostics.

2. Characterizing which changes are possible and which are visible to learning (Proposition A.1; Corollary A.2).

A more fundamental question arises: if sample assignments remain fixed, which changes in row masses are compatible with the OT solver itself?

Proposition A.1 provides an exact necessary-and-sufficient condition for such changes in the stated entropic KL-UOT model, together with the unique source prior that realizes them.

Corollary A.2 then answers a different question: which of these solver-realizable changes actually affect the learning gradient?

The two results reveal an important distinction: an OT solver can permit changes in row masses that are completely invisible to downstream learning. The paper characterizes both the available degrees of freedom and the subset that learning can detect.

This is a structural result about transport and learning, not a restatement of the familiar sample-reweighting identity.

3. Turning the mechanism into prospective optimizer control (Sec. 4; Sec. 5.2).

Understanding a gradient correction is not the same as knowing what an optimizer will do with it. In particular, Adam's update also depends on its accumulated state.

RAA therefore measures the actual optimizer proposal at a shared state, uses training-prefix observations to choose an intervention coefficient, and tests whether the intended update effect persists during subsequent training.

This produces a concrete control capability rather than only a retrospective gradient analysis.

Our new held-out experiment, described in the preceding response, directly tests this value against a development-selected fixed coefficient: RAA succeeds on 20/20 evaluation seeds, compared with 6/20 for Fixed-Dev. The comparison supports the practical benefit of adapting the intervention to the observed optimizer response.

An important additional result: What can local measurements tell us about later task performance? (Appendix E)

The paper also investigates the limits of such measurements. Proposition E.2 shows that even complete local observations need not determine whether an intervention will improve later task utility. Proposition E.3 provides a complementary result identifying additional structure under which finite continuation can be recovered.

This explains why we evaluate optimizer control and downstream task performance separately. The external PROTOCOL and P2OT experiments demonstrate improvements in clustering ACC and ARI from fixed amplitude interventions, while JUMBOT illustrates that successful update control does not automatically imply an accuracy gain.

Summary.

The paper's contribution is not a new formula for sample reweighting. It establishes a connected account of (i) how transport row masses affect optimization, (ii) which amplitude changes an OT solver permits and a learner can detect, and (iii) how these effects can be measured and controlled through the actual optimizer.

The additional theoretical and empirical results clarify when such control is informative and how it relates to downstream task utility.

We will make this three-part structure more prominent in the manuscript, with the technical assumptions stated alongside the corresponding results rather than obscuring the main narrative.

We hope this clearer presentation helps convey the scope and significance of the contribution.

Reviewer_XKYSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how the row amplitudes of transport targets affect gradients and optimizer updates in transport-based learning, while keeping conditional assignments fixed. The authors introduce RMT, which interpolates row masses toward their mean while preserving conditional assignments, active rows, and total mass, and derive an exact relationship between the resulting gradient correction and the covariance of row masses with conditional sample gradients. They further propose RAA to measure the actual optimizer effect, select an intervention coefficient from initial training observations, and validate it prospectively. RAA meets the predeclared update-deviation criterion in P2OT and JUMBOT. In separate fixed-intervention experiments, RMT improves ACC and ARI in PROTOCOL and P2OT, but lowers class-balanced accuracy and purity in P2OT and yields no accuracy gain in JUMBOT.

Critical strengths
  1. The paper provides a framework for analyzing the optimization effect of row amplitude, make a good separation between assignments and sample weights, and it successfully connects changes in sample weights to gradient changes and actual optimizer updates.
  2. The experiments clearly distinguish optimizer controllability from downstream task utility, and the authors transparently report both positive results and limitations.
Critical weaknesses
  1. RAA is well motivated as an optimizer audit, but the practical significance of its prescribed control objective is less proved by experiments. Although RAA successfully controls optimizer update deviation, it is unclear why the prescribed deviation band is desirable and what practical purpose it serves. The reported task gains from fixed RMT interventions do not directly establish the usefulness of this control objective.
  2. The paper mainly formalizes the intuitive effect that changing sample weights changes the gradient and the resulting update. The analysis is rigorous, but it offers limited new understanding of when or why this effect matters for learning.
Questions for authors
  1. RAA demonstrates that the optimizer effect of amplitude intervention can be controlled within a declared band. In what practical scenario would a practitioner want this control, and how should the band be chosen in that scenario?
  2. The diagnostics reveal different relationships between gradient and task effects, so what determines whether equalizing row amplitudes improves task performance?
4 comments
AuthorsOpenReview ↗

Response to Reviewer XKYS: Practical Value of RAA and Row-Amplitude Effects

Response to Reviewer XKYS

We thank the reviewer for recognizing the paper's soundness, its separation of assignments from sample weights, and its distinction between optimizer controllability and task utility. The two questions concern (i) the practical role of RAA and its deviation band, and (ii) when amplitude equalization affects learning. Both are central to our contribution.

1. Practical purpose of RAA and choice of the deviation band

In transport-based learning, row masses can encode confidence, partial selection, or curriculum. Changing these masses may change not only the loss weighting but the actual parameter update, particularly with a stateful optimizer such as Adam. RAA offers an operational way to make that intervention measurable and bounded: at identical model parameters, optimizer moments, auxiliary state, and batch randomness, it compares the optimizer proposals for raw and RMT-modified targets:

Dτ = ‖uτ − u₁‖₂ / ‖u₁‖₂.

A practitioner studying or modifying mass-dependent training can therefore specify an acceptable update-deviation budget, use early training states to select an amplitude coefficient, freeze that choice, and check whether the prescribed optimizer effect persists at future states. This is more informative than comparing mass variance or gradient norms, since two mass configurations can produce different update directions and Adam may transform gradients nonlinearly.

Our nominal 2%–10% band defines a nontrivial but limited optimizer perturbation: the lower bound rejects effectively negligible changes, and the upper bound limits deviation from the raw update. The guarded interval [0.021, 0.099] reduces boundary ambiguity. These thresholds are explicit operational choices, rather than a claim that 2%–10% is universally optimal for accuracy. The paper's objective is to test whether a prefix-chosen intervention can satisfy a stated update budget prospectively.

The observed results answer that question: P2OT selects τ = 0.875, with 19/20, 20/20, and 19/20 future in-band probes across three new seeds (3/3 seeds passing). JUMBOT selects τ = 0 in five fresh seeds and achieves 20/20 future probes in every seed (5/5 passing). A fixed τ = 0.875 policy achieves 0/100 in-band probes in JUMBOT. Thus the same coefficient does not transfer across these settings, while prefix-based selection meets the declared control criterion on each tested continuation. The point is controllability of the actual update, not a promise that RAA-selected τ improves classification accuracy.

2. When and why does row-amplitude equalization matter?

The paper provides more than the generic statement that sample weights change gradients. At fixed conditional assignments and a stopped-target linear-loss interface, its exact mechanism is

gτ = g₀ + τΔ, Δ = c_B m Cov_A(s,h).

Consequently, amplitude heterogeneity alone does not determine the optimizer effect. It matters through its covariance with the conditional sample forces. Nonuniform masses may have Δ = 0; even when Δ ≠ 0, only the transverse component Δ⊥ = Δ − (⟨Δ,g₀⟩/‖g₀‖₂²)g₀ establishes an immediate gradient-direction change beyond scalar rescaling. Moreover, Appendix A.4 characterizes which amplitude interventions are realizable at the KL-UOT solver level and which of those remain visible to the gradient. This gives structural criteria for when an intervention can matter, rather than merely asserting that weighting always matters.

Whether this change improves a particular task metric is a further question. Our fixed-intervention studies demonstrate both value and its limits. On PROTOCOL/Hdigit, RMT improves ACC by 5.09 and ARI by 6.56 percentage points over ten paired seeds. On P2OT/CIFAR100-LT, it improves sample-level ACC by 5.60 and ARI by 6.53 points across five paired seeds, while mACC falls by 3.93 and purity by 2.73 points. Head classes comprise 81.35% of samples; recall changes are +5.90 pp (head), −10.73 pp (middle), and −7.45 pp (tail), illustrating the metric tradeoff. JUMBOT supplies a complementary boundary: prospective optimizer control succeeds, but target accuracy is essentially unchanged (97.604% versus 97.606% for the fixed reference).

These outcomes motivate the paper's deliberate separation of mechanism, optimizer controllability, and downstream utility. We will make the practical update-budget use case, the meaning of the chosen band, and the covariance/solver-visibility conditions more prominent in Sections 3–5, while tightening the exposition to make this three-part distinction easier to follow.

AuthorsOpenReview ↗

Additional Evidence for Reviewer XKYS: Practical Value of RAA and Adaptive Optimizer Control

Thank you for your constructive review. We provide new held-out evidence addressing when RAA offers practical value beyond a fixed amplitude coefficient, and further clarify the relationship between optimizer controllability and downstream learning.

1. When is RAA useful, and why control a deviation band?

In transport-based learning, row masses act as sample weights. Changing their heterogeneity can modify the learning signal even when conditional assignments remain fixed. A practical question is how to introduce a nontrivial amplitude intervention without allowing an uncontrolled change in the actual optimizer update.

RAA addresses this through the same-state relative optimizer-proposal deviation, Dτ=||uτ−u1||/||u1||. Our prespecified guarded band [.021,.099] defines a measurable control objective: the lower bound excludes negligible interventions, while the upper bound limits proposal deviation. The interval is not claimed to be universally optimal for task accuracy.

Rather than selecting τ solely from development data, RAA measures the optimizer response at the current training prefix, selects a qualifying coefficient, freezes it, and evaluates whether the prescribed update effect persists during continuation.

2. New evidence: RAA versus a development-optimized fixed policy

We added a stronger P2OT baseline, Fixed-Dev, which selects a single coefficient from τ∈{0,.25,.5,.75,.875} using ten development seeds (201–210), maximizing mean future in-band coverage. The selected coefficient is τ=.875, with 14.3/20 development probes in band on average and 6/10 seeds passing.

We then compared this frozen policy against RAA-10 on 20 disjoint evaluation seeds (96–115). Both use the same future criterion: at least 18/20 probes within [.021,.099].

Held-out P2OT evaluationRAA-10Fixed-Dev
Successful seeds20/206/20
Future probes in band400/400234/400

The paired seed-level success-rate difference is +70 percentage points (14 RAA-only successes versus 0 Fixed-Dev-only successes; two-sided exact McNemar p=1.22×10⁻⁴, using seeds as the statistical units).

RAA-10 selected τ=.25/.5/.75/.875 across 1/5/8/6 evaluation seeds, respectively. In the six seeds where both policies selected τ=.875, their recorded future-coverage counts agreed.

These results demonstrate a substantial benefit of prefix-conditioned coefficient selection over a genuinely development-optimized fixed policy for the prospective optimizer-control objective in the evaluated P2OT setting. This is a concrete use case for RAA, beyond merely diagnosing gradient changes.

3. When do amplitude interventions matter for learning?

The immediate gradient effect is characterized by Δ=c_B m Cov_A(s,h). Thus, the influence of mass equalization depends on the relationship between row-mass heterogeneity and conditional sample forces, not merely on the existence of unequal weights. Its direction-changing component further distinguishes this effect from scalar gradient rescaling.

Our controlled source-regularization study supports this mechanism: stronger source-marginal regularization compresses mass heterogeneity, the gradient correction, and observed ARI sensitivity. Separately, fixed RMT interventions improve ACC and ARI in PROTOCOL and P2OT, while the reported complementary metrics identify task-dependent tradeoffs.

These findings distinguish two important outcomes: RAA establishes whether a prescribed optimizer intervention can be reliably controlled; downstream task utility depends on how that intervention interacts with learning. Neither outcome is inferred automatically from the other.

Summary. The new held-out experiment establishes a concrete practical advantage for RAA over a development-selected fixed coefficient: 20/20 versus 6/20 successful seeds, with a paired exact p-value of 1.22×10⁻⁴. Combined with the mass–force mechanism and external fixed-intervention results, this strengthens both the practical motivation for RAA and the explanation of when row-amplitude effects become consequential for learning.

We appreciate your careful assessment and hope this additional evidence helps resolve the remaining concerns.

AuthorsOpenReview ↗

Further Evidence for Reviewer XKYS: Practical Utility and Deviation-Budget Sensitivity

Thank you for your continued consideration. We have added a paired P2OT study directly addressing two questions in your review: why controlling optimizer-update deviation is useful, and when row-amplitude intervention benefits learning.

1. A practical control objective

Transport row masses determine how strongly samples contribute to learning. Reducing their heterogeneity changes the learning gradient, but the resulting Adam proposal also depends on the optimizer state.

RAA therefore addresses a concrete question: Can we make a nontrivial amplitude intervention while keeping the change in the actual optimizer proposal within a specified relative-deviation budget?

At a shared optimizer state, RAA measures Dτ=||uτ−u1||/||u1||, selects τ using ten training-prefix probes, and freezes it before continuation.

The guarded band [.021,.099] excludes negligible changes while setting an upper budget for proposal deviation. Its role is an explicit, measurable intervention target rather than a universal accuracy-optimal interval.

2. New evidence: optimizer control and task utility in the same setting

We evaluated Raw, development-selected Fixed-Dev (τ=.875), RAA-10, and full equalization in P2OT/CIFAR100-LT (imbalance ratio .1). The study uses paired seeds 116–125 and a common epoch-20 task endpoint.

Under the original deviation band, RAA-10 met the future-control criterion (at least 18/20 probes in band) on 9/10 seeds, with 176/180 future probes in band. The remaining seed abstained during coefficient selection. Fixed-Dev met the criterion on 4/10 seeds, with 125/200 probes in band.

On the nine seeds with an RAA intervention, the paired task results were:

ComparisonΔARI (pp)ΔACC (pp)
RAA-10 − Raw+5.16+3.52
RAA-10 − Fixed-Dev+0.86+1.23
RAA-10 − Full Equalization+1.60+1.38

RAA-10 improved both ARI and ACC over Raw in all nine evaluated pairs.

Compared with full equalization, RAA-10 also achieved +4.07 points in class-average accuracy (mACC) and +2.76 points in purity.

These results demonstrate a concrete learning benefit in the evaluated setting: a controlled, prefix-selected amplitude intervention improves clustering performance while avoiding some of the class-balanced performance losses associated with unrestricted equalization.

3. Why does the deviation budget matter?

We additionally examined three deviation bands on the same ten seeds:

Deviation bandPrefix abstentionsFuture-success seeds
Strict [.021,.049]10/10No continuation
Original [.021,.099]1/109/10
Wider [.021,.199]0/1010/10

The strict budget admits no qualifying coefficient, whereas the original and wider budgets permit successful prospective control.

On the nine seeds with both original- and wider-budget continuations, the wider band selects stronger equalization on average: the mean selected τ decreases by 0.25.

However, the additional intervention strength produces only +0.20 ARI points on average, alongside −1.06 mACC points.

This demonstrates why the deviation budget is meaningful: it affects intervention feasibility and strength, while enabling an explicit comparison between optimizer perturbation and downstream performance.

A wider budget allows stronger intervention, but need not produce a corresponding improvement in task utility.

4. When does amplitude equalization help learning?

Our theoretical mechanism explains why the effect is state- and task-dependent.

The gradient correction is proportional to the covariance between row masses and conditional sample gradients. Therefore, unequal masses alone do not determine the optimization effect; their alignment with the learning forces matters.

The new paired task results complement this mechanism. They show that amplitude interventions can improve clustering outcomes, while the budget sensitivity results demonstrate that stronger equalization does not necessarily lead to greater downstream gains.

Summary

The additional experiments directly strengthen the practical motivation for RAA:

  • Optimizer control: RAA provides a state-aware method for selecting a nontrivial amplitude intervention under a measurable update-deviation budget.
  • Learning utility: In the paired P2OT study, RAA improves ARI and ACC over Raw on all nine seeds with an eligible intervention.
  • Budget sensitivity: Changing the deviation budget affects intervention feasibility, equalization strength, and the observed task-performance tradeoff.

Thus, RAA contributes more than an optimizer diagnostic: it connects controlled row-amplitude intervention with measurable downstream learning outcomes.

We hope these new results address your concerns about the practical purpose of RAA and the conditions under which row-amplitude interventions are beneficial.

As the discussion period is approaching the deadline, we would like to kindly follow up and ask whether you might be able to share any further comments or considerations regarding our paper. Thank you again for all your time and effort in reviewing our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Main Contributions, New Evidence, and Manuscript Revision

We thank the reviewers and the Area Chair for their thoughtful evaluation and constructive discussion. Their feedback has helped us sharpen the central message and strengthen the experimental evidence.

Our central question is simple: When transport assignments remain fixed, how do row masses affect learning, and can their influence on optimization be understood and controlled?

The paper answers this question through three connected contributions: an isolated optimization mechanism, an exact structural characterization, and prospective optimizer control. Our rebuttal and manuscript revision reinforce each of these contributions.

Structural contribution (Reviewer KQns). The contribution goes beyond the observation that sample reweighting changes gradients. RMT isolates row-amplitude effects while preserving conditional assignments and total mass. We derive their exact mass–force covariance correction and characterize when the gradient changes direction rather than merely scale. Proposition A.1 establishes which fixed-assignment amplitude changes are realizable by the specified OT solver; Corollary A.2 determines which of those changes are visible to learning. Together, these results reveal a distinction between what the transport solver permits and what the learner can detect.

Optimizer control and strong baselines (Reviewers KQns and b6Lt). RAA translates this mechanism into a measurable intervention using actual optimizer proposals. We added prefix-length ablations and a stronger fixed-policy baseline selected on independent development seeds. On 20 held-out P2OT seeds, RAA satisfies the prospective control criterion on 20/20 seeds, compared with 6/20 for the development-selected fixed policy (paired exact p=1.22×10⁻⁴). This directly establishes the value of prefix-conditioned coefficient selection in the evaluated setting.

Practical learning value (Reviewer XKYS). We further added a paired P2OT experiment measuring optimizer control and downstream performance under the same training conditions. RAA selects an intervention on nine of ten seeds and meets the future-control criterion on all nine. On these paired continuations, ARI and ACC improve over Raw by 5.16 and 3.52 percentage points, respectively. New deviation-band experiments show how the update budget governs intervention feasibility and strength, while revealing that stronger equalization does not necessarily yield better task performance.

Manuscript revision. We have reorganized the main narrative around these three contributions, brought the structural results into the main text, and incorporated the new comparisons, evaluation protocols, and task-metric analyses. The distinct objectives of optimizer control and downstream utility are now presented more clearly, with the numerical limitations of the separate full-training scale comparison retained.

In summary, the paper establishes row amplitude as a distinct, measurable, and controllable optimization mechanism in transport-based learning. Its contribution combines mathematical characterization, optimizer-aware intervention, prospective validation, and demonstrated task-level benefits.

We appreciate the reviewers' careful consideration of these strengthened results and the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.