- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper defines Hidden Directional Opposition from negative pairwise inner products and aggregate directional energy. Client endpoints are projected onto a sphere and mapped to a common tangent space. A2 then applies scalar adaptive scaling to the aggregated first moment. Experiments characterize directional opposition across heterogeneity levels, training rounds, and network layers.
Critical strengths
-
The energy decomposition separates positive and negative pairwise contributions to the aggregate update.
-
The experiments cover CIFAR-10 and CIFAR-100 with multiple seeds. The paper also distinguishes tangent-step direction preservation from predictive performance and acknowledges that FedAdam achieves higher accuracy.
Critical weaknesses
-
Table 1 is inconsistent with the stated definitions. For nonnegative weights summing to one, . However, CIFAR-10/C2 at reports and . Several other rows also violate this inequality. Different scaling or aggregation conventions need to be disclosed; otherwise, the central empirical characterization is not reliable.
-
The distinction between HDO and existing conflict measures remains insufficiently established. is a magnitude-weighted sum of negative pairwise inner products, and normalizes it using aggregate energy. Both are computed from client updates. The paper does not demonstrate what additional information they provide beyond conventional conflict and cancellation statistics.
-
The spherical representation can discard the directional opposition being studied. For and with , both projected endpoints equal , so both tangent representations vanish. The paper needs to delimit its analysis to angular opposition and explain what information is lost through projection. Table D1 provides little evidence of an advantage over raw updates.
-
A2 preserves the aggregated first-moment direction, not the individual client directions lost during averaging. It does not use or to recover or respond to hidden opposition. The direction-preservation experiment verifies scalar collinearity, but does not establish preservation of client-level information.
-
Scalar adaptive scaling has precedents in AdaSGD and VectorAdam, which are not discussed. The comparisons also lack a Euclidean scalar-adaptive baseline or a tangent-space momentum baseline, leaving the contributions of geometry, momentum, scalar adaptation, and angular clipping unresolved.
-
Important implementation details are missing, including the momentum transport rule, server learning rates, moment coefficients, angular cap, and numerical safeguards for the spherical maps. The auxiliary methods SW2 and FMO are also insufficiently specified. These omissions prevent reproduction and make the optimizer comparisons difficult to assess.
Questions for authors
-
How can Table 1 be reconciled with ? Are the columns computed from the same vectors, weights, normalization, and temporal aggregation?
-
What information does HDO add beyond magnitude-weighted negative inner products and standard cancellation measures? Do existing results isolate this distinction?
-
How much update energy is removed by spherical projection? How should radial opposition, which maps to zero, be interpreted within the proposed observability framework?
-
What client-level directional information does A2 retain beyond the aggregated first moment? Is there an existing diagnostic demonstrating this retention?
-
How does A2 differ from existing scalar-adaptive optimizers, and are there controlled comparisons separating geometry from momentum and adaptive scaling?
-
Please specify the momentum transport, server hyperparameters, angular cap, and the procedures used by SW2 and FMO.
-
Please retain the standard ICLR body font and formatting. The axis labels and tick values in several figures are too small to read comfortably.