acceptodds
Under review as a conference paper at ICLR 2027

What Determines a Local Update Map?

Abstract

What information determines the result of local optimization? Two groupings of the same strictly convex quadratic losses can preserve the objective and complete separate gradient and Hessian marginals, yet produce different two-step updates. Their within-child pairing resolves this ambiguity. For quadratic children, agreement at the first horizons identifies all later endpoints at that input, where counts the actual union spectrum. Agreement at other inputs additionally depends on matrix-valued spectral weights: even an entire shared trajectory and matching eigenvalue distributions can leave the maps different elsewhere. We realize sharp information limits with partitions of the same atomic losses, including noncommuting children, and quantify the observation depth needed with noisy logs. Smooth remainder bounds and controlled logistic and trained-neural evaluations locate the range of leading-pairing prediction beyond quadratics. A ridge comparison illustrates a further distinction: covariance-aware recovery can resolve later-update comparisons that spectrum-only bounds leave undecided with the same early logs. The results separate information that controls the size of an update change from information that identifies the update itself.

Primary area:
optimization
Keywords:
local optimization, distributed optimization, identifiability, finite-step dynamics, gradient-curvature coupling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_t34AOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This work studies the identifiability of training grouping of losses with gradient descent. If the losses are quadratic functions and matrix-valued spectral weights (defined in Section 4) are known then the local dynamics is determined; bounds are given for non quadratic losses.

Critical strengths

I believe that a strength of the paper is to characterize theoretically what is needed in order to identify the training dynamic by gradient descent starting from a weight w_0 of groupings of quadratic losses and to have bounds in the non quadratic case.

Critical weaknesses

My feeling is that for a non specialist of federate learning like me, it is very difficult to understand the setup and the claims of the paper and one has to persist in order to get to understand the contribution of the paper. Table 1 helped me a lot to wrap up my understanding of the paper.

Questions for authors

No question that could materially affect my assessment.

1 comment
AuthorsOpenReview ↗

Response to t34A: the setup and how to read Table 1

Thank you for pointing out that Table 1 helped. We have rewritten the opening around the computation below and added an explanation of that table.

Divide a fixed set of training losses into groups. Start each group at the same parameters, take E gradient steps on its average loss, then average the final parameters by group size. We compare two groupings with the same losses, overall objective, starting point and update rule. Each test starts afresh; different horizons are not successive server rounds.

The question is what information lets us determine whether the two computations return the same parameters. The aim is to identify the update, not to recover the unique grouping that produced it.

Figure 1 gives the simplest example. Two groupings use exactly the same losses and have the same separate lists of gradients and Hessians at the starting point. After two steps, their differences from training on the unsplit data are 0 and 0.014. The difference comes from which gradient is paired with which Hessian. Knowing the two lists without those pairings loses information needed to predict the update.

Section 4 asks whether observing more updates removes the ambiguity. Even matching every horizon from one starting point need not give the same result elsewhere. The quadratic results specify what spectral information and observations suffice to determine later updates. With noisy observations, the paper bounds how different a later update can still be across the permitted losses.

For example, a participant may assign its data to synchronized workers. An unchanged objective does not ensure that regrouping preserves the returned update. We isolate that distinction with deterministic, stateless, full-batch gradient descent.

The opening, Table 1 and Figure 1 now provide a short route into the problem before the longer-horizon results in Section 4. Section 6 tests where pairing predicts nonlinear updates and where it fails.

Reviewer_CY1TOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper asks what information determines the endpoint produced by taking several gradient steps within groups of losses and then averaging the resulting parameters. The paper establishes exact and noisy identification boundaries and constructs examples realizing them through different partitions of the same atomic losses.

Critical strengths
  • A clear conceptual distinction. The paper separates having the same aggregate objective, the same trajectory from one starting point, and the same update map across inputs. Its demonstration that separate gradient and Hessian statistics can lose the crucial within-child pairing is useful.
  • Sharp constructions with meaningful constraints. The strongest contribution is realizing different behaviors through partitions of the same strictly convex atomic loss pool, while preserving specified derivative or spectral information. The noncommuting and equal-mass constructions deserve credit; describing the entire contribution as elementary algebra would understate this part.
  • Candid reporting of limitations. The authors acknowledge unsuccessful prediction regimes and distinguish agreement between update maps from improvements in downstream task performance.
Critical weaknesses

The theoretical advance is narrower than the manuscript’s breadth suggests. Taylor expansion gives the leading discrepancy; polynomial representations and Vandermonde inversion give finite-spectrum identification; classical approximation and optimal-recovery arguments supply the bounds. Whether that constitutes a substantial contribution is a legitimate concern.

The exposition is unnecessarily difficult. The authors explicitly disclose AI assistance with drafting and editing on p. 10.

The practical information assumptions need stronger motivation. The guarantees concern deterministic full-batch GD, with additional assumptions depending on the result. Their relevance to stochastic updates and stateful optimizers is not clear to me.

The experiments establish limited practical benefit.

Questions for authors

Can the nonlinear validity boundary be predicted quantitatively from the remainder analysis?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.