- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper asks whether two models that fit the training data or have low training loss can be connected by a sequence of local changes without exceeding a given loss budget. The authors apply an existing complexity classification for Boolean constraint reconfiguration to this problem. They then give margin and grid-resolution conditions for connecting halfspaces, show that allowing classification errors can introduce barriers, and construct discrete low-loss paths for two-layer networks. The network result requires width at least twice the sample size, together with the stated loss and parameter assumptions.
Critical strengths
The halfspace and network results give explicit bounds involving margins, grid resolution, and path loss. The paper also explains which results come from prior work and what is added by the discrete analysis
Critical weaknesses
-
The paper gives loss bounds for discrete paths, but no numerical results showing how large the loss increase is at a fixed precision. It is hard to tell how conservative the bounds are or how these paths compare with continuous paths between the same models.
-
The time and number of updates needed to construct and follow these paths are not evaluated. The construction also allows output weights up to mK, where m is the network width and K bounds the endpoint weights. It is unclear whether the paths remain practical when both computation and weight ranges are limited.
-
The paper mentions model migration and local search as motivations. However, it is not clear why these settings need updates that change only one parameter by one grid step. A concrete example would help explain why this restriction matters.
Questions for authors
-
Could the authors construct these paths between trained two-layer networks and report the maximum loss along them for different grid resolutions, widths, and sample sizes? How do the results compare with the bound and with continuous paths between the same models?
-
In the model-migration or local-search examples, why would updates need to change one parameter by one grid step at a time? What would the discrete guarantee help us do in that setting?