- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GRADHOP to adapt a server-held target model to private client data without requiring clients to access or optimize that model. Each client uses a smaller shared reference model to compute learning directions and sends them to the server once. The server uses these directions to select examples from a PUBLIC repository and adapts the target model on the selected examples. The paper analyzes when this cross-model selection can approximate the private target gradient and evaluates the method on Banking77 and CLINC150.
Critical strengths
-
Clients do not need to host or differentiate through the large target model, and they communicate with the server only once. This addresses a practical constraint when clients have limited computational resources or cannot access the target model.
-
The reference and target models need not have matching parameters or gradient dimensions. Selecting PUBLIC examples in the reference space and optimizing them in the target space provides a clear way to connect two independently parameterized models.
-
The experiments include two datasets, a target-gradient alignment analysis, an ablation of mode-specific directions, and tests across several target-model families. These results provide useful evidence for both the adaptation performance and the proposed selection mechanism.
Critical weaknesses
-
The paper discusses related approaches such as FedProxy, Offsite-Tuning, and LESS, but the main comparisons are limited largely to random, label-balanced, and all-PUBLIC selection. Although these approaches have different access assumptions, the authors should explain the differences more carefully and compare with stronger methods under matched constraints. In particular, a reference-embedding or prototype-based PUBLIC retrieval baseline would help establish whether gradient directions are necessary.
-
The main experiments consider one private client per adaptation run, while the multiple-client experiment reports only aggregate results. Since different clients may benefit differently from the adapted target, the authors should report per-client performance, worst-client performance, and the number of clients experiencing negative transfer. Average accuracy and NLL may conceal degradation for individual clients.
-
The PUBLIC repository contains examples from the same set of intent labels used in each client task, and the evaluation is limited to intent classification on two datasets. The authors should test what happens when relevant PUBLIC examples are scarce or absent, when the PUBLIC and private distributions differ substantially, and when the method is applied to other tasks. The construction and size of the PUBLIC repository, as well as its separation from private and test data, should also be stated more clearly.
-
Eq. (11) defines a positive reference-space margin by requiring the target-space oracle support to rank ahead of all other PUBLIC examples. This assumes a strong relationship between the two model spaces, but the paper does not establish when that relationship holds. Moreover, Eq. (13) concerns a single top- ranking, whereas the implemented method combines aggregate and mode-specific rankings; the final error bound in Eq. (16) depends on the unmeasured support mismatch . The authors should measure these quantities or provide a more direct analysis of the implemented retrieval procedure.
-
Corollary 1 requires the re-grounding error to be smaller than the norm of the private target gradient, but Table 1 reports a mean relative error of 1.075 for GRADHOP. Thus, the reported aggregate result does not demonstrate that the condition holds. The target model is also adapted using multiple AdamW steps, while the sufficient condition for final private-objective improvement in Eq. (27) is not evaluated. The authors should report how often these conditions hold or clarify that the theoretical results do not directly explain the observed improvements.
Questions for authors
See Weaknesses.