- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RGM-SL studies communication-constrained socialized learning under disjoint label partitions. After expert-based bootstrap, each expert refines a small synthetic image packet by matching real- and synthetic-data gradients on the current generalist, with teacher classification and feature regularization. The server trains with hard labels, masked soft targets, and reference-expert anchoring. A frozen generalist subsequently supports local feedback heads and a router. Evaluation uses ConvNet3 on CIFAR-10, CIFAR-100, and Tiny-ImageNet, with budgeted baselines and a matched-IPC Random Real comparison.
Critical strengths
-
The receiver-conditioned formulation is well motivated: Eqs. (9)–(13) adapt transmitted examples to the current generalist rather than optimizing only for a separate expert. The formulation clearly identifies the optimized pixels and fixed receiving model.
-
The complete refinement pipeline improves Overall accuracy over Random Real on all three datasets at the same IPC and round count: 47.61 to 49.02, 35.44 to 37.62, and 20.66 to 21.06 (Table 4). This provides evidence that refining packet content can be useful without adding image slots.
-
The evaluation separates generalist performance from routed community performance, reports local versus non-local class accuracy, and uses validation-selected checkpoints with held-out test sets. These distinctions make the intended tradeoffs easier to inspect.
Critical weaknesses
-
Inconsistent evaluation numbers. Under the stated balanced CIFAR-10 test set and 5-by-2 partition, Overall must equal 0.2 times Expert plus 0.8 times Non-expert. Table 1 gives FedAvg as 55.90/45.96/48.54, but these subset values imply 47.95 Overall. FedDM similarly implies 47.77, versus the reported 48.46. These gaps exceed rounding and leave the headline baseline comparisons insufficiently verified.
-
The central receiver-aware mechanism is not isolated. Table 4 compares the complete pixel-refinement pipeline with no pixel optimization, while Eq. (12) also includes teacher classification and feature-statistic regularization. It does not establish whether updating the receiver state is responsible for the gain. A fixed-receiver comparison and a zero receiver-matching-weight control with the auxiliary terms retained would directly test the main contribution.
-
Small gains are supported by only one seed. Section 5.1 and Appendix A explicitly report seed 42 only. The Overall margins over FedAvg are 0.48 points on CIFAR-10 and 0.75 on Tiny-ImageNet; the Tiny-ImageNet gain over Random Real is 0.40 points. Without repeated runs and uncertainty estimates, the robustness of these improvements is unresolved.
-
Communication efficiency is not quantitatively demonstrated. Eqs. (19)–(23) define accounting, but the paper does not tabulate actual uplink/downlink totals, transport precision, or the FedAvg/FedProx round counts admitted by the budget. One-shot methods instead match image counts, which does not establish equal byte budgets. Without these values or accuracy-versus-bytes results, the claimed resource tradeoff cannot be assessed.
-
The added value of feedback remains unclear. Table 2 shows mean Expert accuracy falling from 95.58 to 70.94, 56.45 to 44.95, and 43.93 to 32.22. Community Overall accuracy after feedback is also below using the generalist alone, and Table 3 reports substantially stronger MASC results. Nonzero unseen-class accuracy is partly supplied by routing to the generalist. A generalist-only and an unchanged-expert-plus-generalist routing control are needed to establish what the trained feedback heads contribute.
Questions for authors
Clarifications that could change the assessment:
-
Can you reconcile the inconsistent Table 1 subset and Overall accuracies, specifying test counts and confirming that all three metrics come from the same checkpoint and predictions?
-
With the other losses and packet initialization fixed, how much do the current receiver state and the receiver-gradient term each contribute? Do the small gains persist across repeated seeds?
-
What are the actual byte budgets and eligible baseline round counts? At comparable end-to-end cost, what benefit do the feedback heads provide over using the generalist alone or routing between the unchanged expert and generalist?