acceptodds
Withdrawn from ICLR 2027

ANALYTIC FEDERATED LEARNING IS VIABLE FOR REGION-BASED OBJECT DETECTION

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Federated object detection typically requires repeated exchanges of model updates, while one-shot AFL promises a single upload followed by a closed-form solve. Whether this promise extends beyond fixed-output classification is unclear because detection must handle variable proposal sets, severe class imbalance, and coupled classification and localization. We introduce an analytically solved-head framework that compiles each client's supervision into additive Gram, cross-moment, and count statistics over a shared frozen proposal representation. Aggregating these statistics exactly recovers the pooled classification and box-regression objectives for any horizontal partition, without transmitting raw images, proposal features, gradients, or locally trained detector weights. With one configuration selected on VOC2007 train/validation, the resulting detector achieves AP50 on VOC2007 test and on VOC2012 validation without retuning; its float32 sufficient statistics are smaller than the corresponding raw JPEGs. Additional full-split controls show stable prior calibration for and a 22.2-ms server solve for the selected system. These results show that partition-invariant, one-shot AFL can support a structured, multi-function vision task while retaining useful detection accuracy across dataset releases, establishing object detection as a viable target for AFL.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Learning, Analytic Readout, Sufficient Statistic

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_76ewOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a one-shot analytic federated learning framework for region-based object detection, extending analytic federated learning beyond image classification. The proposed method freezes the pretrained detector and aggregates sufficient statistics from clients to analytically solve classification and bounding box regression heads without iterative communication. The authors theoretically demonstrate exact pooled recovery regardless of data partitioning and evaluate the framework under heterogeneous client distributions. Experiments on PASCAL VOC2007 and VOC2012 demonstrate reasonable detection accuracy with a single communication round, while a prior-calibrated variant further improves performance by leveraging predictions from a pretrained detector.

Critical strengths
  • The paper presents a one-shot analytic federated learning framework for object detection, addressing detection-specific challenges such as variable region proposals, class imbalance, and bounding box regression.

  • The paper provides a theoretical guarantee of exact pooled recovery and evaluates the proposed approach across multiple data partitions and benchmark datasets.

Critical weaknesses
  • Limited contribution and insufficient justification for head-only adaptation. The authors justify freezing the backbone and RPN as necessary for enabling closed-form solutions and exact pooled recovery. However, this is primarily a methodological justification rather than a demonstration of practical utility. The paper does not sufficiently explain why restricting federated adaptation to lightweight classification and box regression heads is beneficial in real-world object detection scenarios. Furthermore, the performance limitations introduced by the inability to update the full detector are not adequately investigated.

  • Questionable communication efficiency. Since the proposed method only adapts lightweight detection heads, conventional head-only FedAvg could also achieve relatively low communication costs. In fact, the proposed method transmits sufficient statistics that are substantially larger than the head parameters themselves, raising concerns about its practical communication advantage. Moreover, the reported 52.88x communication reduction is measured against transferring raw images, rather than against conventional federated learning methods, making the comparison less convincing.

  • Unconvincing accuracy-efficiency trade-off. The independently trained analytic heads achieve substantially lower detection accuracy (78.48 AP50) than the original pretrained detector (89.50 AP50). Given that head-only federated optimization is already communication-efficient, it is unclear whether such a performance degradation is justified by the proposed one-shot formulation. Furthermore, simple local-head averaging achieves higher AP50 in some experiments, further questioning the practical benefits of exact pooled recovery.

  • Overreliance on strong pretrained detection priors. The prior-calibrated approach relies on a pretrained detector whose class vocabulary already covers all target categories and whose detection performance is already strong. The proposed method provides only a marginal AP50 improvement over the frozen detector (89.50 → 89.75). This raises a fundamental question about the necessity of additional federated adaptation in such a setting. More importantly, the effectiveness of the approach remains unverified in scenarios where federated learning would be more valuable, such as novel-category detection or substantial domain shifts.

  • Limited scalability and generalizability. Although the authors investigate scalability with respect to feature dimensions and the number of classes, they do not demonstrate end-to-end federated adaptation of the full detection model. Consequently, it remains unclear whether the proposed approach can be extended beyond lightweight prediction heads or whether its benefits generalize to larger-scale detection tasks and more diverse architectures.

  • Lack of competitive federated learning baselines. The paper lacks direct comparisons with conventional federated optimization methods, such as head-only FedAvg or FedProx, under the same frozen-backbone setting. Without controlled comparisons of detection accuracy, communication overhead, and computational cost, it is difficult to determine whether exact pooled recovery provides meaningful practical advantages over simpler federated learning alternatives.

Questions for authors
  • What is the practical motivation for restricting federated learning to lightweight detection heads? Since the proposed method only updates a small number of head parameters, communication overhead is already minimal even with conventional head-only federated optimization. In this setting, why is a one-shot analytic approach necessary, and what practical advantages does it offer over simpler methods such as head-only FedAvg, particularly in terms of the trade-off between communication cost and detection accuracy?

  • The reported 52.88x communication reduction is measured against raw image transmission rather than conventional federated learning methods. Can the authors clarify the actual communication advantages over head-only FedAvg or FedProx under the same frozen-backbone setting? A comparison accounting for communication rounds, total transmitted bytes, and computational overhead would be important for assessing the practical benefits of the proposed approach.

  • The frozen pretrained detector already achieves 89.50 AP50, while the prior-calibrated approach improves this to only 89.75 AP50. What is the practical motivation for performing additional federated adaptation when the pretrained detector already performs so well? Furthermore, how would the proposed approach remain effective when such a strong prediction prior is unavailable, particularly for novel categories or substantial domain shifts?

  • The independent analytic heads substantially underperform the pretrained detector, and simple local-head averaging achieves higher AP50 in some experiments. What practical benefits justify this accuracy degradation in exchange for exact pooled recovery? Can the authors clarify under what conditions exact pooled recovery would be preferable to simpler federated optimization methods?

  • To what extent can the proposed framework be extended to larger detection architectures or end-to-end federated adaptation? Is the restriction to frozen representations a fundamental limitation of the proposed formulation, and how does this affect its applicability to real-world federated object detection?

Reviewer_rUAaOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper extends analytic federated learning to object detection. With a COCO-pretrained Faster R-CNN kept frozen, each client uploads statistics such as Gram matrices and counts of proposal features only once, and the server sums them to solve ridge heads for classification and box regression in closed form. Since the summed statistics match the pooled objective, the same heads are recovered for any partition, and the paper reports 78.48 AP50 on VOC2007 test (89.75 in the mode that combines the frozen detector's outputs).

Critical strengths

S1. Clear one-upload structure. The structure in which each client uploads only once and the server solves in closed form is clear, and the paper verifies that the pooled result is exactly identical across 15 non-IID partitions.

S2. Design that accounts for detection-specific properties. Global count-based class balancing and separate classification and box heads address the imbalance and localization problems of detection (Table 5).

Critical weaknesses

W1. Lower performance than the untrained frozen detector. The COCO-pretrained detector used here already covers all VOC classes and, without any training, achieves 89.50 AP50 on VOC2007 test and 86.77 AP50 on VOC2012 val. In contrast, the proposed analytic heads achieve 78.48 and 75.32 AP50, about 11 points lower. The prior-calibrated mode also gains only +0.25 AP50 over the frozen detector and +0.45 AP50 over prior-only, and is even lower than prior-only on AP@[.50:.95] (64.70 vs. 65.45). Thus, federated training itself is unnecessary in this setting, and the claim of being "viable" would require evaluation on new classes or domains that the pretrained detector cannot handle. For the same reason, the comparison in Table 3 showing higher performance than Faster R-CNN trained only on VOC (73.2) is weak, since it favors the proposed method, which uses COCO-pretrained features. Overall, the experimental setting itself appears to be poorly chosen for validating the necessity of the proposed method.

W2. Missing key baselines. There is no comparison with iterative training of the same linear heads via FedAvg/FedProx, centralized fine-tuning, or other one-shot FL methods. The communication comparison is also based on raw JPEGs (52.88× smaller) or full detector updates, but iteratively training linear heads of the same size (about 193×25 parameters) requires only tens of KB per upload, so even tens of rounds could cost less than the proposed method's 1.67 MB. Moreover, simple local head averaging sometimes outperforms the pooled head (76.50/76.85 vs. 75.32 on VOC12, L357–359; local averages of 77.93–81.86 in Appendix D), so the practical benefit of exact recovery is also unclear.

W3. The adopted class-balancing rule and inconsistencies across experiments. In Table 9, the inverse-frequency rule adopted by the authors (74.94) is the lowest of the four rules compared, 6–8 points below uniform (80.89) and effective number (83.16). Yet Table 5 reports that removing class balancing drops performance from 79.51 to 38.08, leaving the relationship between the two results unclear. In addition, the single-FC reference in Section 6.6 (74.94) and the AAD three-branch extension (76.09) are below the validation performance of the main configuration (79.51, Table 5), so the significance of the three-branch readout claimed as a contribution is also unclear.

W4. Evaluation limited to outdated detectors. The method is applied only to Faster R-CNN (ResNet-50-FPN and MobileNetV3-FPN), which is distant from the current trend of detection research centered on DETR-style detectors and large-scale pretrained backbones. The experiment in Appendix J that uses DINOv2 features as the RoI representation also yields less than 9 AP50, providing no evidence that the framework can be combined with modern backbones. In addition, the contributions claim that the DINOv2 and FCOS studies establish the division of roles in the three-branch readout, but these are, respectively, an experiment with different features and a control for checking the evaluator, and do not support this claim.

Questions for authors

Please check the weaknesses.

Reviewer_GZp1OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an analytic federated learning (AFL) framework for region-based 2D object detection tasks. Different from previous AFL works, which focus on image classification, this work extends AFL to object detection.

A COCO-pretrained Faster R-CNN is adopted, whose RPN and RoI box head are frozen. For each client, the authors adopt at most 300 RPN proposals for each local image and get the 1024-dim features from the fc head of the R-CNN. Then the 1024-dim features are fed into a fixed shared random map, and the client computes the sufficient statistics (i.e., the class-conditional Gram matrices, cross-moments and counts) from the mapped features and uploads them to the server.

After the server received all information from clients, the server sums the statistics from all clients, computes the class weights from the global counts, and solves the classification head and the box regression head in closed form.

The authors conduct experiments on VOC07 and VOC12 dataset. The proposed analytic mode achieves 78.48 AP50 on VOC07 test and 75.32 AP50 on VOC12 val. Besides, the authors propose a prior-calibrated pooled mode, which combines the recovered classification head with the class probabilities and box deltas of the frozen detector. The prior-calibrated pooled mode achieves 89.75 AP50 on VOC07 test and 86.92 AP50 on VOC12 val.

Critical strengths
  1. The proposed framework for AFL in object detection is simple and reasonable. Since all statistics uploaded by clients are additive, aggregating these statistics exactly recovers the pooled classification and box-regression objectives for any horizontal partition.

  2. The authors conduct fair experiments. They tune the hyper-parameters only on VOC2007 train/val, and then freeze them for testing on VOC2007 test and VOC2012 val. The reported experiment results are not overfitted to the test sets and are reliable.

Critical weaknesses
  1. The experimental setting can't support the motivation of this paper.

The motivation of AFL is that multiple clients can train a model for their own tasks without sharing private data. However, in this paper, the frozen Faster RCNN is pretrained on the COCO dataset, whose categories cover all the categories in VOC (the frozen detector achieves 89.50 AP50 on VOC07 test without training). The proposed analytic mode only achieves 78.48 AP50, which is 11 points lower. The prior-calibrated mode achieves 89.75 AP50, but it relies on the outputs of the frozen detector and is only 0.25 points higher than the pretrained detector. Based on the results, I consider that the federated learning is not necessary in this setting. The authors should conduct experiments where the categories or domain of the target data are different from the pretrained detector, e.g., using the COCO-pretrained detector to detect the categories that are not in COCO.

  1. The novelty is limited. The proposed method of using additive Gram statistics has been used by previous works, such as [1] and [2]. Extending the AFL from image classification to object detection is straightforward. As a result, I consider the proposed method more like a linear probing on the frozen ROI features based on the existing AFL framework.

[1] AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models. CVPR 2025.

[2] Accelerating Heterogeneous Federated Learning with Closed-form Classifiers. ICML 2024.

Questions for authors
  1. Can the proposed method be adopted to one-stage detectors, such as FCOS and RetinaNet?
  2. Can the proposed method be adopted in the scenario where the target categories are not covered by the pretrained detector?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.