acceptodds
Withdrawn from ICLR 2027

Decoupling Direction and Magnitude in Federated Remote Sensing Object Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Federated learning provides a practical paradigm for collaborative remote sensing object detection without centralizing raw data, yet it is severely hindered by client heterogeneity. This heterogeneity manifests as a dual bottleneck: local detectors struggle to isolate discriminative object cues from complex aerial backgrounds, and conflicting classifier updates across clients cause destructive vector cancellation during aggregation, which shrinks classifier norms and severely attenuates prediction confidence. To address this, we propose CoRD-Fed, a dual-level federated framework that synergistically resolves both issues. On the client side, an object-centric region distillation module restricts knowledge transfer to informative target regions via ground-truth-constrained matching and CAM-guided spatial–channel alignment, effectively suppressing background interference. On the server side, a True Energy-Restored Consensus Aggregation (TERCA) strategy explicitly decouples update direction from magnitude. It estimates category-wise directional consensus to suppress conflicting gradients, then restores the aggregated classifier norm using the uncompromised sum of individual update energies, thereby preventing confidence collapse. Extensive experiments on DIOR and DOTA under severe non-IID partitioning demonstrate that CoRD-Fed consistently outperforms state-of-the-art baselines, yielding notable improvements in small-object detection and average recall.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Federated Learning, Remote Sensing Object Detection, Knowledge Distillation, Non-IID Data Heterogeneity, Classifier Aggregation, Consensus-Aware Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WabQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CoRD-Fed. It combines object-region feature distillation from a larger local detector with category-specific rescaling of federated classifier updates. Experiments use Faster R-CNN + FPNand resnet on DIOR and DOTA, Dirichlet label skew, and partial participation. The principal claim is that preserving update magnitude prevents confidence collapse.

Critical strengths

The combined framework has measurable empirical benefit on two datasets. Table 1 (p. 7) reports gains over FedAvg of 2.26 AP on DIOR and 5.27 on DOTA, with higher small-object recall. This supports investigating the combination, although the margins over the strongest baseline are much smaller: 0.39 and 0.46 AP.

The module ablation is useful. Table 2 separates region distillation and TERCA, rather than attributing the complete improvement to aggregation alone. This is consequential because either component can improve the baseline.

Critical weaknesses

The core idea of the paper is very similar to FedMRUR[1], but the paper did not mention it.

The paper conflates update magnitude with classifier confidence. The abstract says cancellation “shrinks classifier norms and severely attenuates prediction confidence” (p. 1), but the method restores the norm of an update, not the resulting classifier. An update can oppose the existing weights. A larger weight norm also need not increase the correct-class logit or improve calibration. The causal chain from cancellation to confidence collapse is therefore not established.

Rescaling a conflicting average is not conflict resolution. TERCA claims a direction that “suppresses conflicting gradients” ( lines 282–284), yet its direction is exactly FedAvg's. A scalar cannot identify or remove a harmful client's directional contribution.

The experiments do not establish the claimed federated advantage of the combined design. Section 4.1 specifies a larger local teacher and ImageNet initialization, but does not explain its detection training or access to client data. The evidence for the claimed synergy is also mixed: RDChannel alone exceeds RDChannel + TERCA in AP and recall (Table 3, p. 8), despite the claim “Combining both achieves optimal performance across all metrics” (p. 8). The complete system's gain does not by itself validate the proposed interaction between local knowledge transfer and global confidence preservation.

Line 140 "Sec. ??:". Missing referenced section.

[1] An, Xuming, Li Shen, Han Hu, and Yong Luo. "Federated learning with manifold regularization and normalized update reaggregation." Advances in Neural Information Processing Systems 36 (2023): 55097-55109.

Questions for authors

See above

Reviewer_Ga1QOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CoRD-Fed, a dual-level federated framework for remote sensing object detection, aiming to address both: i) local detector limitations in isolating discriminative object cues from complex aerial backgrounds; and ii) conflicting classifier updates across clients on parameter aggregation, leading to diminishing classifier norms, and thus an attenuation in prediction confidence. To this end, CoRD-Fed employs an object-centric region distillation module, restricting knowledge transfer to informative target regions via ground-truth-constrained matching and CAM-guided spatial–channel alignment for local federated training. In addition, it also employs a true energy-restored consensus aggregation (TERCA) strategy for parameter aggregation in federated learning to decouple the update direction of local classifiers from their magnitudes. To assess the effectiveness of CoRD-Fed, this paper compares it with several federated learning and knowledge distillation algorithms on DIOR and DOTA datasets, in addition to sensitivity and ablation analyses.

Critical strengths

The presented topic, remote sensing object detection under federated learning across distributed data archives, is timely and highly relevant for Earth observation in general, and the paper is well-written and easy to follow. The challenges of the presented topic resonate well with the proposed solutions.

The proposed TERCA aggregation seems to be well designed with scientific rigor, while the theoretical analysis in Appendix D sets a strong base for its soundness, and the corresponding experimental analysis shows its effectiveness.

Critical weaknesses

I have three major concerns regarding the paper.

First, my main concern is about how the paper frames knowledge distillation (KD) under federated learning (FL). Even though the main focus of the paper is FL, it is not clear: i) the motivation for the use of KD in the context of FL in the introduction and related work; and ii) why KD-based local training of CoRD-Fed is claimed to be a contribution to FL. KD is most probably helpful for any kind of centralized general object detection training. But it is not necessarily helpful for FL-specific challenges (non-IID partitioning, parameter aggregation, etc.). This ambiguity is also reflected in the experimental analysis, which struggles to disentangle KD’s impact on FL. In the main results (Table 1), the FL algorithms compared with CoRD-Fed do not employ KD for local training, while distillation-based baselines use naive FedAvg only. Especially for FL algorithms, I would expect to see how they would perform when coupled with KD to fairly compare them with CoRD-Fed. Otherwise, it is unclear whether CoRD-Fed's main gain comes solely from KD's impact on local training. This is also questionable in Table 2, where KD only achieves either the best or the second-highest results on 4 out of 7 metrics. This would be justifiable if KD in CoRD-Fed local training is specifically favorable for FL challenges, e.g., through varying levels of data heterogeneity (different α\alpha values).

Second, in Tables 1, 2, and 3, most of the results are often very close to each other. However, no measure of variability or uncertainty is reported, e.g., standard deviations, confidence intervals, or statistical significance tests. Considering that the results for only one level of data heterogeneity FL partitioning (α\alpha=0.1) are given, this issue becomes more critical to assess the actual effectiveness of CoRD-Fed for FL. The lack of variability or uncertainty measures also amplifies the ambiguity mentioned in the first weakness above.

Third, even though there are plenty of federated learning studies in remote sensing, there is very limited mention of those works in the paper. Considering the focus of this paper, I would expect to see a dedicated subsection in the related work. This limits the proper positioning of this paper in the existing remote sensing FL studies.

Apart from those, there are other relatively minor weaknesses.

Generally, figures are hard to understand due to the crowded placement of items with small and not visible arrows, and the lack of details in the captions.

I appreciate the sensitivity analysis given in the paper. However, different hyperparameters are not decoupled from each other, and it is not specified how out-of-focus hyperparameters are set, especially λch\lambda_{ch} and λrd\lambda_{rd} for Figures 4a and 4b, respectively.

If I am not mistaken, Tables 4 and 5 report results for client-specific test splits, which is different from the main test split used to report results in Tables 1, 2, and 3. This makes the two sets of tables incomparable, limiting the ability to show the effect of FL compared to centralized training.

In Line 140, the section reference is missing even though there is a placeholder.

Questions for authors

Can you please clarify and show, with relevant experiments, the motivation for the use of KD in the context of FL, the impact of KD on CoRD-Fed superiority over the compared FL algorithms, and why KD-based local training of CoRD-Fed needs to be regarded as a contribution to federated object detection for remote sensing imagery?

Can you please provide variability or uncertainty measures of the results?

Can you please clarify the positioning of the paper in the existing line of research for federated learning in remote sensing?

Can you please clarify whether the results in Tables 4-5 and Tables 1-3 are obtained on the same test split or not so that whether they are comparable?

Can you please clarify how the other hyperparameters are set when the sensitivity of the considered hyperparameter is analyzed? This is especially important for λch\lambda_{ch} and λrd\lambda_{rd}, as they are entangled by definition in Eq. (5).

Reviewer_22FoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates federated remote sensing object detection under heterogeneous data conditions and proposes the CoRD-Fed framework. On the client side, the framework employs region-of-interest distillation, which mitigates the interference of background information on knowledge transfer through three mechanisms: teacher-student prediction matching constrained by ground-truth annotations, sample selection driven by confidence discrepancies, and spatial and channel feature alignment guided by Grad-CAM. On the server side, the paper introduces TERCA, which calculates consensus coefficients based on the degree of divergence in client updates for each class; these coefficients are then used—in conjunction with magnitude recovery, truncation, and enhancement—to adjust the updates of the classification head.

Critical strengths

Federated object detection addresses a practical need in remote sensing, where data are held across institutions, category distributions are uneven, and small objects are common. The paper clearly organizes its approach around local representation learning and server-side aggregation.

Annotation-constrained matching and distillation within object-relevant regions offer a reasonable way to reduce background interference. The spatial and channel branches also account for localization and semantic information, respectively.

Critical weaknesses

Equations (10)–(12) on page 6 only multiply the FedAvg update by positive scalars, so the final update always preserves the FedAvg direction. Proposition 2 in Appendix D explicitly establishes this property. TERCA therefore adjusts the category-wise step size according to the degree of cancellation; it neither estimates a new direction nor selectively downweights conflicting clients. The claims that it “filters out destructive interference” and that “conflicting directions are suppressed” go beyond what the equations implement. The contribution should be described more precisely and compared with simple alternatives, such as category-wise adaptive server learning rates.

The AR values in the hyperparameter plots on page 9 are approximately 50%–55%, whereas the main experiment reports an AR₁₀₀ of 59.27% for the full model. This is not necessarily an error, but the authors should define the plotted AR metric, specify the parameters held fixed in each subplot, and clarify whether the model configuration matches the main experiment.

Several editing issues remain: “Sec. ??” appears on page 3, line 140; the Acknowledgments section on page 10 retains template instructions; and “Forzen” in the legend of Figure 1 on page 4 should be “Frozen.” These should be corrected.

Page 6 states that DOTA contains 18 categories, but the legend of Figure 5 on page 14 lists only 11. If the figure shows a subset, the authors should explain how it was selected. Otherwise, the figure should be completed and checked against the categories used in the experiments.

Questions for authors
  1. Since TERCA always preserves the FedAvg update direction, how does it differ from a category-wise adaptive server learning rate? Could the authors include a simple baseline with matched update magnitudes?
  2. Please resolve the inconsistent APₗ values across Tables 1–3 and explain why adding TERCA to RDChannel reduces several performance metrics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.