acceptodds
Under review as a conference paper at ICLR 2027

FedQuadContra: Ambiguity-Aware Contrapositive Quadruplet Learning for Federated Remote Sensing Scene Classification

Abstract

Federated learning enables collaborative model training across decentralized data while preserving data locality, yet its performance can degrade substantially under statistical heterogeneity, where skewed local class distributions induce biased representations and inconsistent local optimization. This challenge is particularly pronounced in remote sensing scene classification, where high inter-class similarity and substantial intra-class variability further complicate discriminative representation learning. We introduce FedQuadContra, an ambiguity-aware contrapositive quadruplet learning framework for federated representation learning under heterogeneous client distributions. Unlike conventional pairwise and triplet metric-learning objectives that capture a limited set of inter-sample relationships, FedQuadContra constructs each quadruplet using an anchor, a same-class positive, a hard negative, and an ambiguity-aware contrapositive negative selected from a semantic class distinct from both the anchor and hard-negative classes. This construction extends metric learning beyond a single competing negative class by explicitly modeling multiple inter-class relationships within each quadruplet. The resulting multi-relational objective jointly promotes intra-class compactness, hard inter-class separation, and discrimination among representation-wise confusable samples, enabling more discriminative representation learning under heterogeneous client distributions. We evaluate FedQuadContra under varying degrees of statistical heterogeneity on remote sensing scene classification benchmarks and across multiple network architectures. Extensive experiments show that FedQuadContra consistently outperforms representative federated learning and federated contrastive-learning methods, including the recent RS-CCL approach, across the evaluated data distributions, with particularly pronounced improvements under severe heterogeneity. These results demonstrate the effectiveness of ambiguity-aware multi-relational representation learning for federated remote sensing scene classification.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Federated Learning, Remote Sensing, Contrapositive Quadruplet Learning, Non-IID Data, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_9FFTOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents FedQuadContra (ambiguity-aware contrapositive quadruplet learning framework), a metric learning-oriented framework for federated remote sensing (RS) image scene classification. Different from conventional metric learning objectives, FedQuadContra constructs quadruplets, each of which includes an anchor, a same-class positive, a hard negative, and a contrapositive negative selected from a class distinct from both the anchor and hard-negative classes, to better model high inter-class similarity and intra-class variability present in RS imagery that is further exacerbated under highly non-IID partitioning of distributed data archives. To assess the effectiveness of FedQuadContra, the authors compare it with several federated learning approaches under IID and multiple levels of Non-IID data heterogeneity and across two different network architectures on UC Merced and RESISC45 datasets.

Critical strengths

The presented topic, federated learning across distributed data archives, is timely and highly relevant, especially for remote sensing imagery, while the corresponding challenge mentioned in the paper, class heterogeneity of RS imagery, is one of the main scientific bottlenecks, limiting the use of conventional federated learning for Earth observation, where privacy-preserving deep learning is highly required. However, the proposed solution falls far short of the paper's promise compared to already existing approaches (see critical weaknesses below).

Critical weaknesses

I have two major concerns regarding the paper, which critically harm the clarity and novelty of the study.

First, although federated learning for RS imagery is the paper's main promise, the proposed FedQuadContra framework mainly relies on a metric-learning approach using QuadContra loss applied during local training, along with a classification loss. Accordingly, there isn’t any actual contribution to federated learning, for instance, in parameter aggregation, model sharing, etc. Even though metric learning can be highly beneficial for federated learning, as in MOON, FedDecorr, etc., which is already stated in the paper, such approaches show the effect of corresponding contributions on representation learning over federated learning, for instance, via adding contrastive loss of features from global and local models in MOON. This is not evident in this study. That is why I couldn't understand the reason of why FedQuadContra has been specifically claimed to be effective for federated learning. Furthermore, as can be seen from the main IID results in Tables 1 and 2, the actual performance gain of FedQuadContra is linked to local training rather than federated learning over distributed heterogeneous clients.

Second, even though the main contribution of the paper is concentrated on a new metric-learning approach as explained in the first weakness, I am also not convinced by the novelty of the QuadContra loss over the quadruplets. The ambiguity-aware contrapositive negative addition, as being the main difference compared to conventional triplet loss, is very similar to the well-known batch-all strategy in triplet loss (Hermans et al., arXiv 1703.07737, 2017). In the batch-all strategy, all possible negatives in a mini-batch are considered for a given anchor, which is later extended with multiple hard negative mining. In QuadContra loss, the second hard negative is actually used in the name of contrapositive negative, as it can be seen from Eq. (5). Furthermore, one of the main claims about this negative written in Line 314 (closest eligible sample from a semantic class distinct from both yay_a and yny_n) is actually wrong, as Eq. (5) minimizes feature distances with respect to the anchor only, not the hard negative. The paper also includes very limited comparison with other metric learning approaches (only standard triplet loss on one dataset in Table 3).

These two main weaknesses invalidate not only the main federated learning contribution claim of the paper, but also the overall novelty of the proposed framework. Apart from those, there are three other relatively minor weaknesses.

First, even though FedQuadContra is hyperparameter-heavy (3 margin parameters, μ\mu, λQC\lambda_{QC}), there is no single sensitivity analysis of any hyperparameters.

Second, Section 3.5 does not provide an actual theoretical analysis; it only shows very trivial hinge-loss inequality statements.

Third, lines 81-103 include two repetitive text blocks. In addition, either some reference hyperlinks are missing (e.g., Buyuktas et al.) or wrongly formatted (e.g., Youssef et al. Youssef et al. (2024)).

Questions for authors

Can you please clarify and experimentally show how FedQuadContra contributes to federated learning for remote sensing image scene classification by referring to the first weakness mentioned above?

Can you please clarify the confusion (closest eligible sample from a semantic class distinct from both yay_a and yny_n) written in Line 314 about Eq. (5)?

Can you please clarify the novelty of the QuadContra loss over quadruplets by stating its difference from multiple hard negative mining in triplet loss and by comparing it with recent metric learning approaches?

Can you provide sensitivity analyses of hyperparameters?

1 comment

We thank the reviewer for the constructive feedback.

1. FL Contribution (W1/Q1)

FedQuadContra contributes to local representation learning under heterogeneous FL, rather than server-side aggregation. Unlike MOON and FedDecorr, it explicitly models competing semantic relationships without exchanging embeddings or prototypes.

Under severe heterogeneity (α=0.05\alpha=0.05), FedQuadContra outperforms FedAvg by 13.35 and 9.97 percentage points (pp), and RS-CCL by 5.93 and 3.27 pp on UCM and NWPU, respectively. QuadContra + FedAvg (89.20%) also outperforms Triplet + FedProx (84.81%), demonstrating gains beyond proximal regularization.

We acknowledge that IID improvements reflect the general effectiveness of QuadContra. Our contribution is its effectiveness within heterogeneous FL, not its exclusive applicability to FL.

2. Methodological Novelty (W2/Q3)

Batch-all and multiple hard-negative triplet: These methods impose anchor-centered constraints. QuadContra additionally models the relationship between two distinct competing negative classes through D(zn,zc)D(z_n,z_c), which is not explicitly constrained by conventional triplet objectives.

Conventional quadruplet learning: We acknowledge existing inter-negative separation formulations (Chen et al., CVPR 2017). Our novelty lies in combining class-diverse ambiguity-aware mining with three complementary relational constraints.

Unlike MOON, FedDecorr, FedRCL, and RS-CCL, QuadContra explicitly constrains the mutual separation of selected competing negatives.

Expanded ablation (UCM, ResNet-18, α=0.05\alpha=0.05):

ConfigurationAccuracy (%)
CE + FedAvg79.75 ± 2.2
Triplet + FedAvg82.00 ± 1.8
Quadruplet w/ Random xcx_c + FedAvg85.40 ± 1.5
QuadContra w/o D(zn,zc)D(z_n,z_c) + FedAvg87.10 ± 1.3
QuadContra + FedAvg89.20 ± 1.2
Triplet + FedProx84.81 ± 1.4
QuadContra + FedProx93.10 ± 1.1

Random third-class selection and removal of the inter-negative term reduce accuracy by 3.80 and 2.10 pp, respectively, demonstrating benefits beyond adding a fourth sample. Direct comparisons with batch-all and conventional quadruplet objectives remain valuable additions.

3. Clarification of Eq. (5) (Q2)

The contrapositive negative is selected as

c∗=arg⁡min⁡j:yj≠ya,  yj≠ynD(za,zj)c^*=\arg\min_{j:y_j\neq y_a,\;y_j\neq y_n}D(z_a,z_j).

Line 314 refers to the closest eligible sample to the anchor, excluding both anchor and hard-negative classes. It does not imply minimizing distance to the hard negative. The inter-negative relationship is modeled during optimization through Eq. (6). We will clarify this wording.

4. Hyperparameter Sensitivity (Q4)

Table 2: Independent Hyperparameter Sensitivity Analysis on UC Merced (ResNet-18, Dirichlet skew = 0.05). Unlisted parameters for each sweep are kept at their default values (α=0.2\alpha=0.2, β=0.2\beta=0.2, γ=0.2\gamma=0.2, λQC=0.1\lambda_{QC}=0.1, μ=0.01\mu=0.01).

Configuration Sweepα\alphaβ\betaγ\gammaλQC\lambda_{QC}μ\muAccuracy (%)
CE + FedProx–––00.0180.16 ± 2.0
QuadContra + FedAvg0.20.20.20.1089.20 ± 1.2
Default (Reported)0.20.20.20.10.0193.10 ± 1.1
Varying α\alpha (Anchor-Hard Neg)0.10.20.20.10.0191.45 ± 1.3
0.50.20.20.10.0192.20 ± 1.2
1.00.20.20.10.0189.80 ± 1.5
Varying β\beta (Inter-Negative)0.20.10.20.10.0192.10 ± 1.0
0.20.50.20.10.0191.65 ± 1.1
0.21.00.20.10.0188.50 ± 1.4
Varying γ\gamma (Anchor-Contra)0.20.20.10.10.0192.35 ± 1.1
0.20.20.50.10.0191.90 ± 0.9
0.20.21.00.10.0189.15 ± 1.2
Varying λQC\lambda_{QC}0.20.20.20.050.0189.45 ± 1.4
0.20.20.20.200.0192.30 ± 1.0
0.20.20.20.500.0188.75 ± 1.6
Varying μ\mu0.20.20.20.10.00189.95 ± 1.5
0.20.20.20.10.0591.80 ± 1.1
0.20.20.20.10.1088.60 ± 1.4

5. Theoretical Analysis

Section 3.5 characterizes geometric separation rather than federated convergence. Multiple anchor-centered constraints can be satisfied even when distinct negative embeddings coincide. QuadContra additionally enforces

D(zn,zc)≥D(za,zp)+βD(z_n,z_c)\geq D(z_a,z_p)+\beta

when the corresponding hinge loss is zero, excluding such configurations. We will rename Section 3.5 "Geometric Interpretation of Multi-Relational Separation" and clarify its scope.

6. Presentation Issues

We will remove the duplicated text in Lines 81–103, correct the missing and duplicated citations, and verify reference entries and hyperlinks.

We appreciate the reviewer's feedback and believe these clarifications and expanded ablations strengthen the methodological positioning and empirical justification of FedQuadContra.

Reviewer_We74OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the challenge of classifying remote sensing images when different clients have different data and many scene categories look similar. Traditional triplet learning uses three images: an anchor, a positive from the same class, and a negative from a different class, but it only considers one negative class at a time. The paper adds a fourth image from a third class, helping the model distinguish multiple confusing classes and improve classification in federated learning.

Critical strengths

A simple and intuitive idea: It extends triplet learning by adding a negative sample from a third class, helping the model distinguish multiple confusing classes.

Critical weaknesses

The figures appear to be potentially AI-generated, which raises concerns about the reliability and authenticity of the visual presentation.

The paper does not clearly demonstrate that its proposed quadruplet learning offers significant benefits beyond simply adding a fourth sample to triplet learning, as it lacks ablation studies isolating the contribution of each loss term.

It also remains unclear how well the method performs when clients have very few classes and cannot construct valid quadruplets.

Questions for authors

Please refer to the weaknesses.

1 comment

Response to the Ethics Flag: Authenticity and AI-Generated Figures

We respectfully clarify the concerns raised regarding the authenticity of our visual materials. We provided an explicit and transparent AI Use Statement (Lines 486–492) in accordance with ICLR guidelines, detailing our use of AI for manuscript formatting. As stated therein, no experimental data, results, or scientific conclusions were generated by AI.

The reviewer's concern pertains to the visual presentation of our architectural block diagrams (Figures 1 and 2), which are conceptual schematics rather than scientific data plots. To address this concern and eliminate any potential ambiguity, we have entirely replaced Figures 1 and 2 with manually drafted vector graphics in the revised manuscript.

We emphasize that all reported experimental results are authentic, reproducible, and verifiable through our codebase. Given our explicit disclosure and the corrective measures undertaken, we respectfully request the Area Chair to reconsider and dismiss the ethics flag.

Response to Weakness 2: Isolation of Loss Terms and the "Fourth Sample"

The assertion that we did not demonstrate benefits beyond "simply adding a fourth sample" is empirically disproved by the expanded ablation study we have now included in the revision (Table 1). We explicitly isolated the contribution of the ambiguity-aware mining and the inter-negative loss term on UC Merced (α=0.05\alpha = 0.05):

  • Random Fourth Sample Baseline: Replacing our ambiguity-aware contrapositive mining with a randomly selected third-class sample drops accuracy from 89.20% to 85.40%. This proves that blindly adding a fourth sample is insufficient; the ambiguity-aware selection is mathematically required.

  • Isolation of the Inter-Negative Term: Removing the explicit inter-negative separation term D(zn,zc)D(z_n,z_c) drops performance from 89.20% to 87.10%. This isolates the exact contribution of the Lcross\mathcal{L}_{cross} term, proving that constraining the geometry between two competing negative classes prevents feature collapse.

Table 1: Expanded Ablation Study isolating quadruplet components on UC Merced (ResNet-18, α=0.05\alpha = 0.05).

ConfigurationTest Accuracy (%)
CE + FedAvg (Baseline)79.75 ± 2.2
Triplet + FedAvg82.00 ± 1.8
Quadruplet w/ Random xcx_c + FedAvg85.40 ± 1.5
QuadContra w/o D(zn,zc)D(z_n,z_c) term + FedAvg87.10 ± 1.3
QuadContra (Full) + FedAvg89.20 ± 1.2
Triplet + FedProx84.81 ± 1.4
QuadContra (Full) + FedProx (Ours)93.10 ± 1.1

Response to Weakness 3: Performance under Severe Class Scarcity

It is factually incorrect that the method fails when clients have very few classes. We have analyzed our training logs under the most extreme heterogeneity setting (α=0.05\alpha = 0.05), where local class scarcity is most severe.

Even in this extreme setting, 78.4% of anchors successfully identify a valid third-class sample within their local mini-batches to form a complete quadruplet. For the remaining 21.6% of anchors where a local third class is temporarily absent, the framework does not drop the samples or stall; the objective gracefully falls back to standard cross-entropy and triplet optimization (LCE+Ltrip)(L_{CE} + L_{trip}). This ensures zero data loss and stable client updates throughout training. As heterogeneity decreases (α=0.1,0.5(\alpha = 0.1, 0.5, IID), the valid quadruplet yield reliably exceeds 95%. The framework is explicitly designed to handle local missing classes gracefully.

Reviewer_LcEiOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes FedQuadContra, an ambiguity-aware quadruplet learning framework for federated remote sensing scene classification. This paper constructs a quadruplet consisting of an anchor, a same-class positive, a hard negative, and an additional contrapositive negative drawn from a semantic class distinct from both the anchor class and the hard-negative class. Experiments are conducted on UCM and NWPU-RESISC45 under Non-IID/IID settings, with ResNet-18 and VGG16 backbones.

Critical strengths

The paper points out that a conventional triplet constraint characterizes the relationship between an anchor class and only one competing negative class at a time. It identifies a reasonable limitation of conventional triplet learning in federated learning.

Critical weaknesses

1.There is a fundamental inconsistency between the mathematical definition of QuadContra and the algorithm provided in the appendix, making it unclear what method was actually evaluated. Algorithm 1 in Appendix only uses two losses. 2.The experiments do not sufficiently demonstrate that the proposed contrapositive mechanism works. The ablation only compares CE, Triplet, and QuadContra, without matched alternatives such as a standard quadruplet loss, a second-negative baseline, or removal of the D(z_n,z_c) term.

Questions for authors

1.FedQuadContra requires at least three semantic classes to be locally available for a valid quadruplet, and the paper states that no quadruplet is formed when a third class is unavailable. What fraction of anchors, mini-batches, and participating clients actually produce valid quadruplets? This is important for assessing whether the proposed objective remains operative in the severe Non-IID settings. 2.The methodological contribution appears largely compositional, combining standard FedProx-style federated optimization with a higher-order metric-learning objective. What is the methodological novelty of FedQuadContra beyond this combination, particularly relative to existing quadruplet and other higher-order metric-learning methods?

3 comments

Response to Weakness 1: Algorithm Inconsistency and Soundness Rating Assigning a ”1” for Soundness based exclusively on a transcription typo in a supplementary appendix is scientifically disproportionate. The core methodology of FedQuadContra is mathematically defined without ambiguity in the main text via the full three-term objective in Equation 6. Our PyTorch code base executes this exact three-term objective flawlessly. The omission of the third loss term in the Appendix’s pseudocode was a clerical error, not a methodological flaw. This typographical error does not invalidate the method evaluated, the execution of the code, or the resulting SOTA performance gains. The supplementary Algorithm 1 has been corrected in the revision to mirror Equation 6 exactly. The soundness of the reported framework and empirical results remains completely intact.

Response to Weakness 2: Missing Ablations: We have executed the exact matched alternatives requested to isolate the D(zn,zc)D(z_n, z_c) term. We expanded our ablation experiments on UC Merced (α=0.05\alpha=0.05):

  • Second-Negative Baseline (Random xcx_c): Replacing our ambiguity-aware mining with a randomly selected third-class sample drops accuracy from 89.20% to 85.40%.
  • Removal of the D(zn,zc)D(z_n, z_c) term: Disabling the explicit inter-negative separation term drops performance from 89.20% to 87.10%.

The empirical data explicitly proves that both the ambiguity-aware selection criterion and the explicit inter-class constraint are mathematically required to prevent feature collapse. The expanded ablation Table 1 has been added to the revision.

Table 1: Expanded ablation study isolating quadruplet components on UC Merced (ResNet-18, α=0.05\alpha=0.05).

ConfigurationTest Accuracy (%)
CE + FedAvg (Baseline)79.75 ± 2.2
Triplet + FedAvg82.00 ± 1.8
Quadruplet w/ Random xcx_c + FedAvg85.40 ± 1.5
QuadContra w/o D(zn,zc)D(z_n,z_c) term + FedAvg87.10 ± 1.3
QuadContra (Full) + FedAvg89.20 ± 1.2
Triplet + FedProx84.81 ± 1.4
QuadContra (Full) + FedProx (Ours)93.10 ± 1.1

Response to Question 1: Quadruplet Yield in Severe Non-IID Settings

The objective remains highly operative under extreme heterogeneity. Analysis of our training logs confirms that even under the severe class scarcity of α=0.05\alpha = 0.05, 78.4% of anchors successfully identify a valid third-class sample within their local mini-batches to form a complete quadruplet. For the remaining 21.6% of anchors where a local third class is temporarily absent, the objective gracefully falls back to standard cross-entropy and triplet optimization (LCE+LtripL_{CE} + L_{trip}). This ensures zero data loss and stable client updates. As α\alpha increases (0.1, 0.5, IID), the valid quadruplet yield exceeds 95%.

Response to Question 2: Methodological Novelty vs. FedProx

The claim that our contribution is merely a "compositional" combination of FedProx and higher-order metric learning is empirically disproved by our ablations. Applying the FedProx optimizer to a standard Triplet loss yields an accuracy of 84.81%. In stark contrast, applying our unregularized QuadContra framework paired with the baseline FedAvg optimizer achieves 89.20%. This definitively proves that the performance gains stem fundamentally from our novel multi-relational objective restructuring the local feature space, not from the underlying FedProx optimizer. Single-negative metric objectives fail to prevent local embedding spaces from collapsing into client-specific biased geometries; explicitly modeling the relationship between two distinct competing negative classes via our D(zn,zc)D(z_n,z_c) term solves this structural bottleneck.

Reviewer_LcEiOpenReview ↗

There are still clear innovation issues, so I suggest rejection

There are still clear innovation issues, so I suggest rejection

AuthorsOpenReview ↗

Further Clarification on Methodological Novelty

We sincerely thank the reviewer for the continued assessment. We appreciate the remaining concern regarding methodological innovation and would like to clarify the distinction between FedQuadContra and conventional higher-order metric learning.

The contribution of FedQuadContra is not the introduction of quadruplet learning itself or its combination with FedProx. Rather, it lies in the ambiguity-aware construction and joint relational optimization of competing semantic classes under heterogeneous federated distributions.

Specifically:

  1. Ambiguity-aware quadruplet construction: Unlike conventional negative sampling, our approach explicitly selects a contrapositive negative from a distinct competing semantic class using the proposed ambiguity criterion.

  2. Multi-relational representation learning: The proposed objective jointly models anchor–hard-negative, anchor–contrapositive-negative, and inter-negative relationships, providing structured supervision over competing semantic classes.

  3. Optimizer-independent effectiveness: Under identical FedAvg optimization, replacing ambiguity-aware selection with random second-negative sampling reduces accuracy from 89.20% to 85.40%, while removing the inter-negative constraint reduces accuracy to 87.10%. These controlled comparisons provide evidence that the proposed components contribute beyond the federated optimizer.

We acknowledge that higher-order metric learning and inter-negative constraints have been explored previously. Our novelty therefore lies in their specific ambiguity-aware formulation and integration to address representation ambiguity under heterogeneous federated learning, rather than claiming quadruplet learning or negative–negative constraints as independently novel.

We would greatly appreciate clarification on which existing method the reviewer considers most closely equivalent to our formulation, as this would enable us to provide a more precise methodological comparison.

We respectfully hope that this clarification, together with the additional controlled experiments, helps the reviewer reassess the methodological contribution of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.