- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies domain adaptation under sparse target calibration, where labeled source data cover all condition levels, but labeled target data are available only at a subset of calibration levels. The goal is to achieve accurate predictions across the full target condition range, including levels that are not observed during adaptation. The authors formulate the problem from a distributionally robust optimization perspective and derive a theoretical upper bound on the target risk. Based on this bound, they propose CaDRA, which combines coverage-weighted target regression loss with conditional source–target alignment. The proposed method is evaluated on synthetic experiments and three real-world domain adaptation tasks.
Critical strengths
The problem setup is clearly defined and reasonably motivated. In particular, the distinction between observed and unseen target calibration levels is explicit, making the underlying research problem easy to understand.
Critical weaknesses
W1. Limited Methodological Novelty
The proposed training objective appears somewhat conventional, offering mainly incremental methodological contributions. It combines a weighted target calibration loss with source–target alignment, both of which build on established approaches in domain adaptation [1–3]. In particular, Naram et al. [2] derive weighted empirical source losses and a partial Wasserstein alignment term from theoretical bounds. Although the problem settings and weighting schemes differ, the overall bound-driven construction follows a similar approach.
A key methodological distinction of CaDRA lies in its coverage-aware weights and . However, their contribution is not sufficiently validated. A more detailed analysis, including an ablation against uniform weighting while keeping the remaining objective unchanged, is needed to establish their effectiveness.
W2. Insufficient Justification for the Source-Centered DRO Formulation
The paper constructs ambiguity sets centered at the source conditional distributions. However, the proposed DRO formulation does not appear to sufficiently justify the resulting training objective.
Specifically, under the assumed target-side smoothness condition, an alternative formulation would be to center the ambiguity set at the adapted target distribution of the nearest observed calibration level:
Q : W_1 \left(Q,Q^\phi_{a(u)}\right) \leq L_t |u-a(u)| \right\\}$$ Under the target-side smoothness assumption, the true unseen target distribution lies within this ambiguity set. By the triangle inequality, this set is also contained within the proposed source-centered ambiguity set, potentially yielding a tighter robust risk bound. This raises the question of whether the additional source–target alignment term arises primarily from the choice of source-centered ambiguity sets, rather than being inherently necessary under the stated assumptions. Empirically, Table 2 shows only a marginal improvement from Calibration Only 1.670 $\pm$ 0.026 to CaDRA 1.660 $\pm$ 0.023. The authors should provide a clearer theoretical justification for choosing the source-centered formulation over this target-centered alternative, along with stronger empirical evidence demonstrating the additional benefits of the resulting alignment term. **W3. Unclear Experimental Protocol and Baseline Implementation** Several aspects of the experimental protocol remain unclear. Appendix H explicitly mentions the use of target regression loss for CORAL, but does not specify whether the other baselines incorporate the available target labels into their training objectives. This is particularly important because how the labeled data from observed target calibration levels are utilized can substantially affect adaptation performance. The hyperparameter selection procedures also appear asymmetric. While CaDRA's alignment coefficient is selected using validation data, those for CORAL, DANN, and RSD are fixed across experiments, raising concerns about whether comparable tuning was performed. Furthermore, the validation metric used for early stopping and whether the same criteria were consistently applied across baselines remain unclear. Training details are provided primarily for MotorDA, with comparable configurations and early-stopping procedures insufficiently specified for CWRU and Paderborn. The baseline alignment procedures require further clarification. In particular, for DANN and other baselines involving marginal alignment, it is unclear whether source samples from all condition levels were aligned with target samples restricted to the observed calibration levels, or whether the source samples were similarly restricted. Such a configuration could introduce label-distribution mismatch and adversely affect methods such as DANN. Notably, all four baselines underperform Source Only in the Paderborn poor-coverage setting, where only a single target calibration level is available (Figure 3). The authors should provide sufficient implementation details to ensure reproducibility and allow a fair assessment of the baseline comparisons. Without these details, it is difficult to determine whether the reported performance differences reflect genuine methodological advantages or differences in implementation choices. **W4. Insufficient Comparison with Relevant Baselines** The experiments lack several important baselines for the proposed setting. Given the availability of labeled source data and labeled target data at observed calibration levels, joint source–target ERM is a natural supervised baseline. For example, a shared predictor could be trained by minimizing $$\min_{\theta} (1-\eta)\widehat R_s(f_\theta) +\eta\widehat R_{t,\mathcal A}(f_\theta), \qquad \eta\in[0,1],$$ where $\widehat R_{t,\mathcal A}$ is the empirical target regression loss restricted to observed calibration levels. Both the jointly trained predictor and a variant with an additional calibration-only target adapter could be evaluated. These comparisons would help determine whether CaDRA provides benefits beyond straightforward supervised learning with the available labeled data. Supervised and few-shot domain adaptation methods for regression [4, 5], as well as approaches addressing incomplete target-domain coverage [6], are also relevant. Although the latter differ in their target-data assumptions, the authors could discuss whether such methods can be adapted to the proposed setting. The study primarily considers existing alignment-based methods, including CORAL, DANN, RSD, and COD. Broader comparisons would strengthen the empirical evidence supporting CaDRA. Furthermore, the paper reports only overall target MAE. Reporting performance separately for seen and unseen target levels would help distinguish adaptation to observed conditions from generalization to unobserved ones, which is the central goal of this work. **Minor Comments** 1. (Incomplete Related Work) The Related Work section discusses domain adaptation, sparse supervised adaptation, and DRO largely in isolation, without sufficiently addressing research at their intersections. This may give the impression that applying DRO to domain adaptation is itself a novel contribution, although distributionally robust domain adaptation has already been studied [7, 8]. Furthermore, prior work has investigated domain adaptation under incomplete target-domain coverage, such as Li et al. [6]. Although their problem setting and target-data assumptions differ from those of CaDRA, these connections deserve further discussion. I recommend that the authors discuss and cite these relevant studies to better clarify the novelty and positioning of their work. 2. (Ambiguous Axis Labeling in Figure 1) In Figures 1(c) and 1(d), the y-axes are labeled as "Target shift $\delta(c)$." However, $\delta(c)$ is defined as a vector-valued quantity, whereas the plots display its individual scalar components, $\delta_1(c)$ and $\delta_2(c)$. Why is the y-axis labeled with the vector $\delta(c)$ rather than explicitly indicating that scalar components are being plotted? I recommend revising the axis labels to avoid this ambiguity. 3. (Unclear Experimental Settings and Presentation in Figure 3) Figure 3 presents one of the main experimental results, yet its settings are described only as "good coverage" and "poor coverage." These are author-defined rather than standardized evaluation categories, but no precise definition is provided in the main text or figure caption. This is particularly problematic because the calibration settings directly affect the difficulty of the adaptation task. For instance, the Paderborn poor-coverage setting provides labeled target data at only a single level, which is essential for interpreting the substantial performance degradation of the baselines. Should readers have to search through the appendix simply to understand the experimental conditions underlying such a central result? This lack of clarity undermines the interpretability and readability of Figure 3. Moreover, the color scheme appears unnecessarily complex, and the figure omits standard deviations or other measures of variability despite having sufficient space. I recommend explicitly providing the corresponding calibration sets and coverage values in the main text or figure caption, simplifying the visual presentation, and reporting the variability of the results. Overall, given the limited methodological novelty, insufficient justification for the proposed DRO formulation, and concerns about the experimental comparisons, I recommend a clear rejection of this paper in its current form. --- **References** [1] Saeid Motiian, Marco Piccirilli, Donald A. Adjeroh, and Gianfranco Doretto. Unified Deep Supervised Domain Adaptation and Generalization. ICCV, 2017. [2] Jayadev Naram, Fredrik Hellström, Ziming Wang, Rebecka Jörnsten, and Giuseppe Durisi. Theoretical Performance Guarantees for Partial Domain Adaptation via Partial Optimal Transport. ICML, 2025. [3] Okan Koç, Alexander Soen, Shanglin Li, and Masashi Sugiyama. Incorporating Importance Weighting in Optimal Transport Based Domain Alignment. ICML, 2026. [4] Takeshi Teshima, Issei Sato, and Masashi Sugiyama. Few-shot Domain Adaptation by Causal Mechanism Transfer. ICML, 2020. [5] Ori Katz, Ronen Talmon, and Uri Shaham. Supervised Domain Adaptation Based on Marginal and Conditional Distributions Alignment. TMLR, 2024. [6] Xiang Li, Wei Zhang, Xu Li, and Hongshen Hao. Partial Domain Adaptation in Remaining Useful Life Prediction With Incomplete Target Data. IEEE/ASME Transactions on Mechatronics, 29(3):1903–1913, 2024. [7] Akram S. Awad and George K. Atia. Distributionally Robust Domain Adaptation. MLSP, 2023. [8] Seonghwi Kim, Sungho Jo, Wooseok Ha, and Minwoo Chae. Distributionally Robust Classification for Multi-source Unsupervised Domain Adaptation. ICLR, 2026.Questions for authors
Please see the weaknesses.
3 comments
W4. Additional baselines and observed-unseen results
We added both requested supervised baselines under the W3 protocol:
- Calibration Only minimizes (trains the target adapter using only observed target labels).
- Joint ERM minimizes (uses observed target labels and labeled source samples, without alignment loss).
Following Reviewer fwWv's suggestion to use source labels at levels missing from target calibration data, we revised CaDRA to Joint-CaDRA: which adds the original CaDRA alignment loss to Joint ERM. We also evaluated DARE-GRAM, designed for continuous-output domain adaptation.
| Setting | Source Only | Calibration Only | Joint ERM | DARE-GRAM | Joint-CaDRA |
|---|---|---|---|---|---|
| MotorDA-good | 2.7209 | 1.6837 ± 0.0141 | 1.9421 ± 0.0305 | 1.6660 ± 0.0260 | 1.6907 ± 0.0127 |
| MotorDA-poor | 2.7209 | 2.3772 ± 0.0645 | 2.2901 ± 0.0046 | 2.3790 ± 0.0380 | 2.3178 ± 0.0234 |
| Paderborn-good | 0.5828 | 0.2237 ± 0.0128 | 0.2739 ± 0.0010 | 0.2330 ± 0.0020 | 0.1909 ± 0.0145 |
| Paderborn-poor | 0.5828 | 0.5393 ± 0.0899 | 0.3051 ± 0.0057 | 0.4090 ± 0.0540 | 0.2600 ± 0.0195 |
| CWRU-good | 6.5042 | 2.0568 ± 0.0612 | 1.4519 ± 0.0070 | 1.4710 ± 0.0630 | 1.4117 ± 0.0502 |
| CWRU-poor | 6.5042 | 5.5821 ± 0.0128 | 5.2057 ± 0.0091 | 5.3150 ± 0.1260 | 5.0887 ± 0.0179 |
Joint-CaDRA outperforms Calibration Only, Joint ERM, and DARE-GRAM in 5/6 comparisons each, achieving the lowest overall MAE in 4 settings and second-lowest in the other 2.
Below we separate performance on observed vs. unseen target levels:
| Setting | Method | Observed MAE | Unseen MAE | Overall MAE |
|---|---|---|---|---|
| MotorDA-good | Source Only | 3.457 | 1.616 | 2.721 |
| MotorDA-good | Joint-CaDRA | 1.393 | 2.138 | 1.691 |
| MotorDA-poor | Source Only | 2.791 | 2.615 | 2.721 |
| MotorDA-poor | Joint-CaDRA | 1.308 | 3.832 | 2.318 |
| Paderborn-good | Source Only | 0.641 | 0.467 | 0.583 |
| Paderborn-good | Joint-CaDRA | 0.070 | 0.434 | 0.191 |
| Paderborn-poor | Source Only | 1.087 | 0.331 | 0.583 |
| Paderborn-poor | Joint-CaDRA | 0.024 | 0.378 | 0.260 |
| CWRU-good | Source Only | 7.505 | 3.503 | 6.504 |
| CWRU-good | Joint-CaDRA | 0.349 | 4.600 | 1.412 |
| CWRU-poor | Source Only | 2.362 | 10.646 | 6.504 |
| CWRU-poor | Joint-CaDRA | 0.342 | 9.836 | 5.089 |
Joint-CaDRA reduces unseen MAE on Paderborn-good and CWRU-poor, while negative transfer remains in the other four settings.
We added Causal Mechanism Transfer [1] and partial domain adaptation with incomplete target data [2] to Related Work. [1] assumes multiple labeled source domains to learn a shared generative mechanism, whereas our setting starts from a single fixed pretrained source predictor without retraining it. [2] studies unlabeled partial degradation trajectories for remaining useful life prediction, whereas we use a few labeled target levels and evaluate on unseen levels. Because [1, 2] require different data and training access, we directly evaluated Calibration Only, Joint ERM, and DARE-GRAM under the same frozen-source, target-calibration protocol.
[1] T. Teshima et al., "Few-shot Domain Adaptation by Causal Mechanism Transfer," ICML, 2020. [2] X. Li et al., "Partial Domain Adaptation in Remaining Useful Life Prediction With Incomplete Target Data," IEEE/ASME Trans. Mechatronics, 2024.
Minor comments and presentation corrections
We addressed all minor comments directly:
- Defined the ground metric as the unsquared Euclidean norm, making in Eq. (10) explicit.
- Added the missing space before .
- Changed Fig. 3's y-axis label from "Improvement (%)" to "Reduction in Target MAE (%)".
- Replaced the capitalized coupling symbol in Appendix E with a lowercase symbol.
- Added a discussion of negative transfer under poor target coverage.
- Relabeled the scalar shift components in Fig. 1(c,d).
- Added observed anchor sets, unseen sets, and numerical coverage values to the main text.
- Replaced percentage-only results with absolute observed, unseen, and overall MAE values and standard deviations.
- Simplified figure colors and standardized method names.
- Added the Paderborn bearing-to-level mapping and complete CWRU/Paderborn architectures.
- Specified the optimizer, batch construction, search grids, validation metric, and checkpoint rule for all methods.
- Replaced "paired windows" with "independently extracted windows from each domain" (no sample-level correspondence).
I thank the authors for their efforts in revising the manuscript and providing additional experiments. Nevertheless, I believe that several substantial concerns remain unresolved. More generally, I find that the response does not directly address several of the specific points raised in my original review.
First of all, the authors introduce a revised training objective, Joint-CaDRA, without providing a sufficiently clear definition, motivation, or justification in their response. Instead, they refer me to their response to another reviewer. I do not think it is reasonable to expect a reviewer to locate and interpret another reviewer's response simply to understand a major methodological change that is presented as part of the rebuttal to their own concerns.
Methodological Novelty. The authors reiterate the motivation for their problem setting and provide additional experiments, but do not directly explain the methodological differences from the prior works identified in my review. Moreover, the ablation only partially evaluates the proposed coverage-aware weights. My concern regarding methodological novelty therefore remains unresolved.
Source-Centered DRO Formulation. The response does not address my mathematical argument that the target-centered ambiguity set remains valid under sparse calibration and is contained within the proposed source-centered set. The availability of level-specific source distributions does not establish the theoretical justification of the source-centered formulation or its alignment term.
Minor Comments. Although some references have been added, the authors do not provide substantive comparisons with relevant prior work, particularly existing DRO-based domain adaptation studies, as requested. Such comparisons are important for assessing methodological novelty. Other minor comments also remain insufficiently addressed.
I therefore maintain my original assessment.
We thank the reviewer for the constructive feedback.
W1. Goal and methodological novelty
We first identified a practical challenge in industrial condition monitoring. A regression model trained on one machine often needs to be adapted to a new machine, while labeled target data can be collected at only a few operating or severity levels. As a result, part of the target range remains unseen during adaptation. We formalize this setting as sparse target-level calibration.
Based on this problem formulation, we propose CaDRA. We show why sparse calibration data alone cannot determine the target distributions at unseen levels without assumptions across levels. We then derive coverage-aware alignment weights from the observed target levels so that the adaptation objective accounts for the parts of the target range that lack calibration data.
Following Reviewer fwWv’s suggestion to use labeled source samples at levels missing from the target calibration data, we revised the objective to Joint-CaDRA, as defined in our response to Reviewer fwWv, Q2. We then evaluated the coverage-aware weighting within this revised objective. The model, data split, target supervision, and selected loss coefficients are fixed. Only the weighting used in Lalign is changed.
Coverage weighting achieves lower overall MAE in four of the six settings and the same MAE in one setting. This ablation directly evaluates the coverage-aware component of the method.
W2. Why the formulation is source-centered
A target-centered ambiguity set is valid when the observed target levels provide sufficient coverage. Under sparse calibration, however, the target domain may contain only one observed anchor. In that case, every unseen target level would use the same target distribution as its reference. The formulation would therefore concentrate all unseen levels around one observed point and provide no level-specific reference for the rest of the target range.
The source domain is fully labeled. Each unseen target level can therefore use the source conditional distribution at the corresponding label as a separate reference. This preserves information across the full label range instead of forcing all unseen levels to depend on a single target anchor. The source-centered formulation is motivated by this practical difference in coverage. We clarified that it is a modeling choice rather than a theoretical necessity, and that a target-centered formulation is a reasonable alternative when sufficient target anchors are available.
W3. Controlled experimental protocol
We reran all methods under the same experimental protocol. Every adaptable method uses the same labeled target calibration samples, target regression loss, frozen source predictor, target-adapter architecture, data split, validation metric, checkpoint rule, and three random seeds. Each adaptation coefficient is selected through a grid-search sweep using validation MAE. No target test label is used for hyperparameter selection or checkpoint selection.
For every alignment-based method, the source samples used in the alignment term are restricted to the observed target labels. CORAL, DANN, RSD, COD, and Joint-CaDRA therefore use the same source samples for alignment.
The table reports the rerun results.
Joint-CaDRA achieves the lowest overall MAE in five of the six settings and remains close to the best result on MotorDA-good.