acceptodds
Under review as a conference paper at ICLR 2027

Abduction-Deduction Entanglement: Domain Adaptation via Representation Transplants

Abstract

In domain adaptation, unobserved confounding can make prediction rules change across populations even when the response mechanism remains invariant. We show how source data can nevertheless constrain target prediction without identifying the underlying mechanisms. In each domain, the optimal prediction factorizes into abduction, which infers a distribution over unobserved variables from the input, and deduction, which predicts the response given both. Although these maps may not be identifiable, they must jointly reproduce the source predictions. When deduction is shared across domains, this abduction–deduction entanglement allows the sources to restrict the optimal target prediction rule. We assume that objects of abduction are linearly expressed given a pretrained representation, and under further support assumptions, we implicitly parametrize the source and target causal models, and enforce constraints on them via representation transplants – a linear transformation over the representation space that swaps the abduction component between the domains while preserving the deduction content of the representation. Unlabeled target data adds additional restrictions by informing the support of the target representations. We use this parametrization in Causal Robust Optimization (CRO), where a learner minimizes worst-case loss on transplanted data from different sources against an adversary searching the space of valid abduction–deduction maps. Experiments show improvement over Colored-MNIST, attaining 60% accuracy on the hard target domain where association between color and digits is reversed. We also evaluate the method in tasks involving language representations, and a real-world Vision benchmark.

Primary area:
causal reasoning
Keywords:
Domain Adaptation, Domain Generalization, Causal Invariances, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_2cu1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies domain adaptation under unobserved confounding and proposes an abduction–deduction entanglement framework. The method introduces representation transplants to preserve shared deduction while modifying domain-specific abduction, and employs Causal Robust Optimization (CRO) to improve robustness under distribution shifts. Theoretical results characterize partial transportability under specific assumptions, while experiments on Colored MNIST, a synthetic moral questionnaire, and TerraIncognita evaluate the proposed framework.

Critical strengths
  1. The paper addresses an interesting and challenging problem in domain adaptation by considering unobserved confounding and changes in the optimal prediction rule across domains.

  2. The proposed representation transplant mechanism provides an interesting connection between causal modeling and representation learning. The theoretical analysis offers a structured formulation of partial transportability under explicit assumptions.

  3. The experiments on Colored MNIST and the controlled moral questionnaire demonstrate the potential of the proposed approach under severe changes in spurious correlations.

Critical weaknesses

1. Insufficient experimental validation on challenging real-world benchmarks. The experimental evaluation relies heavily on Colored MNIST and a controlled moral questionnaire, which may not adequately reflect complex real-world domain shifts. Although TerraIncognita is included, CRO achieves only 46.1% accuracy, comparable to ERM and below several existing methods. Additional evaluations on standard benchmarks, such as Office-31, Office-Home, or DomainNet, are needed to demonstrate the effectiveness and generalizability of the proposed framework. 

2. Limited evidence from the Colored MNIST benchmark.  The main performance improvement is observed on the artificially constructed reversed-color environment of Colored MNIST. Although this benchmark is useful for studying spurious correlations, it provides limited evidence of generalization to complex real-world shifts. Moreover, the proposed method achieves only 60.5% accuracy on the reversed-color target, while a color-independent digit predictor has a theoretical reference accuracy of approximately 75% under the dataset construction. Comparisons with simple color-invariant training strategies and stronger baselines would be valuable.

3. Strong theoretical assumptions.  The theoretical formulation relies on several restrictive assumptions, including shared deduction mechanisms, linear abduction and deduction maps, and closure under representation transplants. It remains unclear whether these assumptions approximately hold for high-dimensional representations extracted by modern neural networks.

4. The gap between theoretical guarantees and practical implementation.  Theorem 4.1 establishes minimax optimality under exact feasibility constraints and global optimization assumptions. However, the practical implementation relies on approximate support matching, nearest-neighbor evaluation, and local optimization, which do not guarantee satisfaction of these conditions. Moreover, the optimized transplanted-source risk is not necessarily equivalent to the actual target risk.

5. Insufficient ablation studies of the core components.  The paper lacks sufficient ablation studies to isolate the contributions of representation transplantation and adversarial optimization. It remains unclear whether the improvements arise from the proposed abduction–deduction mechanism or conventional robust optimization. Additional component-wise ablations and sensitivity analyses of the latent cardinality would strengthen the experimental validation. 

6. Limited evidence for the effectiveness of unsupervised domain adaptation.  Although the paper evaluates both DG and UDA settings, incorporating unlabeled target data does not consistently improve performance. Moreover, some comparisons use different test subsets, making it difficult to assess the actual contribution of target-domain information. The authors should provide controlled DG and UDA comparisons using identical evaluation sets and analyze why unlabeled target data sometimes fails to improve performance.

Questions for authors

Please see the Weaknesses section above.

Reviewer_LcquOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies domain adaptation and generalization under unobserved confounding, where a shared causal mechanism generates each domain's covariates and label from an unobserved variable and only the mechanism that generates labels is assumed invariant across domains. The optimal prediction rule in every domain factorizes into an abduction map, which infers a distribution over the unobserved variable from the covariates, and a deduction map, which predicts the label from the covariates and the inferred variable. Many pairs of abduction and deduction maps reproduce the same domain's response probabilities, so this factorization is not unique, a non-identifiability the paper calls entanglement between abduction and deduction. Under a linearity assumption on abduction log odds and deduction logits in a fixed representation, the paper defines representation transplants, invertible linear maps that swap the abduction component of a representation between two domains while leaving the deduction component fixed, and uses them to characterize which target conditional distributions remain consistent with the shared deduction map and the observed source distributions (partial transportability). Causal Robust Optimization (CRO) searches the resulting space of valid transplants with a game between the learner and adversary and provably terminates within a chosen tolerance of the minimax optimal risk on the target domain when both players solve their subproblems to global optimality. Experiments on Colored MNIST, a questionnaire derived from MoralChoice with labels elicited by an LLM, and TerraIncognita compare CRO's heuristic numerical implementation against domain generalization and adaptation methods.

Critical strengths
  1. Table 1 shows CRO reaching 60.5 ± 2.9% target accuracy on Colored MNIST's -90% environment. Table 3 shows all fourteen comparison methods, including methods built to resist spurious correlation (IRM, V-REx, ARM), stay near 10% there, a large and independently verifiable gap on a benchmark known to break standard invariance-based methods.

  2. TerraIncognita (Table 8) shows CRO matching ERM (46.1% vs 46.3%) and trailing SagNet (48.6%), and the paper states this plainly as "comparable to ERM." This kind of disclosed null result is reported directly in the main text instead of omitted.

Critical weaknesses
  1. CRO's two largest gains are both in settings where the method's structural assumptions hold by design: Colored MNIST is a synthetic benchmark with a known causal mechanism, and the moral questionnaire is explicitly constructed to "instantiate Figure 2." TerraIncognita, the one benchmark where this construction is absent, shows CRO matching ERM (46.1% vs 46.3%) and trailing SagNet (48.6%).

  2. No experiment measures how well the linearity assumption holds for the frozen ConvNet or RoBERTa representations used on the natural benchmarks. Table 10's only related quantity is a fit standard error presented as a search tolerance, not a diagnostic of the assumption. Without this check, the TerraIncognita non-result cannot be attributed to a specific failure mode, and it is not clear whether the Colored MNIST gains would replicate in a natural setting where linearity holds only approximately.

Questions for authors
  1. The paragraph following Table 6 reports results "with three sources" (CRO 65.4 ± 0.4%, DANN 44.5 ± 0.7%, Deep CORAL 44.3 ± 0.7%), but Table 6 has columns only for K=1, K=2, and K=4, and none of its cells matches these three numbers exactly. Where do the numbers for three sources come from, and should Table 6 include a K=3 column?

  2. Appendix D.1 states that complete accelerator time and search cost were not retained. Is there a partial record or rough estimate of CRO's wall-clock cost relative to one training run of ERM, DANN, or Deep CORAL?

  3. Section D.3 confirms the moral questionnaire generator's response probabilities are known. Is there a direct measure, for example, a cross-entropy or calibration comparison against the fitted linear deduction map, of how closely the linearity assumption holds for the frozen RoBERTa-based representation used there?

Reviewer_Z68wOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a causal framework for domain adaptation that tackles the entanglement between latent-variable inference and outcome prediction. The framework introduces representation transplants to align abduction components across domains while preserving shared deduction mechanisms, and develops a causal robust optimization strategy to improve prediction under potential domain shifts. The experimental results show that the method performs well when spurious correlations are reversed.

Critical strengths
  1. The paper provides an causal perspective on domain adaptation by explicitly modeling the entanglement between abduction and deduction, offering a theoretical explanation for why shared prediction mechanisms may not guarantee successful domain transfer.
  2. The experiments demonstrate robustness against reversed spurious correlations, particularly on Colored MNIST, and include evaluations on both synthetic and real-world benchmarks.
Critical weaknesses
  1. The paper introduces an interesting causal perspective by modeling the entanglement between abduction and deduction. However, it remains unclear how this formulation advances existing domain adaptation theories or translates into practical benefits for modern deep learning models. In particular, the framework is not evaluated with widely used pretrained models or modern adaptation strategies, leaving its broader applicability insufficiently demonstrated.

  2. Although CRO achieves substantial improvements under specific distribution shifts, particularly the reversed color-label correlation in ColoredMNIST and the framing shift in the moral questionnaire, its advantages are not consistent across benchmarks. On TerraIncognita, CRO achieves 46.1% accuracy, which is comparable to but slightly lower than ERM (46.3%). It also underperforms several baselines in the other ColoredMNIST environments. These results suggest that the effectiveness of CRO may be highly dependent on particular types of distribution shifts, rather than providing a general performance advantage for domain adaptation.

  3. The experimental results demonstrate improvements under specific correlation and framing shifts, but do not directly verify that these gains arise from the proposed abduction–deduction mechanism or that representation transplants successfully preserve the deduction component.

  4. The theoretical framework relies on strong assumptions, including linear parameterizations of abduction and deduction in a fixed representation space, as well as specific rank and support conditions. It remains unclear how well these assumptions hold in complex, nonlinear visual representations.

  5. The paper introduces a relatively sophisticated theoretical framework based on abduction–deduction entanglement, together with representation transplants and a minimax optimization strategy. However, it remains unclear whether this additional complexity provides substantial benefits over existing approaches.

Questions for authors

Please address the above issues.

Reviewer_GcMaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper explores ways to disentangle abductive and deductive inference in modelling causal processes. It provides a modelling framework that makes this possible, and an algorithm for finding the maps that are used for disentanglement in that framework.

The algorithm is applied to three small problems; on two of them it substantially reduced the effects of an intervention of the data that requires a transformation to maintain classification accuracy.

Critical strengths

The model is interesting, and points at the possibility of more widespread applicability of causal representation learning in ML systems. The model and algorithm are tested on existing small datasets, with substantial improvements in two of three cases.

Critical weaknesses

The algorithm seems likely to be restricted to small problems; while it will converge, it is likely to do so slowly. The paper is not very clear about the path in the prior literature that led to the problem formulation or about the nature of the competing approaches (it is not sufficient for them to be explained in the appendices).

Questions for authors

What are the prospects for extending these techniques to settings not involving pre-trained models, or to cases where the current search algorithm given would be intractable?

Comments on the paper

2 comments
AuthorsOpenReview ↗

Thank you for reviewing our work

General response (to all reviewers)

We thank the reviewers for their careful reading, and for recognizing the central idea of the paper: when the mechanism of the label is shared across domains but unobserved confounding changes, prediction rules differ from domain to domain, and the sources still constrain what can be predicted. The revision we have uploaded keeps this idea and strengthens the paper on every point you raised.

  • Stronger results on the standard benchmarks. We now report the complete DomainBed suite, seven datasets and 31 held-out domains, under its own training-domain-validation protocol and next to all fourteen published algorithms. On Colored MNIST, where all fourteen stay at 10% on the reversed domain, our algorithm reaches 73.4 ± 0.4%, against a 75% digit-only reference; its average over the three domains is 73.4%, against 56.2% for the best published algorithm. On DomainNet, the largest dataset of the suite, it is above every published algorithm on all six domains.
  • A sharper formulation with a single assumption. One assumption relates the domains: a common mechanism of the label. Abductive causal models parametrize the abduction and deduction maps directly on a pretrained representation, with an exact likelihood and no support or closure condition. CRO keeps its guarantee of minimax optimality, and a new algorithm derived from it, CIAO, runs as one regularized fit on a frozen network, in seconds to minutes.
  • Every analysis you asked for. Ablations of each component, a direct test of linearity in the representation, sensitivity to the method's constants, and wall-clock cost are all in the appendix.

The title is now Abductive Causal Models for Robust & Generalizable Learning, after the object the paper contributes. We invite you to read the revised manuscript: we believe it answers each of your concerns, as detailed in the individual responses.

Response to Reviewer GcMa

Thank you for the positive assessment, and for seeing the potential of the model beyond this paper.

  • Scale (W1). The method now runs on the largest benchmark of DomainBed. On DomainNet (345 classes, 0.6 million images) it takes under three minutes on top of one ERM run, and it is above all fourteen published algorithms on all six domains. On Colored MNIST it takes 3 seconds.
  • Prior literature and competing approaches (W2). The introduction now positions the work in the main text, against the theory of domain adaptation, invariance-based methods and partial identification, and Section 2 explains why the existing parametrizations of causal models, canonical partitioning and neural causal models, do not reach texts and images. A related-work section with a side-by-side comparison of related methods complements it.
  • Beyond pretrained models (Q1). The framework does not depend on pretraining: abductive causal models are defined for any differentiable maps, and on Colored and Rotated MNIST we use DomainBed's small network, trained from scratch on the sources.

Response to Reviewer Z68w

Thank you for highlighting the causal account of why a shared prediction mechanism does not guarantee transfer. The revision builds the whole paper on it.

  • Modern pretrained models (W1). The method is designed for pretrained representations. We evaluate it on DomainBed's ImageNet-pretrained ResNet-50 across five image benchmarks, and on RoBERTa for the language task.
  • Consistency across benchmarks (W2). DomainBed's own conclusion is that none of its fourteen algorithms is above ERM by more than one point on average. Under the same protocol, CIAO is above ERM on 19 of 31 held-out domains, above every published algorithm on all six domains of DomainNet, and 63 points above ERM on the reversed domain of Colored MNIST.
  • Source of the gain (W3). The ablations isolate it. Removing the orthogonality between the common rule and the source-specific corrections, or the corrections themselves, sends the reversed domain of Colored MNIST from 74.5% back to 11%.
  • Assumptions (W4). The formulation needs one assumption, a common mechanism of the label, and the model admits any differentiable maps. The linear form is used only by CIAO, and we test it directly: on the frozen representations, a linear rule is at least as accurate as a two-hidden-layer network on every benchmark but Colored MNIST.
  • Complexity and benefit (W5). The algorithm is a single regularized linear fit on a frozen network, with one set of constants for all benchmarks.

Response to Reviewer Lcqu

Thank you for the precise summary, and for noting the large and independently verifiable gap on Colored MNIST. The revision widens it, from 60.5% to 73.4 ± 0.4%, with all fourteen published algorithms at 10%.

  • Benchmarks and the method's assumptions (W1). Colored MNIST is the community's benchmark for spurious correlations and part of DomainBed. It was not built for our method, and the method is never told that color is the feature that changes. The gains now extend to natural images: on DomainNet, CIAO is above every published algorithm on all six domains. The moral survey is controlled on purpose: it is the one setting where the causal effect is known, so the objective itself can be measured.
  • Linearity (W2, Q3). Now measured directly. On the frozen representation of every benchmark, we compare the linear rule with a two-hidden-layer network on held-out source records: the linear rule is at least as accurate everywhere except Colored MNIST, where the network is 1.5 points better. On the survey, where the abduction is known, we report how closely a linear decoder recovers it.
  • Three sources (Q1). The survey tables now cover every set of sources, including three, from the same runs.
  • Cost (Q2). CIAO adds 3 seconds on Colored MNIST and under three minutes on DomainNet to one ERM run; the full timings are in the appendix.
  • Presentation. The paper now follows one argument, from the problem to the model to the algorithms, with a single running example.

Response to Reviewer 2cu1

Thank you for recognizing the problem, and the connection the paper draws between causal modeling and representation learning.

  • Real-world benchmarks (W1). Done: the complete DomainBed suite, including the OfficeHome and DomainNet you asked for. CIAO is above every published algorithm on all six domains of DomainNet, and above ERM on three of the four domains of OfficeHome.
  • Colored MNIST (W2). The result on the reversed domain is now 73.4 ± 0.4%. The 75% reference is the accuracy of a predictor that is told to ignore color. No algorithm in DomainBed's comparison, IRM and V-REx included, gets above 10.5% without that knowledge; ours comes within two points of the reference.
  • Assumptions (W3). The formulation rests on a single assumption, a common mechanism of the label. Abductive causal models need neither closure nor support conditions, and linearity is confined to CIAO, where we test it directly on the frozen representations.
  • Theory and implementation (W4). The implementation optimizes exactly the objective stated in the paper and is verified against it numerically. No support matching or nearest-neighbor step is involved.
  • Ablations (W5). Provided: each component of CIAO, the number of directions of its corrections, its penalty, and the amount of data. Without its two key components, the reversed domain of Colored MNIST drops from 74.5% to 11%.
  • Target data (W6). The method needs no target data. Every result is in the domain-generalization setting, on DomainBed's evaluation sets, under one protocol.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.