acceptodds
Under review as a conference paper at ICLR 2027

CounterFlow: Target-Conditioned Flow Matching for Visual Counterfactuals

Abstract

In eXplainable AI, counterfactual explanations have gained significant traction by offering users direct, intuitive, and actionable insights into model decisions. We introduce CounterFlow, a novel framework based on Conditional Flow Matching that generates visual counterfactual explanations by directly steering a source image toward a user-specified target class. By leveraging target-conditioned vector fields, CounterFlow identifies highly realistic semantic modifications while requiring substantially lower computational overhead than traditional GAN- or diffusion-based architectures. Empirical evaluations on standard benchmarks demonstrate that CounterFlow achieves state-of-the-art performance in both visual realism and target class switching. By working directly in the raw pixel space, CounterFlow offers a more practical solution for real-world deployment.

Primary area:
interpretability and explainable AI
Keywords:
Counterfactual Explanations, Flow Matching

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_5b5bOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CounterFlow, a conditional flow-matching framework for visual counterfactual generation. The main idea is to directly learn a transport field from a source image toward a user-specified target class, avoiding the intermediate prior-space traversal and repeated optimization used by several existing approaches. The method combines source/target class conditioning, latent-space nearest-neighbor pairing, and an auxiliary classification loss. Experiments on MNIST and CIFAR-10 show strong conditional validity and favorable computational cost compared with several GAN-, diffusion-, and flow-based baselines.

Critical strengths

Simple and well-motivated formulation. Directly learning a source-to-target transport field is conceptually clean and avoids several expensive steps required by prior generative counterfactual methods. The use of a single model for multiple class transitions is also practically attractive. Strong empirical efficiency and validity. On CIFAR-10, CounterFlow achieves 100% conditional validity while substantially reducing FLOPs and model size relative to some competing methods. The latent-pairing ablation also supports the proposed training design.

Critical weaknesses

The experimental scope is too limited to establish generality. The evaluation is restricted to MNIST and CIFAR-10, both low-resolution benchmarks. This makes it difficult to judge whether direct pixel-space transport remains effective on higher-resolution or more complex images. The paper itself acknowledges this limitation.

The evidence for instance-level proximity/minimality is weak. The paper uses FID between generated counterfactuals and the source-class distribution as a proximity metric. This is a distribution-level measure and does not directly quantify how much an individual counterfactual changes relative to its corresponding source image. Therefore, claims such as “near-minimal transport” are stronger than what the current evaluation directly supports.

Questions for authors

Could the authors report instance-level proximity metrics, e.g. LPIPS, feature-space distance, or pixel-level distances between each source and its counterfactual? This would materially strengthen the minimal-edit claim.

Do the authors have results on a more challenging dataset or higher-resolution setting? Even one additional experiment would help clarify whether the method scales beyond MNIST/CIFAR-10.

How sensitive are the results to the choice of pretrained VAE embedding used for nearest-neighbor pairing?

Reviewer_KUizOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper looks at the task of generating visual counterfactual explanations from the perspective of class-conditional image editing. The proposed CounterFlow method is a flow matching model that is trained to predict a velocity field between two similar images with different labels, which allows it to produce counterfactual explanations at inference. The experiments on MNIST and CIFAR-10 demonstrate that CounterFlow can generate counterfactuals for simple low-resolution data.

Critical strengths

The area is of great importance

Critical weaknesses
  1. Contribution and positioning. The paper considers the generation of counterfactual explanations as purely a class-conditioned image editing task; At no point of image editing, any signal from the classifier we want to explain is used (it only provides an initial label). This can work for simple data where classifiers achieve almost perfect test-accuracy (which is the case here), but will not work for real-world high-resolution images, or medical data, where even state-of-the-art models have a considerable mispredictions and uncertainty. Once the classifier cannot near-perfectly approximate gt-labels, this method will fail to explain such a model as it is using just labels and not a signal from the model to be explained. Furthermore, other methods (like FastDiME baseline) use explicit classifier guidance to produce counterfactuals and don't have this issue. The proposed method also requires one to train a whole generative model, while other methods (like the mentioned DVCE) allow for training-free use of general pretrained diffusion models.

  2. Experiments. The paper considers only 2 toy-datasets with 3 total baselines (only 2 reported per dataset). While this can serve as preliminary experiments, it cannot be considered as an evidence of the approach being "state-of-the-art", especially when the considered baselines were originally evaluated on different and much more complex datasets (e.g. see original FastDiME paper).

Questions for authors
  1. What is the rational behind considering the generation of counterfactual explanations as purely class-conditional image editing, and what are its limitations?
  2. Why don't you use any signal from the explained model at any step of image editing?
  3. How can the proposed CounterFlow explain not near-perfect classifiers?
Reviewer_Mm9TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes CounterFlow, a method that generates visual counterfactual explanations for an image classifier with conditional flow matching. Instead of generating images from noise, it learns a velocity field, conditioned on the source and target classes, that transports an input image of one class directly to an image of another class in pixel space. Training pairs are formed within each minibatch by matching every source image to its nearest target-class image in the latent space of a frozen pretrained VAE. An auxiliary cross-entropy loss on the explained classifier, applied after a small Euler step and weighted by time, pushes the flow toward the target class. At inference, the counterfactual is obtained by integrating the learned ODE once from the input, with no VAE and no iterative optimization. Experiments on MNIST and CIFAR-10 compare the method with C3LT on both datasets, LeapFactual on MNIST and FastDiME on CIFAR-10. The metrics are conditional validity, FID to the target and source classes, FLOPs per sample and model size. CounterFlow reports 99.6% and 100% validity and the lowest target FID on both datasets, and it is cheaper than C3LT and FastDiME. The appendix ablates the weight of the classification loss, the class-pair conditioning and the explained classifier.

Critical strengths

S1. A simple, well-motivated formulation with cheap inference. Learning a direct source-to-target transport with one field conditioned on the class pair avoids several costs of prior work: one model per class pair (C3LT), a round trip through the prior and repeated VAE decoding (LeapFactual), and classifier guidance at every denoising step (diffusion methods). Inference is a single ODE integration without classifier gradients, which suits interactive use. The method is described clearly, with algorithms for pairing and training, and the hyperparameters are listed in Table 2.

S2. Informative ablations. Appendix A.3 isolates each component on CIFAR-10. Without the classification loss, validity is only 68.9%. With the classification loss alone, the model collapses (FID above 400). Without the class-pair context, validity falls to 8.6%. With an adversarially robust classifier, validity drops to 83.5% while target FID improves. The authors mark the rows in which FID is not meaningful because many samples stay in the source class, which is a careful reading of their own results.

Critical weaknesses

W1. The evidence suggests that part of the 100% validity comes from non-semantic changes. Measuring validity with the explained classifier ff is the correct definition. However, validity alone cannot distinguish semantic counterfactuals from adversarial perturbations, especially when the flow is trained by backpropagating through the same ff (lines 341–342). Without the classification loss, validity is only 68.9%. Adding it raises validity to 100%, while target FID changes by only about 5 points. The paper itself attributes the drop to 83.5% under a robust classifier to "non-robust directions that a gradient-based guidance term can exploit" (lines 702–703). A mis-specified classifier with 86.3% accuracy is still "explained" with 100% validity (Table 5). In Figures 2b and 5, most CIFAR-10 predictions flip at the first step while the image is visually unchanged, which is what adversarial perturbations look like. The paper distinguishes counterfactuals from adversarial examples (lines 35–36), but its headline validity numbers do not yet show that the counterfactuals are semantic.

W2. The scope is narrow, the baselines are weak, the efficiency claim is overstated, and the novelty is limited. Experiments use only MNIST and CIFAR-10, not the CelebA benchmark on which DiME, ACE and FastDiME are usually evaluated. FastDiME reaches only 16% validity on CIFAR-10, which suggests a configuration mismatch rather than a representative baseline. On MNIST, CounterFlow needs about 200 times more FLOPs than LeapFactual (174.66M vs. 0.86M, Table 1a). This contradicts the claim of "substantially reducing inference complexity" (line 431). Several components are already established: data-to-data flow matching with minibatch couplings (Tong et al., 2024), rectified-flow transfer between image domains (Liu et al., 2023b), and label-conditioned multi-domain translation (StarGAN, Choi et al., 2018). Against this background, the class-pair conditioning, the nearest-neighbor latent pairing and the classifier loss are each a modest step.

Questions for authors
  1. Validity under an independent classifier. For counterfactuals generated against RepVGG-A2, what is the validity under a classifier that was not used in training?
  2. Memorization and independent realism. Several counterfactuals replace most of the image (for example horse→truck in Figure 1). Can you report each counterfactual's distance to its nearest training image of the target class, to rule out near-copies? For MNIST, can you report FID with a feature extractor independent of the explained classifier?
  3. Baselines and benchmark. How was FastDiME configured on CIFAR-10, and why does it reach only 16% validity? Can you evaluate on CelebA or CelebA-HQ, where DiME, ACE and FastDiME have established numbers and metrics? For LeapFactual on MNIST, do the reported FLOPs include VAE decoding and classifier queries for all leaps?

Comments on the paper

1 comment

We thank the reviewers for their feedback and for recognizing that CounterFlow, which steers images directly in pixel space toward any user-specified target class using Conditional Flow Matching, is original, and that the paper is well written and easy to follow. The main concern raised is that MNIST and CIFAR-10 are too simple to show the method’s potential. We would like to point to Appendix A.4, which explains our dataset choices, in particular why CelebA, though used by some competing methods, does not allow us to evaluate CounterFlow’s ability to transport samples from one multi-class distribution to another. Given that the reviews contain few factual criticisms and that all reviewers recognize the idea as interesting, we believe that the very negative current scores do not fully reflect the assessments written in the reviews.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.