acceptodds
Under review as a conference paper at ICLR 2027

GRACE: Population-Level Adversarially Robust Counterfactual Explanations

Abstract

Counterfactual explanations (CFs) suggest changes to an input to flip a machine learning model's prediction toward a desired outcome. Most existing CF generators are not robust because they assume a static model and data distribution. However, models deployed in practice are retrained periodically, so a CF that is valid today may not be so tomorrow. Institutions typically issue a population of CFs, making robustness a population-level property, rather than an individual one. Existing robustness methods, however, optimize each CF against its own worst-case perturbation rather than the shared population-level risk induced by retraining, which may cost more proximity than necessary. We introduce Geometrically Restricted Adversarially robust Counterfactual Explanations (**GRACE**), a distributionally robust wrapper that treats the population of CFs produced by any CF generator as the robustness target. GRACE controls a surrogate for the population's worst-case invalidity under future retraining while keeping the refined CFs close to both the original CFs and the observed distribution. Across tabular and image datasets, GRACE substantially improves the future validity of the base CFs with only a small increase in proximity, matching or exceeding the future validity of recent state-of-the-art robust CF baselines on tabular benchmarks.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, counterfactual explanations, explainable AI

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_u7feOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors of this paper motivate that counterfactuals generated for a model now may be invalid after the training distribution shifts, inducing changes in the model through retraining. In this paper, the authors propose GRACE, a post-hoc refinement that aims to provide counterfactuals to a model that improve future validity. They consider that population-level optimisation could achieve comparable robustness to individual worst-case optimisation, while being less conservative and thereby preserving greater proximity to the original inputs. The authors evaluate GRACE against robust CF baselines on four tabular datasets, alongside image and a text experiments. They assess GRACE and its comparisons via future validity and proximity, concluding that GRACE improves future validity with relatively small increases in proximity, while matching or exceeding the robustness of existing methods on tabular datasets.

Critical strengths
  1. Placing GRACE as a post-hoc refinement rather than a brand new CF generator. This makes GRACE much easier to integrate with existing techniques. The evaluation with Wachter and DiCE provide evidence that the method also isn't dependent on a specific base generator.

  2. The tabular results for GRACE show great improvements in terms of Future Validity, often approaching 100% while maintaining competitively low proximity. The paper then also extends to image benchmarks and a small text one, too.

Critical weaknesses
  1. I think my biggest criticism of this paper is that I cannot find any direct ablation showing the proposed benefit of population-level optimisation. The central argument within the motivation for the methodology is that "each CF may use a different worst-case model and may guard against scenarios that never jointly arise, potentially moving CFs further than necessary", meaning I would have expected to see a demonstration of this point through an individual-level version of GRACE. Therefore, it is difficult to say whether the gains seen in the results arise from population-level optimisation or from the other components of GRACE.

  2. Due to the proposed methodology focusing on population-level optimisation, it is possible that there could be some individuals receiving more costly counterfactuals, such as a few with large proximity, hidden by the reported averaged proximity. I assume the standard deviations in the results table represents standard deviation across individual counterfactuals, which would address this concern somewhat. However, this is not explicitly stated.

Questions for authors
  1. (Relating to W1): Could the authors provide an ablation that compares GRACE against an individual-level version of GRACE, keeping every other component as similar as possible? This is my main concern with the paper, and a convincing ablation could change my recommendation.

  2. (Relating to W2): Could the authors first clarify what the standard deviations are calculated across and add this to the table captions? Could the authors also comment on the effect of population-level optimisation on individual counterfactuals, particularly whether some individuals could experience substantially larger increases in proximity despite the relatively low average? Maybe reporting distributions of individual proximity changes before and after refinement would help clarify this. Also, a side note: I noticed that Table 7 and Table 1 have identical standard deviations, despite showing different mean results and using different classifiers. Could the authors explain why this is the case? Also, can the authors clarify why ROAR and PROPLACE cannot be applied to German Credit and FICO in Table 7?

  3. Tables 1 and 7 report FV without CV, but both of these metrics are reported for the image experiments in Table 10. Could the authors also report CV for the tabular datasets?

Reviewer_RxRROpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes GRACE, a post-hoc distributional refinement method for counterfactual explanations (CFs) to improve their robustness against future model updates induced by training-distribution shifts. Instead of treating each CF independently, GRACE models the entire CF population as a distribution and couples all CFs through a shared retrained classifier. It introduces a two-anchor Wasserstein geometry constraint (Section 3.2) to prevent the refined population from drifting too far from the base CFs and the favorable-class data. To make this tractable, it uses an update-aware first-order sensitivity approximation (Section 3.4) that propagates training-data perturbations through a prescribed T-step gradient update. Experiments are conducted on tabular, images, and text data.

Critical strengths

This paper proposed a new problem of updating CF as a goup.

Critical weaknesses

Due to the heavy mathematical overload and the absence of intuitive explanations, understanding the core ideas of this paper required an excessive amount of time and cognitive effort. The paper fails to clearly define what a CF is, why an explanation becomes invalid after retraining, or to provide a concrete deployment timeline example. The formal definition of robustness does not appear until Section 3.1, long after it has been heavily used in the first half of the technical discussion. The paper assumes readers are already experts in both Algorithmic Recourse and DRO.

The paper seems to propose a toy problem. It assumes a batch of CFs is generated and frozen before a model update. In reality, CFs are generated on-demand for each specific user; when a model is retrained, a user simply receives a new CF. Optimizing a population of old CFs is mathematically contrived to fit Wasserstein DRO, rather than reflecting a real deployment need. The post-hoc one-to-one assignment (Section 3.4) merely patches this artificial problem and can lead to mismatched recourse.

There seems to be a problem in the experimental design, making the results questionable. To approximate future model changes, the method uses an unrealistically small number of gradient steps (e.g., T=3 for MNIST; line 361), whereas practical retraining involves thousands of step

Questions for authors
  1. Can you give a specific, real-world example where a system would actually need to optimize a group of old CFs?

  2. Your method uses only a few update steps to estimate future model changes. How do you know this accurately reflects real-world retraining, which usually takes thousands of steps?

  3. Your method uses a minimum-cost matching to assign the optimized CFs back to individual users. How do you make sure the CF assignment is correct?

Reviewer_ExeeOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces GRACE (I flagged the authors that it might collide with other acronyms), a method that makes counterfactual explanations more reliable when models are updated. GRACE improves its reliability by considering the full set of explanations together and adjusting them to remain effective after future model updates. Experiments on tabular and image datasets show that GRACE improves the validity of explanations while making only small changes to the suggested solutions (although I noticed that the reported proximity in Table 1 is no lower than that of the base models DiCE and Wachter).

Critical strengths

The paper is nicely written and, with a few exceptions in the math part (not because the math is not understandable, but it's missing the why), easy to follow.

The problem they're tackling is real and is now exacerbated by the EU AI Act, which requires recourse; however, data rarely stays static, thus invalidating counterfactuals.

I really enjoyed the paper.

Critical weaknesses

Nothing critical, just some questions around which explainer is better in terms of FV and efficiency (s / CF). I don't think they're detrimental to the paper's soundness. Let's see how the authors respond to the questions and whether those responses lead to underlying critical weaknesses I might have missed at this stage (I sure hope not).

Questions for authors

1 and 2 are almost the same question. 2 is more specific to population distribution vs. a single anchor point.

Q1: Figure 2 and the rest of the section scream "image style transfer" to me. In this line, I noticed that GIST [1] addresses style transfer in graph counterfactual explanations. So, while analyzing Figure 2, it seems like the same idea is present in image-style transfer, where you have style and content loss. Would you clarify how this relates to the suggested paper? Just food for thought.

Q2: The more I read the math in Sections 3.2 and 3.3, the more I think that you're trying to overengineer image style transfer for counterfactual explanations. Can you please explain if this is in any way related to that problem? I am aware that you're approaching the work from a different perspective, but having the P1P_1 and PCFP_{CF} anchors actually makes it like that. In [1], the authors use single anchor points, whereas you use distributions over the training data and CFs. This does make sense when the data undergoes a shift in distribution. However, how would you put GRACE if the data didn't undergo a shift? Would two anchor points suffice?

Q3: What's the underlying model you're trying to explain, and how faithful are its predictions with respect to the data?

Q3.1: How do these performances change when the data undergoes a distribution shift? Would the chosen counterfactuals be faithful to the new model predictions?

Q4: Is the validity you're reporting based on the ground truth labels or the predictions of the underlying classifier/model (see pp. 22-23 of [2] for a distinction between validity and fidelity)? I'm not sure whether you're using validity or fidelity here; that's why I ask.

Q4.1: Why don't you report both?

Q5: How come the unrefined baselines still perform well under distribution shifts (see Table 1)? Am I missing something here? They sometimes outperform the other robust SoTA methods.

Q6: In Table 5, why don't we prefer ELLiCE according to these fixed 10 instances? I mean, it's orders of magnitude faster than GRACE and has comparable validity under distribution shifts (Table 1).

[1] Prenkaj et al. Graph Inverse Style Transfer for Counterfactual Explanations. ICML'25.

[2] Prado-Romero et al. A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challenges. CSUR'24


Suggestion: I saw that there are two other papers in graph counterfactual explainability with distribution shifts that use the same acronym somehow (i.e., DyGRACE [3] and GRACIE [4]) by the same research group. Nothing to worry about, just saying.

[3] Prenkaj et al. Adapting to Change: Robust Counterfactual Explanations in Dynamic Data Landscapes. Workshop@ECML-PKDD'24

[4] Prenkaj et al. Unifying Evolution, Explanation, and Discernment: A Generative Approach for Dynamic Graph Counterfactuals. KDD'24

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.