acceptodds
Under review as a conference paper at ICLR 2027

Recourse on Borrowed Time: Counterfactuals that Survive a Model Update

Abstract

Recourse has to respect what a person can actually change: age and sex are fixed, income moves only one way, employment type is categorical. The methods that honour such constraints exactly are mixed-integer programs, and we examine how their recommendations fare when the model changes. The leading exact method asks only that the decision flip, and it shows: under its shipped validity threshold a third of its counterfactuals sit exactly on the decision boundary, and only about half survive an ordinary retrain of the deployed classifier. That threshold is a parameter, and raising it repairs both, at the cost of a mixed-integer solve per individual. We introduce PACER, which obtains the same guarantee at gradient cost. It compiles a dataset's metadata into a symbolic rule base, projects every gradient step onto it, and optimises to a confidence margin the user sets. Plausibility comes from a sum–product network distilled into a differentiable surrogate: the exact circuit scores every result while the surrogate only supplies the search direction. PACER is the only gradient-based method in our study whose counterfactuals are fully actionable on all three benchmarks, and it keeps – of them valid after a model update, at full coverage, in one to two seconds each. We report what this costs as well as what it buys, and release a harness that reproduces the exact baseline's published numbers cell for cell.

Primary area:
interpretability and explainable AI
Keywords:
algorithmic recourse, counterfactual explanations, robustness to model updates, actionability, confidence margin, tabular data, sum-product networks, explainable machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_hJ9DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies actionable counterfactual recourse that remains valid after a classifier update. It proposes PACER, combining projected gradient optimisation, a user-set logit-margin target, a categorical softmax relaxation, and an SPN-derived plausibility surrogate, while scoring plausibility with the original SPN. The authors position this as an alternative to mixed-integer recourse and diagnose the fragility of LiCE’s default validity threshold. Experiments on GMSC, Adult and German Credit use five folds and 100 factuals per fold, with comparisons, margin sweeps and ablations; a smaller German Credit experiment varies LiCE’s threshold. Reported retention improvements concern same-data, same-architecture retraining with a different seed; they do not establish robustness to distribution shift or certify the requested margin.

Critical strengths
  • The robustness/cost-frontier experiments are interesting: they examine how retention after retraining changes with the recourse cost as the margin varies (Lines 367–378). This is an interesting empirical component of the paper.
Critical weaknesses
  • The presentation is difficult to follow. “Exact” and the identity of the “leading exact method” remain unclear in the opening; notation appears before it is explained, and terms such as coverage, confidence level, robustness and adaptive penalty are introduced without enough immediate context (Lines 020–027, 046–069, 078–079). The repeated forward references and vague statements such as “raising it repairs both” make the contribution difficult to assess (Lines 049, 057).
  • Some framing and empirical choices need clearer justification. The claim that exact satisfaction of the constraints has required integer programming seems too strong: could a gradient-based approach with rejection sampling also do this (Lines 014–015, 160–161)? The abstract’s numerical claims seem sensitive to the data, sampling and classifier (Lines 016–019), and Table 3 reports two vastly different thresholds without making their selection or the intermediate behaviour clear (Lines 378–387). These concerns leave the scope of the claims and the interpretation of the comparison unclear to me.
Questions for authors
  • Clarification — Lines 014–015, 046–054 and 160–161: What precisely does “exact” mean here, and which method is the “leading exact method”? Why would exact constraint satisfaction require integer programming rather than, for example, a gradient-based approach with rejection sampling?
  • Clarification — Lines 061–064 and 162–168: How should the confidence margin be interpreted relative to the scale of the space, and what is the significance of automatically deriving the rule base from metadata rather than specifying the constraints by hand?
  • Clarification of experimental choice — Lines 378–387 and 395–405: How were the two reported τ values selected, and what happens at intermediate thresholds? The Table 3 caption refers to intermediate thresholds in the text, but I could not locate their numerical results.
2 comments

Thank you for the careful reading, and for highlighting the robustness/cost frontier; Figure 1 now also shows FACE, ElliCE and the exact method's threshold sweep. We answer in your order.

Scope (from your summary). We agree on both points and the revised paper says so. Retention is now also tested under LiCE's training recipe and two other model families, a wider MLP and a logistic regression (Table 22), but distribution shift remains out of scope (§7). The margin is a target, not a guarantee (abstract, §7), and Table 24 shows where it is lost.

Weaknesses

  • W1 (presentation). The abstract and introduction are rewritten in short sentences. "Exact" is defined at first use and LiCE is named as the leading exact method (Q1). Signed logit, coverage and the margin are defined where they first appear, with the margin's meaning in probability (Q2); the adaptive penalty is described in plain words in the introduction and named "penalty escalation" only in §3.3, where it is defined. "Raising it repairs both" now names both effects: counterfactuals leave the boundary, and they survive a retrain. Section references now come at the end of a claim, not in the middle of it.
  • W2 (framing and choices). The integer-programming claim is softened (Q1). The abstract now states its results without point estimates; we measured three sources of variation: rerunning PACER on new hardware moves five-fold means by at most 0.03, retention at m=1.0m = 1.0 varies by at most 0.03 across six retrains, and it stays at 0.85–1.00 under a change of model family (Table 22). Table 3 now shows the intermediate thresholds (Q3).

Questions

  • Q1 (what "exact" means; is integer programming required?). "Exact" means a method that encodes the constraints and the decision in a mixed-integer program and solves it, to optimality or to the time limit; the leading exact method is LiCE. Integer programming is not required, and the claim is softened (§3.1). Rejection sampling returns feasible points but does not steer the search: among DiCE's ten candidates per factual, a valid one satisfying both the shared rule base and LiCE's feasibility model exists for every GMSC factual, but for only 65% of Adult and 3% of Credit factuals (App. N). Projection, as in PACER and in ElliCE's gradient mode, keeps every iterate feasible.
  • Q2 (reading the margin; the metadata rule base). The margin is set on the deployed classifier's output, not in feature space, so it does not depend on how the features are scaled: m=1m = 1 asks for a probability of at least σ(1)≈0.73\sigma(1) \approx 0.73 under that classifier (§3.1). Because logit scales differ between models, it is not comparable across classifiers. Deriving the rule base from metadata means no constraint is written by hand, every method is scored against the same constraints, and each rule carries exactly what the closed-form projection needs.
  • Q3 (choice of τ\tau; intermediate values). τ\tau runs from LiCE's default (10−410^{-4}) to the largest margin we use for PACER (1.5). Table 3 now includes 0.5 and 1.0, as its caption promised. On Credit, retention rises 0.57 → 0.77 → 0.86 → 0.96 as distance rises from 2.52 to 4.98; PACER at m=1.0m = 1.0 retains 0.99 at distance 6.29. Table 23 extends the sweep to Adult and GMSC on identical instances.

Thank you again for your review. In case it helps before the discussion closes: the revised PDF addresses each of your points. "Exact" and the leading exact method (LiCE) are now defined at first use, and the introduction is rewritten in short sentences. Table 3 now includes the intermediate thresholds (τ=0.5\tau = 0.5 and 1.01.0), and Table 23 extends the sweep to all three datasets. The integer-programming claim is softened (§3.1). The abstract now states its results without point estimates, and Table 22 reports their sensitivity to retraining and model family. We would be glad to clarify anything that remains.

Reviewer_jMNEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper considers the problem of actionable counterfactuals and their validity after the deployed classifier is retrained. For example for LiCE, one of the main competitors, the default validity threshold is essentially zero, so the counterfactuals sit on the decision boundary and a large share stops working after a retrain. The proposed method is PACER: gradient-based, with a user-specified classification margin, actionability constraints compiled from dataset metadata, a deterministic softmax relaxation for categoricals, and a distilled surrogate of a sum-product network for plausibility. On GMSC, Adult and German Credit it reports higher retention than LiCE at its shipped threshold and full actionability under the shared constraint checker. The paper also shows that just raising LiCE’s threshold fixes both problems, and presents PACER as a way to get the same guarantee without a solver.

Critical strengths
  1. The measurement of recourse under retraining is useful as robustness measure. Near-boundary counterfactuals turn out to be fragile under an ordinary retrain, and raising the threshold improves retention. I also appreciated that results are re-scored against the released model artifacts rather than regenerated models
  2. The framework is practical. Constrained gradient optimization, categorical relaxation and SPN-based plausibility are put together sensibly, retention is high under the retraining protocol used here, and the shared actionability checker is satisfied on all three datasets. The authors are also aware that raising the threshold on the exact method would work too.
Critical weaknesses
  1. It seems hard to tell how much is gained over what already exists. ElliCE is the closest competitor, but Appendix M only runs its data-supported variant, on a single fold with 20 factuals per dataset. From what I can see of the released library, the gradient mode with actionability constraints may also have been available. The LiCE comparison has a similar problem. Table 3 shows that raising the existing threshold takes retention from 0.57 to 0.96 while still landing closer to the factual than PACER does (4.98 vs 6.40), though the comparison is τ=1.5\tau=1.5 against m=1.0m=1.0, so the margins are not matched. The speed argument cannot close the gap either, since Tab.13’s own caption says the runtimes come from different hardware and different solvers, and speed is what the framing rests on.
  2. Margin control is weaker than the framing suggests. For a method built around a margin the user sets, PACER misses it fairly often; only 37% of GMSC counterfactuals reach m=1.0m=1.0 (Tab.16). Algorithm 1 does not say what happens to the ones that fall short after decoding. Robustness is then tested against a single retrained classifier per fold, same data and same architecture. Section 7 does concede that PACER targets a confidence level rather than certifying one, but given the size of the gap in Tab. 16, this reads as more than a limitations note.
Questions for authors
  1. Is it possible to run ElliCE’s constraint-aware gradient mode, and margin-adjusted LiCE, on all three datasets under comparable settings? How much of the advantage survives once achieved margins or retention levels are matched?
  2. Tab. 16 shows the requested margin is often not reached, even on GMSC, which is fully continuous and where I would have expected it to be easiest. Why? And what does the algorithm do with counterfactuals that are still valid but fall short of m, are they returned as-is, retried, dropped?
  3. Would the retention results hold across two or three independently retrained classifiers per fold rather than one?
  4. Why does the GMM beat the distilled SPN on NLL at m=1.0 (Tab. 6)? Relatedly, the distance you optimize drops categorical changes and uses different numeric weights than the distance you report, which seems like it could explain part of the proximity gap, unless I am misreading it.
  5. Minor: removing bounds costs validity on GMSC (1.00 →\rightarrow 0.71, Table 5) and the captions note this, but why does this happen?
3 comments

Thank you for the review and the precise questions; each led to an experiment. We answer in your order. Table numbers refer to the revised PDF (your Tab. 13 is now Table 14, Tab. 16 is now Table 17).

Weaknesses

  • W1 (gain over existing methods). Both comparisons are now run as you asked (Q1). ElliCE runs in its constraint-aware gradient mode on all three datasets with 5 folds × 100 factuals, and the exact method runs with a raised threshold on all three datasets on identical instances. On speed, Table 14 now states its hardware: PACER’s median is 1–2 s on a 2-core CPU. With HiGHS, the exact method takes 31–39 s per person on Credit and hits its time limit on every Adult and GMSC instance (Table 23). With Gurobi the gap is smaller: LiCE’s published Credit median is 3.1 s.
  • W2 (margin control; one retrain). Counterfactuals that are valid but short of mm are returned as they are (§3.4, now stated); Table 24 shows where the margin is lost and a retry reduces it (Q2). §7 and the abstract now say that PACER targets a margin rather than guaranteeing it. Robustness is now tested on six retrains per fold and two other model families (Q3).

Questions

  • Q1 (ElliCE and margin-adjusted LiCE under comparable settings). ElliCE (Table 19; also Fig. 1, Tables 2 and 4) is fully actionable, faster, and more robust than PACER on Adult and Credit (1.00 vs 0.96/0.99), less so on GMSC (0.95 vs 1.00). Its counterfactuals are 4.0× (Adult) and 4.6× (Credit) farther at its default objective and about 3× with a distance term at validity ≥ 0.99; at w=10w = 10 the gap is 1.8–2.4×, but validity drops to 0.37/0.44. This replaces our preliminary comparison. LiCE with a raised threshold (Table 23): at matched retention its counterfactuals are 1.2–1.8× closer on Adult and Credit, so there PACER's advantage is cost, not proximity; GMSC (five instances) is too small to compare. Table 3 now includes τ=0.5\tau = 0.5 and 1.01.0: at τ=1.0\tau = 1.0, LiCE retains 0.86 at distance 3.78, against 0.99 at 6.29 for PACER at m=1.0m = 1.0.
  • Q2 (why GMSC misses the margin; what happens to short counterfactuals). They are returned as they are. Table 24 tracks the margin through decoding. On Adult the optimiser always reaches mm, and the margin is lost when categories are rounded. On GMSC the optimiser stops at or just below mm for about a third of instances (median achieved margin 0.98 at m=1.0m = 1.0), and rounding its integer count features then pushes more just below it. A retry that raises the margin by the shortfall lifts the share reaching m=1.0m = 1.0 from 0.65 to 0.90 on Adult and from 0.36 to 0.66 on GMSC, at +0.15 and +1.6 in distance.
  • Q3 (several retrained classifiers). Six per fold (three with our recipe, three with LiCE's), plus a wider MLP and a logistic regression (Table 22). At m=1.0m = 1.0, PACER's retention varies by at most 0.03 across retrains and stays at 0.85–1.00 under a change of model family; LiCE (opt.) ranges over 0.45–0.88 and keeps 0.10–0.57. Table 2 now averages three retrains.
  • Q4 (GMM vs SPN on NLL; optimised vs reported distance). The mixture's log-density enters unscaled while the surrogate's target is standardised, so at equal λp\lambda_p the mixture pulls harder toward dense regions: lower NLL at much lower validity (Adult 0.70 vs 0.96 at m=1.0m = 1.0, Table 6), and NLL is averaged over valid counterfactuals only. On distance, you read it correctly: much of the proximity gap comes from optimising a surrogate rather than the reported distance. Optimising the reported distance (Table 25) reaches LiCE's Credit distance (2.08 vs 2.19) with retention 0.94 vs 0.53, at a cost in Adult validity (0.93 → 0.68).
  • Q5 (bounds on GMSC). A decode artefact: removing the bounds widens each range sevenfold, and the decode threshold scales with that width, so edits below about 21% of a feature's range were zeroed. With the threshold on the real bounds, validity returns to 1.00 (§6, App. O).

I thank the authors for their answers.

My understanding is that PACER's advantage compared to LICE is computational efficiency, while compared to ElliCE, it is proximity. I acknowledge that in Table 23 (LICE comparison), the results show similar retention in most of the cases with better proximity in some datasets and better time. However, the drop in coverage when m= 0 implies that the explicit margin may have robustness but at the cost of validity. For ElliCE, the gain in proximity is not universal and does not seem to add enough gain, since ElliCE does not require a solver, is faster, and aligns better with LiCE's strict feasibility.

On GMSC (Table 24), even though increasing the margin helps, only 36% of the returned counterfactuals achieve the requested m =1. The masking and repair variant addresses the feasibility gap but is not part of the original submitted method.

The authors acknowledge the limitations I stated. I believe that this version of the paper is stronger; however, I tend to maintain my original score.

Thank you for the careful follow-up and for noting that the revision is stronger. Two clarifications, in case they help.

The margin and validity. The margin raises validity rather than costing it. Without a margin, counterfactuals that only just cross the boundary often fall back below it when they are decoded (consistent with Table 24), so validity drops at m=0m = 0: 0.73 on Adult's 30 instances and 2 of 5 on GMSC (Table 23). It rises with the margin on every dataset (Table 12, mm from 0.2 to 1.5: GMSC 0.94 → 1.00, Adult 0.81 → 0.99, Credit 0.97 → 1.00).

ElliCE. You are right that the proximity gain is not universal: on GMSC ElliCE is closer, while on Adult and Credit PACER is about 3× closer at validity ≥ 0.99 (Table 19). We agree that ElliCE is faster and closer to LiCE's stricter feasibility model, as the paper states.

We agree with your other points. Thank you again for the engagement.

Reviewer_Q92UOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes PACER, a new method for counterfactual explanations (CEs) that improves on an earlier method in terms of robustness while retaining its beneficial properties, such as actionability and plausibility. It identifies brittleness in the method that generates actionable CEs, and non-actionability in the robust methods.

Critical strengths
  • The paper identifies a robustness issue in an existing method for plausible counterfactual explanations.
Critical weaknesses
  • The manuscript would benefit from some editing. It is difficult to read in several places. Some passages (e.g., lines 81-83 and the "Our Position" paragraph) are hard to follow because of long, convoluted sentences or poor narrative flow; revising the paper for clarity would help me understand the contribution.
  • Further, the comparison does not use appropriate baselines. The method mostly compares to LiCE (Němeček et al., 2025) and the methods that it compared to. Except for PROPLACE (Jiang et al., 2024a), none of those were designed for robustness, so there is a lack of proper baseline methods. The authors list many in Table 1 and 11, but evaluate only PROPLACE. Why are other recent, robustness-oriented methods, e.g., Kostrzewa et al. (2026) from Table 11, not included in the experiments?
Questions for authors

Clarification requests

  • Why was the SPN extracted into a differentiable surrogate, rather than learning a differentiable density directly?
  • The plausibility results of PACER in Table 12 appear weaker than those of the other methods. Given that PACER optimizes for plausibility, what causes this discrepancy? How was the surrogate quality measured?
  • Does the claim at l161, that a MIP was needed for actionability, hold? For example, in regards to ELLICE (Turbal et al., 2026)?
  • Is there no penalty for changes to categorical features? How are discrete features handled more generally?
  • Why is the sweep over \tau for the baseline shown only for Credit (Table 3)? How does the method perform on other datasets?

Changes that would raise my score

  • Substantial editing for clarity.
  • FACE and sweep results for the baseline would be useful to visualize in Fig. 1. to better compare the available methods at once.
  • Some preliminary results with a more relevant measure (ELLICE) are left to the appendix. These should be run in full and placed in the main text.

Minor issues and typos

  • f seems to take samples from R^p, given the use of an encoder \phi, but is then used directly on x's
  • What is isclose, and where is it defined?
  • How is the validity weight initialized (at 0, or otherwise)?
  • In the abstract, why do you assume that age cannot increase and income cannot decrease (or increase)?
4 comments

Thank you for the review. We address each point in your order; the three changes you listed as raising your score are all made.

Weaknesses

  • W1 (clarity). The abstract, the introduction (including lines 81–83, now three short, numbered contributions), "Our position" and the results are rewritten in shorter sentences, and each term is defined at first use.
  • W2 (robustness baselines). We added ElliCE, the closest robustness method that supports actionability on mixed data. It now runs in full (5 folds × 100 factuals, both modes, with its constraints) and is in the main text (Figure 1, Tables 2 and 4, §5.2–5.5; all metrics in Table 19). We also ran the exact method at raised thresholds, as a robustness-targeted variant, on all three datasets (Table 23). CROCE is so far available only as an arXiv preprint, and its released code and experiments cover numeric features only; on our benchmarks it could run only on GMSC, where 494 of 500 targets are the adverse class, so it would not test recourse. BetaRCE and T-Rex have no actionability constraints. BetaRCE is post hoc: it moves a base method's counterfactual through feature space until enough retrained models agree, and its experiments use numeric data only. T-Rex's gradient version also starts from a base method's counterfactual and moves one-hot categorical features as continuous values, and its nearest-neighbour version returns training points, which can differ in immutable features. Both would act on an actionable base method's output rather than replace it; we did not run them (Table 11 positions all three).

Questions

  • Q1 (distil vs. learn a density). A normalising flow trained directly on each fold (Table 26) reaches lower NLL on Adult and Credit but lower validity (Adult 0.48 vs 0.80 at m=0.2m = 0.2). Distillation keeps the exact circuit as the scorer and gives higher validity.
  • Q2 (plausibility; surrogate quality). LiCE maximises the exact likelihood; PACER follows a surrogate gradient at a small weight (λp=0.1\lambda_p = 0.1), fitted at y=1y = 1, while 72% of Credit targets are y=0y = 0. We measured the deployed surrogate against the exact circuit on held-out test rows: Pearson rr = 0.77–0.93 by dataset (App. C). §7 states the plausibility gap.
  • Q3 (is a MIP needed?). No, and the claim is softened (§3.1): ElliCE's gradient mode also satisfies the shared rule base, and PACER keeps every iterate feasible by projection.
  • Q4 (categorical and discrete features). Categorical changes carry no gradient penalty; they are discouraged by initialising at the factual category, and each costs one unit in the reported distance. Integer features are rounded at decode, categorical ones are decoded by argmax, and binary features are held fixed. Optimising the reported distance instead (Table 25) brings Credit to LiCE's distance (2.08 vs 2.19) with retention 0.94 vs 0.53, though Adult validity falls to 0.68.
  • Q5 (the τ\tau sweep on other datasets). Table 3 now includes τ=0.5\tau = 0.5 and 1.01.0, and the sweep covers all three datasets on identical instances (Table 23). On Adult and Credit the exact method reaches matched retention with counterfactuals 1.2–1.8× closer than PACER's; with HiGHS, every Adult and GMSC solve hit its time limit, and GMSC (five instances) is too small to compare.

Changes that would raise your score

  • Editing for clarity: done (W1).
  • FACE and the sweep in Figure 1: Figure 1 now shows FACE, ElliCE and the exact method's threshold sweep.
  • ElliCE in full, in the main text: done (W2).

Minor issues

  • ff on xx. §3.1 now states that ff is applied to ϕ(x′)\phi(x') and abbreviates f(ϕ(x′))f(\phi(x')) as f(x′)f(x').
  • isclose. Sparsity is now defined in words in §4: the features whose normalised value changes. The exact rule is in App. C: NumPy's isclose (absolute tolerance 10−610^{-6}, default relative tolerance 10−510^{-5}) on the normalised encoding, as in LiCE.
  • Validity weight. It starts at λv=1\lambda_v = 1 (§3.3).
  • Age and income in the abstract. It was an example. In our code, as in LiCE’s metadata, age can only increase and income can change in either direction (GMSC; Adult has no income feature). The revised abstract states the constraints in general terms and no longer mentions income.

I thank the authors for their substantial edits and for addressing my other concerns. The paper has changed significantly (the main text now runs to p. 10). However, the merits of the method remain unclear to me.

The first contribution, that LiCE is fragile at its default threshold, is expected: counterfactuals on the decision boundary are known to be fragile, which the paper itself acknowledges (l. 52–53). It is also addressed directly by raising tau in LiCE, which is what the “exact” sweep in Fig. 1 and Table 23 does.

I struggle to see a clear benefit of the second contribution, PACER. In Figure 1, the sweep over tau in "exact" (i.e., LiCE) seems to dominate PACER. In Table 23 the exact method with a raised tau mostly reaches PACER’s retention with closer counterfactuals. PACER’s remaining advantage is runtime, which depends on the solver in LiCE. If speed is the main contribution, the paper should be framed and evaluated around it.

The benefit of the third contribution, the MLP distilled from the SPN, is also unclear. It approximates an approximation, and even when scored by the same SPN it is often outperformed in NLL by a directly trained flow (Table 26) or by a Gaussian mixture (Table 6).

The paper would benefit from a step back: a thorough rewrite focused on a few key findings, the experiments needed to test them, and a discussion of their broader implications.

Thank you for the follow-up. We agree with much of it, and answer each point.

1. The diagnosis is expected. Agreed: boundary fragility is known (l. 52–53), and raising τ\tau fixes it. What the paper adds is the measurement for the leading exact actionable method (a third to a half of its counterfactuals fail after an ordinary retrain) and a controlled sweep showing that its default threshold is the cause (Table 3). We present this as a finding, not as the main contribution.

2. PACER versus the exact method with a raised τ\tau. You are right that, where the exact solve completes, it reaches PACER's retention with closer counterfactuals; the abstract and §5.4 say so. PACER's case is margin-robust, actionable recourse without a solver:

  • Cost depends on the solver, as you note. With HiGHS, raised-threshold solves took 31–39 s per person on Credit, and every Adult and GMSC solve hit its 300–600 s limit and returned the best solution found (Table 23). With Gurobi, LiCE's published medians at its default threshold are 3.1 s on Credit, 34 s on Adult and 133 s on GMSC, on other hardware; PACER's median is 1–2 s on a 2-core CPU (Table 14). The gap is large on Adult and GMSC and small on Credit.
  • Among solver-free methods, PACER is the closest: about 3× closer than ElliCE on Adult and Credit at validity ≥ 0.99 (Table 19), though ElliCE is faster.
  • Much of the proximity gap comes from PACER's surrogate distance rather than from gradient optimisation. On Table 23's 75 Credit instances, scored against the same retrain, PACER at m=1.0m = 1.0 with the reported distance as objective reaches retention 0.90 at distance 1.98 (validity 0.95). The exact method needs 3.78 for retention 0.86, 4.98 for 0.96, and 2.52 even at its default threshold (Table 23, new row). It optimises its own distance and likelihood inside LiCE's stricter feasibility model, which adds ordered categories and causal links that PACER does not enforce, so the two are not constrained identically. We report this objective as a variant because on Adult it lowers validity to 0.68 and retention to 0.82 (Table 25); a natural next step, which we have not tested, is to down-weight its categorical cost so that validity holds on Adult.

We agree that cost should be the frame; the contributions now state PACER's cost against the exact method (a median of 1–2 s per person, against tens of seconds to minutes).

3. The distilled surrogate. Agreed that it does not improve NLL; the contributions now say its gain is validity, not likelihood. It approximates the scorer itself: the same circuit that LiCE optimises and that scores every likelihood we report, so distillation gives the optimiser a gradient toward that measure without a solver. Against a Gaussian mixture and a flow it raises validity (Adult 0.96 vs 0.70 at m=1.0m = 1.0, Table 6; 0.80 vs 0.48 at m=0.2m = 0.2, Table 26).

4. Focus. We take the point. For the final version we would restructure the paper around three findings: the exact method's fragility comes from its default threshold; margin-robust recourse that satisfies the shared rule base can be computed without a solver, at a price in proximity that we quantify; and a distilled circuit raises validity over the density terms we tested while the exact circuit scores plausibility. We would value your view on whether this framing addresses the concern.

I thank the authors for the detailed and candid reply. My concerns are partly addressed, and I see some merit in the method. I am therefore raising my score to weak reject.

I do not raise it further because the proposed restructuring around three findings would substantially change the paper’s framing and claims. A change of that scope should be reviewed as a whole rather than made after review. I encourage the authors to pursue the reframing; I think it would make for a clearer paper.

Comments on the paper

1 comment

Thank you for the reviews. They prompted new experiments and a more precise statement of our claims. Changes are in red, new results are in Appendix O, and the supplement reproduces the new results. Table numbers refer to the revised PDF.

Bottom line

  • Robustness. At m=1.0m = 1.0, PACER keeps 0.85–1.00 of its counterfactuals under every model update tested (six retrains, a wider MLP, a logistic regression), varying by at most 0.03 across retrains. On favourable targets it retains 0.96 (Adult) and 1.00 (Credit) after one retrain, against 0.55 and 0.65 for LiCE.
  • Actionability. Fully actionable under the shared rule base on all three datasets, like LiCE and ElliCE; no other baseline exceeds 0.71.
  • Cost. 1–2 s per person, without a solver. The exact method with a raised threshold is 1.2–1.8× closer on Adult and Credit, but takes 31–39 s on Credit with HiGHS (LiCE's published Gurobi) and hits its 300–600 s limit on Adult and GMSC.
  • Against ElliCE. About 3× closer on Adult and Credit at validity ≥ 0.99; ElliCE is faster, more robust on Adult and Credit, and closer on GMSC.
  • Scope. PACER is less plausible than LiCE, and the margin matters most on GMSC and under a model change on Credit.

New experiments

  • ElliCE in full, both modes, with its constraints (App. M, Table 19).
  • The exact method with a raised threshold on all three datasets (Table 23).
  • Six retrains per fold, a wider MLP and a logistic regression (Table 22).
  • Retention on favourable targets (§5.2) and acceptance under LiCE's own feasibility model (Table 27).
  • Margin diagnostics (Table 24), the reported distance as objective (Table 25), and a normalising flow (Table 26).

Revised claims

  • Actionability is claimed under the shared rule base. LiCE's stricter model accepts 0.18 (Adult) and 0.76 (Credit) of PACER's counterfactuals; a masking-and-repair variant reaches 1.00 (Table 21). The full ElliCE run replaces our preliminary comparison.
  • Most GMSC and Credit targets are the adverse class (494/500 and 359/500), so there counterfactuals explain decisions (§4).
  • Numeric edits are optimised in raw units, so GMSC's income and Credit's loan amount stay fixed (§7).
  • Validity against LiCE is reported as 0.99 vs 1.00 rather than as a significance test; timing is reported as medians against LiCE's published times (Table 14); Figure 3 and Table 17 (Credit) are recomputed with the density term.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.