- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes ACCORD, a diffusion model that generates recourse conditioned on an individual’s cost function. Its main contribution is a tractable cost prior combining Gamma-distributed feature weights with random preference windows or categorical sets. Conjugacy enables an inverted training sampler: sample a positive target first, then a compatible cost, avoiding repeated cost-dependent searches over positive examples.
Critical strengths
- Significance of considered problem. Serving heterogeneous preferences with one trained model is a practically relevant objective.
- The strongest contribution is the combination of expressive, tractable cost priors with cost-marginalized pairing and posterior cost sampling. Equations 3–7 and Proposition 1(A) provide a clear mechanism for amortizing training-data construction across user preferences.
- The paper evaluates both a synthetic setting with an available target distribution and three mixed-type tabular benchmarks.
Critical weaknesses
- It is not consistently the best method. The paper makes broad superiority claims, but FACE achieves a higher overall score on COMPAS and HELOC in Table 1.
- The test preferences may favor its approach. The main experimental description says user preferences are sampled based on compatibility with available positive examples. Real users may have difficult preferences that do not fit the data. Testing independently chosen preferences would better establish practical usefulness.
- Missing related work or comparisons: The closest personalized amortized method, iCARMA, is discussed but not experimentally compared. Its causal setting makes direct comparison nontrivial, but a controlled comparison on shared capabilities would strengthen the contribution.
- The comparison budgets differ. ACCORD uses 50 generation steps, while BFS and DFS use 16 timesteps with additional search operations. This does not automatically make the comparison unfair, but quality should be compared at similar runtime or computational cost.
Questions for authors
- How should the coverage claims change under the constant-cost counterexample? What additional assumptions connect low coverage to low ESS and meaningful target-estimation error?
- What is the actual candidate set used in experiments?
- How are evaluation preferences generated for each dataset? Please specify the distributions of both weights and windows, the positive data used for posterior sampling, and results under independent preferences.
- What exact reward do guidance baselines use?
5 comments
Q3.2: Evaluation under Independent Sampling
We report results under posterior-sampled costs not to favor our method, but to evaluate performance under a balanced regime of covered (achievable) and low-coverage costs. As discussed in Sec. 3.3.1 (Lines 314–316), costs sampled independently (forward) of the data have little to no coverage in . For instance, only approximately 15%, 10%, and 0% of independently sampled costs are covered on COMPAS, ADULT, and HELOC, respectively. In contrast, posterior sampling under inverted sampling provides a more balanced representation of both regions (Tab. 3). Thus, evaluating on posterior-sampled costs is a sensible choice, as it includes costs for which recourse is achievable on the data manifold.
| Data | Sampling | % of Covered costs sampled |
|---|---|---|
| ADULT | FOW | 9.91% |
| INV | 54.95% | |
| COMP | FOW | 15.25% |
| INV | 85.12% | |
| HELOC | FOW | 0.00% |
| INV | 62.41% |
Table 3: Percentage of covered costs sampled under Forward and Inverted Sampling. On high-dimensional HELOC, Forward Sampling predominantly generates unrealizable costs.
However, following the reviewer's suggestion, we repeat the evaluation under prior-sampled (Forward) preferences, where all methods struggle to satisfy user preferences due to low coverage (Table 11). Among generative methods, ACCORD and FOW satisfy more preferences than guidance-based baselines at lower cost, note that FOW is ACCORD without inverted sampling. FACE and DiCE achieve lower costs but suffer from substantially poorer validity, falling below random on two of the three datasets (both on ADULT and DiCE on COMPAS), making them less effective for recourse. This setting is largely low-coverage, as evidenced by low PRF and only 0–15% covered cost samples under Forward sampling (Table 3). ACCORD is also computationally more efficient to train than FOW, reducing complexity from to (Line 171-173).
| ADULT | COMPAS | HELOC | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | PRF ↑ | InDist+ ↑ | Val ↑ | Cost ↓ | PRF ↑ | InDist+ ↑ | Val ↑ | Cost ↓ | PRF ↑ | InDist+ ↑ | Val ↑ | Cost ↓ | |||
| RERD | 0.02 | 0.81 | 0.91 | 24.10 | 0.11 | 0.74 | 0.97 | 3.52 | 0.00 | 1.00 | 0.95 | 33.68 | |||
| BFS | 0.04 | 0.80 | 0.91 | 24.33 | 0.20 | 0.74 | 0.97 | 3.44 | 0.00 | 1.00 | 0.95 | 34.31 | |||
| DFS | 0.01 | 0.79 | 0.91 | 24.98 | 0.14 | 0.70 | 0.97 | 3.66 | 0.00 | 1.00 | 0.95 | 34.07 | |||
| SVDD | 0.02 | 0.80 | 0.91 | 24.41 | 0.11 | 0.69 | 0.96 | 3.58 | 0.00 | 1.00 | 0.95 | 34.14 | |||
| FACE | 0.10 | 0.74 | 0.28 | 15.24 | 0.46 | 0.85 | 0.82 | 1.44 | 0.00 | 0.99 | 0.64 | 20.53 | |||
| DiCE | 0.11 | 0.70 | 0.32 | 15.85 | 0.40 | 0.71 | 0.45 | 2.03 | 0.00 | 0.93 | 0.51 | 22.65 | |||
| FOW | 0.08 | 0.76 | 0.90 | 17.17 | 0.27 | 0.75 | 0.98 | 2.32 | 0.00 | 0.99 | 0.94 | 30.97 | |||
| ACCORD | 0.09 | 0.76 | 0.89 | 17.35 | 0.23 | 0.75 | 0.99 | 2.74 | 0.00 | 1.00 | 0.96 | 30.45 |
Table 11 Comparing ACCORD against baselines across datasets under independently sampled preferences. FOW is ACCORD without inverted sampling.
Further, on ADULT, we use to partition preferences into covered and low-coverage regions and evaluate all methods separately in each (Table 12). ACCORD achieves one of the best cost–validity trade-offs in both regions (Fig. 3), whereas methods such as FACE and DiCE achieve lower costs at the expense of substantially poorer validity. These results demonstrate ACCORD's practical usefulness even under independent sampling and in low-coverage settings. We have incorporated these results into the paper (Fig. 3, Sec. 4.5, App. G.6).
| Covered | Region | Low-coverage | Region | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Method | PRF ↑ | Val ↑ | InDist+ ↑ | Cost ↓ | PRF ↑ | Val ↑ | InDist+ ↑ | Cost ↓ | ||
| RERD | 0.19 | 0.92 | 0.84 | 14.35 | 0.01 | 0.91 | 0.81 | 24.31 | ||
| BFS | 0.37 | 0.93 | 0.84 | 11.93 | 0.03 | 0.91 | 0.80 | 24.63 | ||
| DFS | 0.20 | 0.91 | 0.87 | 16.03 | 0.01 | 0.91 | 0.79 | 25.40 | ||
| SVDD | 0.20 | 0.96 | 0.77 | 13.60 | 0.02 | 0.91 | 0.80 | 24.76 | ||
| FACE | 0.42 | 0.42 | 0.76 | 9.14 | 0.09 | 0.28 | 0.74 | 15.44 | ||
| DiCE | 0.49 | 0.42 | 0.76 | 7.35 | 0.10 | 0.32 | 0.69 | 16.14 | ||
| FOW | 0.52 | 0.94 | 0.78 | 7.88 | 0.06 | 0.90 | 0.75 | 17.58 | ||
| ACCORD | 0.56 | 0.92 | 0.78 | 6.35 | 0.07 | 0.89 | 0.75 | 17.85 |
Table 12: ADULT under independently sampled preferences.
Addressing the weaknesses
FACE vs ACCORD
We acknowledge that FACE achieves a higher RScore on COMPAS and HELOC. However, RScore is a combined metric, that possibly does not give as much importance as it should to validity. Table 1 shows that FACE's validity is poor (0.85 and 0.65 vs. 0.99 and 0.96), making it ineffective for recourse. Its higher RScore is partly driven by lower recourse cost. But low cost without validity defeats the very purpose of recourse. A second limitation of FACE, is that it assumes access to the training data and searches within it during inference. This results in significantly higher computate overheads. We show the comparison below. FACE needs 25.17 and 57.01 ms/query on COMPAS and HELOC, vs. 1.24 and 0.57 for ACCORD (Fig 7: RScore vs Time). That makes it roughly 20× and 100× slower, since it searches at inference time. Worse, as we added a new RQ5 covering this comparison (App. G.7, Line 1550).
| Method | ADULT (ms/query) | COMPAS (ms/query) | HELOC (ms/query) |
|---|---|---|---|
| RERD | 17.00 | 14.78 | 23.20 |
| BFS | 4.49 | 7.41 | 6.43 |
| DFS | 1.40 | 5.92 | 2.10 |
| SVDD | 3.90 | 5.02 | 5.54 |
| DiCE | 523.03 | 696.41 | 132.84 |
| FACE | 2237.08 | 25.17 | 57.01 |
| ACCORD | 0.41 | 1.24 | 0.57 |
Table: Inference time comparison of ACCORD against baselines.
Comparison with iCARMA
We thank the reviewer for this suggestion. We have added a controlled comparison with iCARMA (App. G.1.1, Line 1330).
Setup. We compare against ACCORD Version 1, whose preference-weighted cost corresponds directly to iCARMA's individualized action cost. The preferred-range cost of the later version of accord has no straightforward counterpart in iCARMA. Both methods share the same preference weights, classifier, train/test individuals, and parameter budget. As also mentioned by the reviewer, iCARMA needs a causal graph, since our datasets lack a it, we built a plausible causal ordering for each and derived an SCM from it. iCARMA does not report its validity-term weight , so we tested . We show , which has the highest average validity; full results are in App. G.1.1.
| ADULT | COMPAS | HELOC | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | InDist$^+$ ↑ | Val ↑ | Cost ↓ | InDist$^+$ ↑ | Val ↑ | Cost ↓ | InDist$^+$ ↑ | Val ↑ | Cost ↓ | ||||||
| iCARMA () | 0.35 | 0.59 | 2.12 | 0.71 | 0.46 | 0.62 | 0.87 | 0.53 | 1.05 | ||||||
| ACCORD-v1 | 0.80 | 0.89 | 3.23 | 0.74 | 0.99 | 0.95 | 1.00 | 0.96 | 2.39 |
Findings. ACCORD achieves substantially higher validity on every dataset and higher plausibility (InDist$^+$), most notably on Adult. iCARMA finds cheaper interventions (lower cost), but many fail to change the outcome (only 0.45-0.60 validity). Tuning does not close the gap: raising improves iCARMA's validity at the cost of higher action cost and lower plausibility. This follows from its objective, where plausibility is only a soft regularizer. The iCARMA authors report the same validity–plausibility trade-off. ACCORD instead samples recourse from the positive data distribution, keeping validity and plausibility high.
Comparison budget discrepancy
To compare at a matched budget, we re-ran all guidance baselines (RERD, BFS, DFS, SVDD) with 50 sampling steps, the same as ACCORD. At this matched budget, ACCORD has the highest RScore and PRF and the lowest cost on all three datasets.
| ADULT | COMPAS | HELOC | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | RScore ↑ | PRF ↑ | Val ↑ | Cost ↓ | RScore ↑ | PRF ↑ | Val ↑ | Cost ↓ | RScore ↑ | PRF ↑ | Val ↑ | Cost ↓ | |||
| RERD | 0.91 | 0.18 | 0.91 | 16.42 | 1.33 | 0.61 | 0.98 | 0.88 | 1.03 | 0.08 | 0.95 | 9.21 | |||
| BFS | 1.10 | 0.39 | 0.90 | 13.16 | 1.54 | 0.86 | 0.97 | 0.29 | 1.14 | 0.20 | 0.96 | 8.18 | |||
| DFS | 0.95 | 0.23 | 0.90 | 16.62 | 1.49 | 0.79 | 0.98 | 0.51 | 1.06 | 0.12 | 0.95 | 8.82 | |||
| SVDD | 0.99 | 0.27 | 0.91 | 14.64 | 1.28 | 0.59 | 0.91 | 0.85 | 1.05 | 0.10 | 0.96 | 8.93 | |||
| ACCORD | 1.27 | 0.57 | 0.90 | 7.01 | 1.61 | 0.91 | 0.99 | 0.17 | 1.24 | 0.30 | 0.96 | 4.81 |
Table 14: ACCORD against the guidance baselines with every method run at 50 sampling steps.
Response
Thank you very much for your valuable feedback. We have uploaded a revised version of the paper incorporating your feedback. All the line numbers reported below are as per the revised version of the paper.
The reviewer is right to point out that low coverage does not lead to low ESS in the constant-cost case, as stated below Equation (15). For this statement to hold, we additionally assume a normalization of the cost so that minx+∈D+CU(x+,x)=0, i.e., coverage is read relative to the cheapest recourse available in D+. The bounds in Equation (15) are correct since both ESS and Z(x,U)⋅eCˉ(x,U) are shift invariant.
Across all experiments, the candidate set N(x) in Algorithm 1 is the entire positive-class training split, i.e., N(x)=Dh+. We do not apply any truncation (e.g., Top-(v) selection) to construct the candidate set. We have added this detail below Algorithm 1 at Line 950. If the reviewer had a different candidate set in mind, we would be happy to clarify.
The evaluation preferences for each dataset are generated with the same posterior sampling as in training, using the test split of the dataset [Lines 406–407].
Distribution of weights and windows:
Positive data used for posterior sampling: The positive test instances, reweighted by M(x+,x)=EU∼P(CU)[e−CU(x+,x)], where x+ is a positive test instance and x is the rejected (negative) test instance [Eq. 3, Lines 162–164].
Based on the prior work, all baselines use a model trained for Pθ(x+∣x)∝Ph+exp−λˉ⋅ϕ(x,x+), on which we apply the guidance reward exp−CU(x,x+)+λˉ⋅ϕ(x,x+), the second term is used to offset the tilt of the baselines. This makes the final sampling distribution as Ph+exp−λˉ⋅ϕ(x,x+)∗exp−CU(x,x+)+λˉ⋅ϕ(x,x+)=Ph+exp−CU(x,x+). We have explicitly added this in App. F.2.1 (Line 1296).