- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
FlowCF is a model-agnostic, amortized method for sparse counterfactual generation on mixed-type tabular data. It combines a mixed-type flow-matching transport with a feature-wise gating network that can keep selected numerical features exactly unchanged. The method is trained in three stages for transport, sparsity, and proximity, and is evaluated on six tabular datasets against six baselines.
Critical strengths
Exact numerical sparsity by construction. The gating mechanism can leave selected features exactly unchanged, which is useful for actionable counterfactual explanations.
Efficient and model-agnostic inference. FlowCF is amortized and does not require black-box queries at explanation time.
Critical weaknesses
The headline sparsity gain is partly by construction. FlowCF reports much lower exact numerical sparsity than the baselines, but the paper itself proves that continuous generators almost surely modify every real-valued feature under an exact-equality metric. I would like to see results under several -sparsity thresholds to assess whether the practical advantage remains large.
The proximity gain is not fully disentangled from sparsity. The proximity metric averages displacement over all numerical features. Since FlowCF sets many feature changes exactly to zero, this automatically improves proximity. Reporting displacement only over changed features would make the comparison more informative.
Questions for authors
Can the authors report numerical -sparsity at several thresholds and compare against the baselines? Can the authors report the average displacement per changed feature, rather than averaging over all numerical features? Can the authors provide a quantitative ablation for the three training stages, especially the effect of gating and the final joint fine-tuning?
4 comments
Thank you again for the clear and helpful review. We wanted to check whether the new tolerance-based sparsity results, displacement per changed feature, and ablation studies address your concerns. If anything remains unclear before the discussion closes, we would be very happy to clarify.
Thank you for the additional experiments and detailed response. I appreciate the authors' efforts to address my concerns. However, the new results confirm that the proximity improvement mainly comes from changing fewer features, rather than reducing the displacement of individual changed features. The underlying sparsity–displacement trade-off therefore remains unresolved, particularly on continuous-only datasets. Therefore, I will maintain my original rating.
Thank you for engaging with our response. We agree that Prox Num combines how many numerical features change with how far they move, and that against dense methods, the gain of FlowCF comes mainly from changing fewer features. We see this as the intended behaviour of a sparse method, and with the points below, we hope to convince you of this and that you would consider raising your score.
We would also like to recall Table 1: every baseline queries the black box when it explains an instance, and all except DiCoFlex optimise or search each instance anew. FlowCF is the only method with all the properties of Table 1. It only needs black-box labels during training, explains a new instance in a single forward pass, and still achieves the best numerical sparsity and proximity on average.
-
Prox Num vs. displacement per changed feature. Prox Num is the standard L1 proximity, the type of distance that the CARLA benchmarking framework reports. Wachter et al., who introduced counterfactual explanations, chose an L1 distance specifically because it favours counterfactuals that change few features. Unchanged features enter Prox Num with their true change of zero. Displacement per changed feature instead leaves out exactly the features that a sparse method does not change, and therefore favours dense changes. We would like to make this point with a simple example. For a linear classifier that weighs features equally, moving one feature by or all features by crosses the boundary with the same total change, but the dense counterfactual has a times smaller displacement per changed feature while asking the user for changes instead of one. In general, the two factors pull in opposite directions: the fewer features change, the further the changed ones typically have to move. We agree that this measure was missing from our original results, and we added it as you suggested (Table 5, and per dataset in Table 7). However, judging FlowCF on this measure alone would, in our opinion, be unfair to any sparse method, as it ignores exactly the features such a method leaves unchanged.
-
Both factors (numerical sparsity and displacement) together: FlowCF is Pareto-optimal on every dataset. Nonetheless, we added Figure 7 (Appendix C.9), which plots numerical sparsity against your measure, using only results already in the paper. FlowCF lies on the Pareto front on all six datasets and on the mean, and it is the only method that does so on every dataset. NICE and DiCE lie on it for four of the six datasets, and DiCoFlex, methodologically our closest competitor, for none. No baseline changes fewer features and also moves them less. Every baseline that moves the changed features less changes at least 86% of the numerical features, except NICE on the two continuous-only datasets.
-
Continuous-only datasets: FlowCF vs. NICE. With Prox Num (Table 4), FlowCF and NICE are the only Pareto-optimal methods on MAGIC and HTRU2, and neither dominates the other (table below). NICE moves less, but it relies on stronger access: it queries the black-box scores and searches each instance anew. With the same label-only access as FlowCF, the validity of NICE drops from 1.000 to 0.763, against 0.970 for FlowCF (Appendix C.3).
| Features changed (MAGIC / HTRU2) | Prox Num (MAGIC / HTRU2) | Access at explanation time | |
|---|---|---|---|
| FlowCF | 15.8% / 32.6% | 0.327 / 0.616 | none (one forward pass) |
| NICE | 22.8% / 41.7% | 0.275 / 0.509 | black-box scores, per-instance search |
| DiCE | 99.9% / 100% | 0.369 / 0.710 | gradients, per-instance optimisation |
-
Across all datasets. On average, FlowCF changes the fewest numerical features (0.159) with the lowest Prox Num (0.229, Table 3), and on every dataset, it changes fewer features with a smaller Prox Num than every baseline except NICE. Compared with NICE, it also moves each changed feature less on average (1.2 vs. 1.7 standard deviations, Table 5). Step 3 shortens the retained changes directly: it lowers Prox Num from 0.298 to 0.229 at nearly unchanged sparsity (Appendix C.4).
-
An adjustable operating point. Like other counterfactual methods, FlowCF is tunable. NICE, for example, comes in variants that favour sparsity, proximity or plausibility (NICE-sparsity, NICE-proximity and NICE-plausibility). In FlowCF, a four times larger lowers Prox Num from 0.229 to 0.183 and numerical sparsity to 0.139 (Appendix C.5).
We hope this shows the trade-off as a deliberate and adjustable operating point of FlowCF rather than an unresolved limitation. Thank you again for the constructive review.
Response to Reviewer Lt7G
We sincerely thank the reviewer for the clear and helpful review, and for recognising the value of exact numerical sparsity and of amortised, model-agnostic inference. Your suggestions made our evaluation more transparent.
W1 / Q1. ϵ-sparsity at several thresholds. Thank you. We agree that this is essential to assess the practical advantage. The advantage holds at every threshold. Table 5 (Section 5.1) reports numerical sparsity exactly, at the precision each feature is recorded in, and at ϵ=0.01,0.1,0.5 training-set standard deviations, and Appendix C.1 gives per-dataset results. Baselines do benefit (DiCE: 0.897 to 0.416 at ϵ=0.1), but FlowCF remains the sparsest method on average at every tolerance (0.146 at ϵ=0.1, against 0.218 for the next best method, NICE, a new baseline suggested by Reviewer 6WFu) and is first on five of six datasets.
W2 / Q2. Displacement per changed feature. Thank you, this is a good suggestion. We added it as the last column of Table 5. Per changed feature, FlowCF moves 1.2 standard deviations, as much as DiCE and TABCF (baselines: 0.8–1.2), and less than NICE (1.7), the other exactly sparse method. We agree with your point: the proximity gain comes mainly from changing fewer features, which is what the gate is designed for. Section 5.1 now states this.
Q3. Ablation of the three training stages. Thank you. We agree that this was missing, and we added it in Appendix C.4 (Table 10), summarised in Section 5.1. Step 1 alone changes every numerical feature (sparsity 1.000, Prox Num 1.998, validity 0.943). The gate (Step 2) gives exact sparsity (0.155) and raises validity to 0.957. The joint fine-tuning (Step 3) reduces Prox Num from 0.298 to 0.229 and raises validity to 0.970. The table also ablates the main components (multiple heads, pairing, validity and drift terms), and Appendix C.5 varies the hyperparameters.
Thank you again for the helpful feedback. We hope the revision addresses your concerns, and we would be glad to answer any further questions.