- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper looks at relational constraints in counterfactual (CF) explanations and proposes MARBLE to generate CFs that satisfy them alongside the existing requirements. The method builds candidates from nearest unlike neighbors, repairs constraint violations, and refines them with a constrained multi-objective search, and it is evaluated on four tabular datasets across several model families.
Critical strengths
- The paper looks into a crucial problem in counterfactual generation;
- The empirical study covers a broad range of topics.
Critical weaknesses
- It is hard to see the contribution relative to existing work. There is no comparison with SCM or causal-recourse methods, so the empirical comparison only covers methods that do not model relational constraints. There is one direct and close competitor CFM-FW but is not evaluated since the code is not public, which is not a convincing reason in this era of strong AI-assisted coding.
- The paper itself has relational constraints highlighted in the title, while it has limited discussion of what relational constraints should look alike, how they should be defined clearly, or other critical issues.
- More importantly, MARBLE does not seem tied to relational constraints closely. It looks like a generic constrained optimizer, so the same pipeline could be applied to any task with fitted constraint set, not just CF generation with relational constraints. The paper does not show what is specific to relational constraints here, so it is unclear why MARBLE is the chosen method for the task.
Questions for authors
- There are math symbol typos, e.g. (should be ?) and .
- "Nearest unlike neighbours (NUNs)" is not a standard concept; please define it explicitly.
- In Eq. (1), o1 and g0 look like the same quantity. Is the constraint redundant with the first objective?
- Eq. (4) needs more justification: why do we design this decay schedule (from to zero)?
- In Table 1, Val and CS are identical for MARBLE. If they are not mis-recorded, it needs an explanation. The same happens in Table 3 again.
- A few pages appear to be changed in rendering, which makes the formatting inconsistent.
- The writing is somewhat disjointed; it lacks coherence between some paragraphs/sections.
The rebuttal discussions include new experiments and clarifications to my concerns. However, the contributions are still not clear to me so that I change my score to 2, weak rejection.
4 comments
[W2]: Discussion on relational constraints
We agree that the paper should define relational constraints formally, and we will add this definition to Sec. 3.1 with the full grammar in a new appendix section. A relational constraint is a formula over two or more features. Its terms are arithmetic expressions over features and constants, including sums, products, powers, logarithms, and safe divisions with a fallback value when the denominator is zero. These terms are compared with , , or (optionally within a tolerance ) and combined with and . Single-feature bounds are boundary constraints and are handled separately. Each has a violation function that is zero exactly when holds (see our answer to Reviewer 2vi2, [W2, Q2], for the per-construct forms).
The five subtypes in Sec. 2 all fall within this grammar. For example, LCLD contains:
open_acc <= total_acc— linear inequality(term == 36) || (term == 60)— booleanratio_open_acc_total_acc = open_acc / total_acc— non-linearratio_pub_rec_bankruptcies_pub_rec = safediv(pub_rec_bankruptcies, pub_rec, -1)— safe division- Loan-amortization formula linking installment, loan amount, interest rate, and term (non-linear, )
In practice these constraints come from domain rules and from feature engineering. Derived features computed by the deployment pipeline must stay consistent with their inputs (App. A.2).
[W3]: Contribution on relational constraints
Regarding MARBLE being a "generic" optimizer: MARBLE is explicitly tied to tabular relational constraints. Stage 1 (Admission) uses Nearest Unlike Neighbors (NUN) to assemble candidate features directly from observed tabular data manifolds. Stage 2 (Repair) uses a directed greedy repair operator designed specifically to parse and repair tabular functional dependency graphs. As our ablation study (Table 24) shows, removing these tabular-specific mechanisms causes constraint satisfaction to collapse on complex datasets like CTU-13, proving MARBLE is highly specialized for tabular constraints.
[Q1]: Math typo
We thank the reviewer for catching the math typos. We will fix all typographic errors in the revised manuscript.
[Q2]: Definition of NUN
"Nearest Unlike Neighbor (NUN)" is a concept in counterfactual literature which we cite in the manuscript (Wexler et al., 2019). Furthermore, we defined it in Algorithm 1, Line 5 as . We agree with the reviewer, that it should be in the main text of Section 3.2 and we will update the manuscript accordingly.
[Q3]: Clarification on and
and serve entirely different and non-redundant purposes. While is a hard feasibility boundary constraint requiring the counterfactual to cross the decision boundary (e.g., confidence ) to be considered valid, is a Pareto optimization objective that pushes valid counterfactuals deeper into the target class to maximize margin robustness.
Without , invalid solutions would dominate the Pareto front (by having 0 sparsity and 0 distance). Conversely, as shown in our Objective-space ablation (Table 26), removing causes target confidence to collapse (e.g., from on URL). They are not redundant.
[Q4]: Justification on Eq. 4
Equation 4 defines an -constrained decay schedule that is a standard and necessary technique in evolutionary optimization. Starting with a larger allows the search to explore slightly infeasible regions, which is crucial for crossing the decision boundary early in the search. Decaying forces the population to strictly converge onto the feasible manifold by termination.
[Q5]: Clarification on Val and CS in Table 1
In Table 1, Val and CS are identical for MARBLE because MARBLE's archive design guarantees that 100% of its valid counterfactuals satisfy all constraints and MARBLE will fail to generate a counterfactual if it is not valid and do not satisfy all the constraints.
[Q6, Q7]: Rendering and writing
We will address the formatting and rendering issues (such as the varying table font sizes due to \resizebox commands across the appendix) and improve paragraph transitions for better overall coherence.
Thanks for the response
We thank the authors for the response.
After adding the new experiments, the experimental part looks way better. Also, a few discussions can really help the paper be self-contained and more clear to a broad audience. Thus, I will raise my score to 2.
However, the contribution part, in my opinion, is still weak. I personally still cannot see the non-triviality of the framework. Or, say, the strong indication of why this framework is a breakthrough fit for the problem is not clear.
Thank you for your constructive feedback and for raising your score after reviewing our new experiments and clarifications. Please see our Summary above for how we will foreground MARBLE’s constrained decoding formulation in the Introduction and Section 3 to make the methodological contribution unmistakable in the final manuscript.
[W1]: Comparison against CFM-FW
We did include a causal-recourse method in our benchmark: FastAR (Verma et al., 2022) generates recourse with a structural causal model, so its actions respect the causal relations it encodes. Despite this, FastAR reaches a coverage CS of 0.00 on both CTU-13 and LCLD (Table 1). A causal model does not express the inequalities, boolean conditions, and algebraic identities (e.g., derived ratio features) that these datasets require. More generally, SCM-based methods need a causal graph and structural equations, which are not available for these datasets and cannot in general be recovered from observational data (Karimi et al., 2020; Uhler et al., 2013). MARBLE instead takes the domain constraints as stated by the practitioner.
AI-assisted tools can facilitate reimplementation, but they cannot verify details omitted from the published paper. We usually follow the practice of avoiding baseline reimplementations when the official code, exact hyperparameter settings, or data preprocessing pipeline are unavailable, since undocumented choices can affect baseline performance and bias the comparison. Although we understand the value of this comparison, we implemented CFM-FW based on the paper’s description. We have also thus selected and included new and most-recent baselines in our benchmark suggested by other reviewers: i) CERTIFAI (Sharma et al.) (a heuristic-search based CE method), ii) MCCE (Redelmeier et al.) (an instance-based method), and iii) L2C (Vo et al.) (a generative method).
The results are shown in the table below and will be added to the manuscript.
As shown in the table, while CFM-FW's generic NSGA-II search can find nearby points that flip the classifier (scoring high Validity and low Proximity), it systematically fails to satisfy the relational constraints on complex datasets (e.g., CS = 0.05 on WIDS, 0.09 on LCLD, and 0.10 on CTU-13). It cannot randomly mutate its way onto strict mathematical manifolds without MARBLE's directed repair.