- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposed a new model for tabular fairness. They designed it as a tabular transformer in which each feature gets a single scalar embedding. Attention is computed directly from these scalars with no query, key or value projections, using a single head and no residual connection around attention. The authors argue this makes the attention matrix readable as pairwise feature dependencies. On top of this they propose CAR, a regularizer that penalizes changes in the sensitive feature’s attention column when the sensitive attribute is switched to other categories, plus a cheaper single-pass version for training. A theorem states that a small CAR loss bounds the change in prediction when only the sensitive attribute changes. Experiments on BAF, an insurance dataset and Adult show near-zero counterfactual gaps for FCorrTransformer+CAR with a small drop in AUROC and AUPRC compared to FT-Transformer.
Critical strengths
- The most valuable point of this paper is the evidence showing regularizing attention in a standard transformer does not make predictions invariant, because information about the sensitive attribute bypasses attention through residual paths and projections. The residual-connection ablation (Tables 7 and 8) is a useful, honest observation. FT-Transformer+CAR retains a clear gap, and reintroducing the residual in FCorrTransformer brings the gap back. This is a fair caution for the attention-based debiasing literature. Although by itself it is a modest contribution.
Critical weaknesses
-
There are issues in the evaluation pipeline that hurt paper claims. The paper cites Kusner et al. for counterfactual fairness, but the definition in Section 2 and the metrics in Section 4 only switch the sensitive column while keeping every other feature fixed. The authors themselves call the result “direct counterfactual fairness” after Theorem 1. This only checks whether the model reads the sensitive column, and any model that ignores the column satisfies it perfectly. Kusner’s definition relies on a causal model where features influenced by the attribute also change, and that indirect path is exactly what matters in practice. The paper’s own synthetic setup is built around indirect discrimination (X1 affects y only through X2), yet the metric cannot measure it. The same applies to the insurance motivation, since the regulations cited in the introduction are mainly about proxy discrimination. Simply dropping the sensitive feature is therefore the natural baseline. The paper reports it in the appendix (Table 8, “+ Rem”), and it also gives zero gaps with essentially the same AUROC and AUPRC as CAR. Group fairness does not improve much either: in Table 3, test DPD and EqOdd for FCorrTransformer+CAR remain high (0.106 and 0.659), and the plain FFN baseline is actually lower on both. I do not see what CAR offers beyond removing the column.
-
There are some real concerns about the interpretability claim and the baseline comparison. Because each embedding is a single scalar, the attention score between two features is just the product of their two scalars. The heatmaps therefore show how the learned scalars co-vary, not how the features depend on each other. Figure 2 illustrates this. X1 and X3 are independent by construction, yet their entry (-0.98) is larger in magnitude than the X1-X2 entry (0.37) that the text describes as the strongest dependency. X2 and X3, which together determine y, have the largest off-diagonal entry, and the text calls these negative values negligible. A partial overlap with LightGBM feature importance is weak support. On the experimental side, the only comparisons are generic tabular models with and without CAR. No established fairness method is included, such as adversarial debiasing, the reductions approach, FairPFN or causal counterfactual-fairness methods, so it is hard to place the result.
Questions for authors
- What does CAR achieve that dropping the sensitive feature does not? Table 8 suggests the two are equivalent on every metric reported. It would help to show the learned embedding of the sensitive feature across categories after CAR training.
- Can you evaluate under a causal counterfactual, where features influenced by the sensitive attribute also change? The synthetic dataset is a natural place for this, since X2 depends on X1.
- How should the Figure 2 heatmap be read as feature dependencies, given that the independent pair X1 and X3 receives a large entry?
- Can you add at least one established fairness baseline and LightGBM to Tables 2 and 3, and state which data were used for hyperparameter tuning and threshold selection?