- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents FLARE to replace a network’s prediction head with a first-order Takagi–Sugeno fuzzy layer: Gaussian antecedents, softmax-normalised firing and local linear consequents. It adds an L1 penalty on the consequents and an entropy penalty on rule usage, and explains predictions with Integrated Gradients through the whole model. It reports accuracy close to linear heads on four neural tasks, attribution that beats a random ordering against lesion masks (BUSI) and sentiment spans (SST-2), and reduced effective rule counts on Iris and Digits.
Critical strengths
-
The paper is straightforward. It states that the entropy term is inactive in the main experiments, that the accuracy comparison is confounded, that the faithfulness results depend on protocol, and that the single-cell analysis explains a different model.
-
The statistical analysis is thorough and reliable, using multiple methods to compare results, estimate uncertainty, and clearly explain the limitations of statistical testing.
-
The Grad-CAM++ control provides a useful check to see whether the results remain similar across different output classes and when the relationships between variables are randomized.
Critical weaknesses
-
The claimed combination is never tested. In all four neural experiments the head has K = 2 rules, with centres fixed at class means, widths fixed at 2, and only the consequents trained.
-
The parsimony objective and learned antecedents appear only on Iris and Digits, on raw features with no learned representation.
-
The introduction criticises post-hoc methods because “the representation and the resulting rule geometry are already fixed”, but that describes the paper’s own main experiments.
-
The fuzzy head in the main experiments is nearly a linear head. With two rules and equal isotropic widths, the firing weight reduces to a fixed logistic function along the class-mean difference. The head is then two linear models blended by a fixed gate. No rule from a neural experiment is shown, and it is unclear why this is more inspectable than the single linear head it replaces.
-
Integrated Gradients is applied to the whole predictor, not to a rule, so the rule structure plays no role in the explanation. The only baseline is a random ordering, which any gradient method on any trained classifier should beat.
-
The novelty is limited and prior work is missing such as a differentiable, trainable Takagi–Sugeno layer with Gaussian memberships. The layer is also a Gaussian-gated mixture of experts or normalised radial-basis network. The log-space firing, L1 and entropy terms are standard.
-
Evaluation uses only correctly classified samples (about 35 images and 150 sentences per seed). Integrated Gradients uses 12–16 steps, with no completeness error reported. The token-level L2 norm discards sign. Whole-pixel deletion is not significant on BUSI and is worse than random on PneumoniaMNIST.
-
The parsimony study is small and confounded. It uses only Iris and Digits. The L1 and entropy terms are switched on together, so their effects are not separated.
-
The accuracy comparison lacks the obvious control. A linear or logistic head fitted post hoc on the same frozen, standardised representation would isolate the effect of the head. The single-cell experiment does not bear on FLARE. It explains the base Transformer, and its reference gene signature comes from the same line of work.
Questions for authors
- How do the attribution metrics compare between the fuzzy and linear predictors on the same samples?
- What happens with learned antecedents and K greater than the number of classes on BUSI and SST-2, or with end-to-end training?
- In the Iris example, how is “THEN versicolor” derived from an affine consequent? The rule pairs high sepal width and low petal width with versicolor, which looks odd.
5 comments
W4, W5, and Q1: Near-linear head and attribution controls
W4. The fuzzy head is nearly a linear head.
A. The algebra is exactly right: for σ = 2, , where z̃ is the standardised representation (i.e. two affine maps blended by a fixed one-dimensional logistic gate), which contains the linear head as a special case.
B. New: the gate is unsaturated (w₁ ∈ [0.05, 0.95]) on 36% of BUSI test images, and the gradient of the margin has a cosine similarity of 0.71 with that of a linear head; on SST-2, the values are 12% and 0.94 (99.7% identical predictions). On BUSI, the head is therefore not linear in practice, yet this yields no gain in accuracy or attribution (W5, W9).
C. The class purity of the rules governing test images is 0.77 for the submitted BUSI head and 0.93 for the selected learned head (a single rule would score ≈0.83 on BUSI, the majority-class rate), and 0.94 on SST-2. With three classes on BUSI, the two rules within the benign class separate lesions by echogenicity, a BI-RADS descriptor (lesion-surround contrast: AUC 0.83), not by size, position, or shape; two-cluster k-means on the same representations recovers the same axis (AUC 0.86), so the rules make this structure explicit. We also explained rule membership itself (the IF part) in input space. Changing only the lesion's grey level moves images between the two benign rules in the expected direction (standardised effect 0.66 [0.56, 0.76], 8 seeds). Still, the membership maps do not concentrate on the lesion more than the class maps, and two-cluster k-means on the same representations does at least as well. The structure therefore belongs to the representation.
D. We accept that in this regime the head is not meaningfully more inspectable than a linear head. The revised manuscript shows the rules of a seed chosen in advance, with per-rule maps and exemplars (Figure 5).
W5 and Q1. IG is applied to the whole predictor, and the only baseline is a random ordering. Both points are correct. We therefore compared FLARE with a linear head trained with the same procedure on the same standardised representations and samples (new evidence; FLARE minus linear head):
| Pointing-game accuracy (FLARE − linear) | IoU (FLARE − linear) | Accuracy, FLARE / linear | |
|---|---|---|---|
| BUSI | −0.088 [−0.136, −0.040] | −0.030 [−0.038, −0.022] | 0.927 / 0.927 |
| SST-2 | +0.000 [−0.009, +0.009] (+0.004) | −0.000 [−0.004, +0.005] (+0.002) | 0.947 / 0.946 |
A. BUSI: the fuzzy head localises worse. On unstandardised representations, the gap vanishes, and every head matches the linear classifier trained jointly with the backbone (the submission's comparator; IoU 0.218-0.220 vs 0.217). A further control on linear heads shows that the difference between standardised and unstandardised representations comes from centring the head's input under our fitting procedure, not from its scale: it persists on the post-ReLU representations (+0.027 [+0.019, +0.036]), shrinks to +0.006 with a converged logistic regression, and is absent on SST-2. Randomising the fuzzy consequents does not reduce localisation (IoU, trained minus randomised: −0.008 [−0.022, +0.007]), and the channel-mean feature map (IoU 0.259) and a fixed centre prior (0.194), which do not depend on the head, exceed FLARE's IoU (0.158), although FLARE is better on pointing-game accuracy, attribution mass, and deletion. On unstandardised representations, randomising the consequents does reduce localisation (IoU, trained minus randomised: +0.021 [+0.011, +0.034]).
B. SST-2: the fuzzy head is equivalent to the linear head; a fixed sentiment lexicon localises better (IoU 0.381 vs 0.259) but is less faithful (deletion 0.670 vs 0.624, insertion 0.802 vs 0.853). The localisation in Table 2 thus reflects the composed predictor and is dominated by the representation.
C. Rule-level IG: IG is linear in the explained function, and the margin is m = Σ_k g_k with g_k = w_k (f_{k,c} − f_{k,c̄}), where c̄ is the other class, so IG(m) = Σ_k IG(g_k) exactly. On BUSI, about two rules each carry > 10% of the attribution, and the gate switches the dominant rule along the integration path. In an exploratory analysis, the dominant rule's map localises better than the whole-predictor map (IoU 0.190 vs 0.158, +0.031 [+0.020, +0.044]), matching the linear head; unregularised learned heads show no such gain. For the submitted and the selected learned SST-2 heads, the decomposition is degenerate (77-100% of the attribution in one rule).
D. A pre-registered replication on new splits and seeds confirms the gain (submitted head +0.022 [+0.015, +0.029], p = 0.002; selected learned head +0.018 [+0.011, +0.026]); the pointing-game gain does not replicate (p = 0.13). Heads fitted on unstandardised representations localise better still, so the decomposition shows which rule carries the evidence rather than improving localisation.
W6 and W7: Prior work, sample selection, and IG settings
W6. Novelty and missing prior work. The head can be seen as a first-order ANFIS (Jang, 1993), i.e., a normalised Gaussian RBF network with local linear models (Jang & Sun, 1993; Hunt et al., 1996), or a mixture of linear experts with Gaussian gating (Jacobs et al., 1991; Xu, Jordan & Hinton, 1994); fuzzy heads on CNN features with gradient saliency have been proposed (Yeganejou et al., 2020; 2023). Log-space firing is standard (Cui et al., 2021), the ℓ1 term is a lasso penalty, and the rule-usage entropy inverts load-balancing terms computed on the same statistic (Shazeer et al., 2017; Cui et al., 2020). We also tested three properties that a rule-structured head could offer and a linear head could not: the unnormalised rule firing detects out-of-distribution inputs worse than a Mahalanobis detector on the same representations (AUROC 0.69 vs 0.95 on BUSI; 0.80 vs 0.85 on SST-2); with the entropy term, rules applied before pooling localise worse than their single-rule control, which on BUSI is exactly CAM (IoU −0.043 [−0.075, −0.010]); and mixing class probabilities instead of logits changes neither accuracy nor attribution materially (at most 0.009 IoU). The revised manuscript cites these works (Section 2) and restates the contribution as empirical: a controlled study of such a head on deep representations.
W7. Sample selection, IG steps, completeness, sign, and deletion.
A. Correctly classified samples only: this restriction is stated in the submission (Table 1, Appendix B), but we agree that it inflates agreement with the annotations. New results: on all ≈162 test images per seed that contain a lesion, FLARE's IoU is 0.151 on correctly classified and 0.081 on misclassified images, and stays below that of the linear head (IoU difference −0.026 [−0.034, −0.020]). On SST-2 (all eligible test sentences, 3 seeds), the head comparison holds under 9 alternative thresholds for defining the annotated sentiment spans (|Δ| ≤ 0.014).
B. BUSI: the submitted 12 steps leave a median completeness gap of 67% (< 5% with ≥ 256 steps), yet localisation is unchanged (IoU 0.159 at 12 and 2,048 steps), and every conclusion holds at both settings.
C. SST-2: for ModernBERT, the zero-embedding baseline is degenerate (half of the margin change occurs before α≈1e-4 on the integration path), so the submitted 16-step IG has a completeness gap and behaves like a scaled gradient × input; refining it drives localisation towards chance (3 seeds: pointing-game accuracy from 0.449 to 0.204; random 0.182). IG with a padding-token baseline converges (median gap 1.3% at 128 steps), keeps localisation near the submitted values (IoU 0.259 vs 0.273) and makes it specific to the trained head (trained vs randomised consequents: IoU +0.050 [+0.032, +0.068]; submitted IG: −0.002, not significant).
D. Sign: we agree. Signed token scores (padding-baseline IG) localise better than the unsigned ℓ2 norm on pointing-game accuracy under every span threshold and on IoU under 8 of 9 (with the submitted threshold, 3 seeds: IoU 0.282 vs 0.250).
E. Deletion: we also agree, including on PneumoniaMNIST (Section 4.1 of the submission, Appendix D); the BUSI IG baseline is classified as abnormal (p ≈ 1), which limits whole-pixel deletion. We have revised Appendix C accordingly.
W1-W3 and Q2: Learned antecedents, parsimony, and end-to-end
We sincerely thank the reviewer for an expert, precise, and constructive review. We agree with its central evaluation: the submission never shows that the attribution depends on the fuzzy head. To address this, we ran new controls. FLARE matches the accuracy of a linear head but does not improve attribution. The rule structure adds something different: attribution decomposes exactly into per-rule contributions, and with more than two classes, the rules align with the classes. They can be inspected through the samples they regulate. We have narrowed our claims accordingly. Below, W1-W9 refer to the weaknesses and Q1-Q3 to the questions in the review.
Setting of the new experiments. Unless stated otherwise, all new comparisons are paired: we fit every head on the same frozen representations of the submitted models and evaluate them on the same data splits and test samples, over 10 seeds, with the primary analyses specified before running the experiments. Square brackets indicate 95% confidence intervals. Integrated Gradients (IG) uses 1,024 steps on BUSI and, on SST-2, a padding-token baseline (128 steps; hereafter padding-baseline IG) instead of the submitted zero baseline (W7); values with the submitted IG are given in parentheses.
W1, W2, and Q2. Learned antecedents, more rules than classes, parsimony, and end-to-end training on learned representations.
A. Correct: all four neural heads used K = 2 fixed prototypes and cross-entropy only (λ_ℓ1 = λ_H = 0; Appendix B now states this).
B. New, on frozen standardised BUSI representations: learned-antecedent heads with K = 16 reach 0.931 accuracy (vs 0.927; not significant), and the entropy term lowers the effective number of rules, K_eff, measured on held-out data, from 8.21 to 3.00 (Δ −5.21 [−5.91, −4.60]) without collapse to a single active rule for K ≥ 4 (K = 2 collapses to one rule). The configuration chosen by our pre-specified selection rule (K = 16, λ_H = 0.05) localises better than the unregularised learned head (IoU +0.020 [+0.010, +0.029]) and the submitted head (+0.023 [+0.013, +0.033]), but stays slightly below a linear head (−0.007 [−0.014, −0.0001]). On unstandardised representations, the entropy term lowers K_eff from 12.16 to 1.31, about one rule.
C. New, on frozen SST-2 representations: the entropy term collapses every K to one rule at λ_H ≥ 0.05 with unchanged macro-F1 (0.946). On this nearly linearly separable representation, parsimony reduces FLARE to a linear head.
D. New, end-to-end (BUSI, K ∈ {4, 16}, 40 runs, all stable): with a standardisation layer inside the network, accuracy is 0.7-1.3 points below the two-stage pipeline and localisation does not improve, although entropy still lowers K_eff (K = 16: from 6.48 to 4.14, p = 0.01); without it, accuracy matches the submission and BUSI IoU is the highest of any IG map in our study (0.229 vs 0.155 with the layer; +0.073 [+0.051, +0.096]), but the rule base collapses to about one rule (K_eff 1.0-1.7). On SST-2, end-to-end training with the fuzzy head matches the submission (accuracy 0.946; IoU 0.264 vs 0.260 for the linear head), and entropy again collapses it to one rule.
E. New, with three or four classes on frozen representations: the rule base no longer collapses but organises by class, with K_eff 3.71 [3.16, 4.28] on BUSI (normal/benign/malignant), 4.18 on SST-3, and 6.33 on AG News (ModernBERT); macro-F1 stays equivalent to a linear head, with no attribution gain. Trained end-to-end on three-class BUSI, the rules collapse again.
W3. The introduction's critique describes the paper's own main experiments. We agree. In the fixed-prototype regime, both the representation and the rule geometry are fixed before the consequents are fitted. The revised manuscript rewrites that sentence (Section 1) and describes the regime as post-hoc fitting of a rule-structured head.