acceptodds
Under review as a conference paper at ICLR 2027

FLARE: A Differentiable and Parsimonious Fuzzy Rule Layer for Grounding Learned Representations

Abstract

Neural networks learn representations that support accurate prediction, but the decision structure built on these representations is typically opaque. Attribution methods can identify input features associated with individual predictions, while rule-based models expose explicit decision logic; connecting these two forms of interpretability remains challenging. We introduce FLARE, a differentiable and parsimonious fuzzy rule layer for learned representations. FLARE replaces a conventional prediction head with a first-order Takagi–Sugeno system whose Gaussian antecedents define fuzzy regions of representation space and whose local linear consequents produce the prediction. An explicit regularisation objective encourages concentrated rule usage and sparse consequents. Because the complete predictor remains differentiable, its decisions can be traced through the representation model back to the input domain. Across tabular, image, and text experiments, FLARE achieves predictive performance close to conventional predictors while providing an explicit rule-structured decision layer. On breast ultrasound image classification and sentiment analysis, attribution of the fuzzy predictor consistently concentrates on independently defined lesion masks and sentiment annotations across multiple localisation measures and random seeds. FLARE therefore connects three levels that are usually studied separately: learned representations, explicit rule-based decisions, and the input evidence associated with those decisions.

Primary area:
interpretability and explainable AI
Keywords:
Interpretable Machine Learning, Differentiable Fuzzy Systems, Interpretable Representation Learning, Neuro-Symbolic Learning, Feature Attribution, Parsimonious Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vLbtOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents FLARE to replace a network’s prediction head with a first-order Takagi–Sugeno fuzzy layer: Gaussian antecedents, softmax-normalised firing and local linear consequents. It adds an L1 penalty on the consequents and an entropy penalty on rule usage, and explains predictions with Integrated Gradients through the whole model. It reports accuracy close to linear heads on four neural tasks, attribution that beats a random ordering against lesion masks (BUSI) and sentiment spans (SST-2), and reduced effective rule counts on Iris and Digits.

Critical strengths
  1. The paper is straightforward. It states that the entropy term is inactive in the main experiments, that the accuracy comparison is confounded, that the faithfulness results depend on protocol, and that the single-cell analysis explains a different model.

  2. The statistical analysis is thorough and reliable, using multiple methods to compare results, estimate uncertainty, and clearly explain the limitations of statistical testing.

  3. The Grad-CAM++ control provides a useful check to see whether the results remain similar across different output classes and when the relationships between variables are randomized.

Critical weaknesses
  1. The claimed combination is never tested. In all four neural experiments the head has K = 2 rules, with centres fixed at class means, widths fixed at 2, and only the consequents trained.

  2. The parsimony objective and learned antecedents appear only on Iris and Digits, on raw features with no learned representation.

  3. The introduction criticises post-hoc methods because “the representation and the resulting rule geometry are already fixed”, but that describes the paper’s own main experiments.

  4. The fuzzy head in the main experiments is nearly a linear head. With two rules and equal isotropic widths, the firing weight reduces to a fixed logistic function along the class-mean difference. The head is then two linear models blended by a fixed gate. No rule from a neural experiment is shown, and it is unclear why this is more inspectable than the single linear head it replaces.

  5. Integrated Gradients is applied to the whole predictor, not to a rule, so the rule structure plays no role in the explanation. The only baseline is a random ordering, which any gradient method on any trained classifier should beat.

  6. The novelty is limited and prior work is missing such as a differentiable, trainable Takagi–Sugeno layer with Gaussian memberships. The layer is also a Gaussian-gated mixture of experts or normalised radial-basis network. The log-space firing, L1 and entropy terms are standard.

  7. Evaluation uses only correctly classified samples (about 35 images and 150 sentences per seed). Integrated Gradients uses 12–16 steps, with no completeness error reported. The token-level L2 norm discards sign. Whole-pixel deletion is not significant on BUSI and is worse than random on PneumoniaMNIST.

  8. The parsimony study is small and confounded. It uses only Iris and Digits. The L1 and entropy terms are switched on together, so their effects are not separated.

  9. The accuracy comparison lacks the obvious control. A linear or logistic head fitted post hoc on the same frozen, standardised representation would isolate the effect of the head. The single-cell experiment does not bear on FLARE. It explains the base Transformer, and its reference gene signature comes from the same line of work.

Questions for authors
  1. How do the attribution metrics compare between the fuzzy and linear predictors on the same samples?
  2. What happens with learned antecedents and K greater than the number of classes on BUSI and SST-2, or with end-to-end training?
  3. In the Iris example, how is “THEN versicolor” derived from an affine consequent? The rule pairs high sepal width and low petal width with versicolor, which looks odd.
5 comments
AuthorsOpenReview ↗

W1-W3 and Q2: Learned antecedents, parsimony, and end-to-end

We sincerely thank the reviewer for an expert, precise, and constructive review. We agree with its central evaluation: the submission never shows that the attribution depends on the fuzzy head. To address this, we ran new controls. FLARE matches the accuracy of a linear head but does not improve attribution. The rule structure adds something different: attribution decomposes exactly into per-rule contributions, and with more than two classes, the rules align with the classes. They can be inspected through the samples they regulate. We have narrowed our claims accordingly. Below, W1-W9 refer to the weaknesses and Q1-Q3 to the questions in the review.

Setting of the new experiments. Unless stated otherwise, all new comparisons are paired: we fit every head on the same frozen representations of the submitted models and evaluate them on the same data splits and test samples, over 10 seeds, with the primary analyses specified before running the experiments. Square brackets indicate 95% confidence intervals. Integrated Gradients (IG) uses 1,024 steps on BUSI and, on SST-2, a padding-token baseline (128 steps; hereafter padding-baseline IG) instead of the submitted zero baseline (W7); values with the submitted IG are given in parentheses.

W1, W2, and Q2. Learned antecedents, more rules than classes, parsimony, and end-to-end training on learned representations.

A. Correct: all four neural heads used K = 2 fixed prototypes and cross-entropy only (λ_ℓ1 = λ_H = 0; Appendix B now states this).

B. New, on frozen standardised BUSI representations: learned-antecedent heads with K = 16 reach 0.931 accuracy (vs 0.927; not significant), and the entropy term lowers the effective number of rules, K_eff, measured on held-out data, from 8.21 to 3.00 (Δ −5.21 [−5.91, −4.60]) without collapse to a single active rule for K ≥ 4 (K = 2 collapses to one rule). The configuration chosen by our pre-specified selection rule (K = 16, λ_H = 0.05) localises better than the unregularised learned head (IoU +0.020 [+0.010, +0.029]) and the submitted head (+0.023 [+0.013, +0.033]), but stays slightly below a linear head (−0.007 [−0.014, −0.0001]). On unstandardised representations, the entropy term lowers K_eff from 12.16 to 1.31, about one rule.

C. New, on frozen SST-2 representations: the entropy term collapses every K to one rule at λ_H ≥ 0.05 with unchanged macro-F1 (0.946). On this nearly linearly separable representation, parsimony reduces FLARE to a linear head.

D. New, end-to-end (BUSI, K ∈ {4, 16}, 40 runs, all stable): with a standardisation layer inside the network, accuracy is 0.7-1.3 points below the two-stage pipeline and localisation does not improve, although entropy still lowers K_eff (K = 16: from 6.48 to 4.14, p = 0.01); without it, accuracy matches the submission and BUSI IoU is the highest of any IG map in our study (0.229 vs 0.155 with the layer; +0.073 [+0.051, +0.096]), but the rule base collapses to about one rule (K_eff 1.0-1.7). On SST-2, end-to-end training with the fuzzy head matches the submission (accuracy 0.946; IoU 0.264 vs 0.260 for the linear head), and entropy again collapses it to one rule.

E. New, with three or four classes on frozen representations: the rule base no longer collapses but organises by class, with K_eff 3.71 [3.16, 4.28] on BUSI (normal/benign/malignant), 4.18 on SST-3, and 6.33 on AG News (ModernBERT); macro-F1 stays equivalent to a linear head, with no attribution gain. Trained end-to-end on three-class BUSI, the rules collapse again.

W3. The introduction's critique describes the paper's own main experiments. We agree. In the fixed-prototype regime, both the representation and the rule geometry are fixed before the consequents are fitted. The revised manuscript rewrites that sentence (Section 1) and describes the regime as post-hoc fitting of a rule-structured head.

AuthorsOpenReview ↗

W4, W5, and Q1: Near-linear head and attribution controls

W4. The fuzzy head is nearly a linear head.

A. The algebra is exactly right: for σ = 2, w1=sigmoid((c1−c2)⊤z~/4−(∥c1∥2−∥c2∥2)/8)w_1=\mathrm{sigmoid}\big((c_1-c_2)^\top\tilde z/4-(\Vert c_1\Vert^2-\Vert c_2\Vert^2)/8\big), where z̃ is the standardised representation (i.e. two affine maps blended by a fixed one-dimensional logistic gate), which contains the linear head as a special case.

B. New: the gate is unsaturated (w₁ ∈ [0.05, 0.95]) on 36% of BUSI test images, and the gradient of the margin has a cosine similarity of 0.71 with that of a linear head; on SST-2, the values are 12% and 0.94 (99.7% identical predictions). On BUSI, the head is therefore not linear in practice, yet this yields no gain in accuracy or attribution (W5, W9).

C. The class purity of the rules governing test images is 0.77 for the submitted BUSI head and 0.93 for the selected learned head (a single rule would score ≈0.83 on BUSI, the majority-class rate), and 0.94 on SST-2. With three classes on BUSI, the two rules within the benign class separate lesions by echogenicity, a BI-RADS descriptor (lesion-surround contrast: AUC 0.83), not by size, position, or shape; two-cluster k-means on the same representations recovers the same axis (AUC 0.86), so the rules make this structure explicit. We also explained rule membership itself (the IF part) in input space. Changing only the lesion's grey level moves images between the two benign rules in the expected direction (standardised effect 0.66 [0.56, 0.76], 8 seeds). Still, the membership maps do not concentrate on the lesion more than the class maps, and two-cluster k-means on the same representations does at least as well. The structure therefore belongs to the representation.

D. We accept that in this regime the head is not meaningfully more inspectable than a linear head. The revised manuscript shows the rules of a seed chosen in advance, with per-rule maps and exemplars (Figure 5).

W5 and Q1. IG is applied to the whole predictor, and the only baseline is a random ordering. Both points are correct. We therefore compared FLARE with a linear head trained with the same procedure on the same standardised representations and samples (new evidence; FLARE minus linear head):

Pointing-game accuracy (FLARE − linear)IoU (FLARE − linear)Accuracy, FLARE / linear
BUSI−0.088 [−0.136, −0.040]−0.030 [−0.038, −0.022]0.927 / 0.927
SST-2+0.000 [−0.009, +0.009] (+0.004)−0.000 [−0.004, +0.005] (+0.002)0.947 / 0.946

A. BUSI: the fuzzy head localises worse. On unstandardised representations, the gap vanishes, and every head matches the linear classifier trained jointly with the backbone (the submission's comparator; IoU 0.218-0.220 vs 0.217). A further control on linear heads shows that the difference between standardised and unstandardised representations comes from centring the head's input under our fitting procedure, not from its scale: it persists on the post-ReLU representations (+0.027 [+0.019, +0.036]), shrinks to +0.006 with a converged logistic regression, and is absent on SST-2. Randomising the fuzzy consequents does not reduce localisation (IoU, trained minus randomised: −0.008 [−0.022, +0.007]), and the channel-mean feature map (IoU 0.259) and a fixed centre prior (0.194), which do not depend on the head, exceed FLARE's IoU (0.158), although FLARE is better on pointing-game accuracy, attribution mass, and deletion. On unstandardised representations, randomising the consequents does reduce localisation (IoU, trained minus randomised: +0.021 [+0.011, +0.034]).

B. SST-2: the fuzzy head is equivalent to the linear head; a fixed sentiment lexicon localises better (IoU 0.381 vs 0.259) but is less faithful (deletion 0.670 vs 0.624, insertion 0.802 vs 0.853). The localisation in Table 2 thus reflects the composed predictor and is dominated by the representation.

C. Rule-level IG: IG is linear in the explained function, and the margin is m = Σ_k g_k with g_k = w_k (f_{k,c} − f_{k,c̄}), where c̄ is the other class, so IG(m) = Σ_k IG(g_k) exactly. On BUSI, about two rules each carry > 10% of the attribution, and the gate switches the dominant rule along the integration path. In an exploratory analysis, the dominant rule's map localises better than the whole-predictor map (IoU 0.190 vs 0.158, +0.031 [+0.020, +0.044]), matching the linear head; unregularised learned heads show no such gain. For the submitted and the selected learned SST-2 heads, the decomposition is degenerate (77-100% of the attribution in one rule).

D. A pre-registered replication on new splits and seeds confirms the gain (submitted head +0.022 [+0.015, +0.029], p = 0.002; selected learned head +0.018 [+0.011, +0.026]); the pointing-game gain does not replicate (p = 0.13). Heads fitted on unstandardised representations localise better still, so the decomposition shows which rule carries the evidence rather than improving localisation.

AuthorsOpenReview ↗

W6 and W7: Prior work, sample selection, and IG settings

W6. Novelty and missing prior work. The head can be seen as a first-order ANFIS (Jang, 1993), i.e., a normalised Gaussian RBF network with local linear models (Jang & Sun, 1993; Hunt et al., 1996), or a mixture of linear experts with Gaussian gating (Jacobs et al., 1991; Xu, Jordan & Hinton, 1994); fuzzy heads on CNN features with gradient saliency have been proposed (Yeganejou et al., 2020; 2023). Log-space firing is standard (Cui et al., 2021), the ℓ1 term is a lasso penalty, and the rule-usage entropy inverts load-balancing terms computed on the same statistic (Shazeer et al., 2017; Cui et al., 2020). We also tested three properties that a rule-structured head could offer and a linear head could not: the unnormalised rule firing detects out-of-distribution inputs worse than a Mahalanobis detector on the same representations (AUROC 0.69 vs 0.95 on BUSI; 0.80 vs 0.85 on SST-2); with the entropy term, rules applied before pooling localise worse than their single-rule control, which on BUSI is exactly CAM (IoU −0.043 [−0.075, −0.010]); and mixing class probabilities instead of logits changes neither accuracy nor attribution materially (at most 0.009 IoU). The revised manuscript cites these works (Section 2) and restates the contribution as empirical: a controlled study of such a head on deep representations.

W7. Sample selection, IG steps, completeness, sign, and deletion.

A. Correctly classified samples only: this restriction is stated in the submission (Table 1, Appendix B), but we agree that it inflates agreement with the annotations. New results: on all ≈162 test images per seed that contain a lesion, FLARE's IoU is 0.151 on correctly classified and 0.081 on misclassified images, and stays below that of the linear head (IoU difference −0.026 [−0.034, −0.020]). On SST-2 (all eligible test sentences, 3 seeds), the head comparison holds under 9 alternative thresholds for defining the annotated sentiment spans (|Δ| ≤ 0.014).

B. BUSI: the submitted 12 steps leave a median completeness gap of 67% (< 5% with ≥ 256 steps), yet localisation is unchanged (IoU 0.159 at 12 and 2,048 steps), and every conclusion holds at both settings.

C. SST-2: for ModernBERT, the zero-embedding baseline is degenerate (half of the margin change occurs before α≈1e-4 on the integration path), so the submitted 16-step IG has a completeness gap and behaves like a scaled gradient × input; refining it drives localisation towards chance (3 seeds: pointing-game accuracy from 0.449 to 0.204; random 0.182). IG with a padding-token baseline converges (median gap 1.3% at 128 steps), keeps localisation near the submitted values (IoU 0.259 vs 0.273) and makes it specific to the trained head (trained vs randomised consequents: IoU +0.050 [+0.032, +0.068]; submitted IG: −0.002, not significant).

D. Sign: we agree. Signed token scores (padding-baseline IG) localise better than the unsigned ℓ2 norm on pointing-game accuracy under every span threshold and on IoU under 8 of 9 (with the submitted threshold, 3 seeds: IoU 0.282 vs 0.250).

E. Deletion: we also agree, including on PneumoniaMNIST (Section 4.1 of the submission, Appendix D); the BUSI IG baseline is classified as abnormal (p ≈ 1), which limits whole-pixel deletion. We have revised Appendix C accordingly.

Reviewer_y5KrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents FLARE, a differentiable fuzzy rule layer designed to make neural representations more interpretable. It uses a Takagi–Sugeno fuzzy system with regularization to encourage compact rule structures, while Integrated Gradients connects predictions back to input features. Experiments on tabular, image, and text datasets show competitive accuracy and reasonable attribution results against lesion masks and sentiment annotations. The framework is interesting, although I find its methodological contribution somewhat incremental.

Critical strengths
  1. The paper addresses a meaningful interpretability problem. Connecting fuzzy decision rules with learned representations and input attribution is a reasonable direction.

  2. I appreciate the evaluation across different modalities. The BUSI and SST-2 results suggest that the fuzzy predictor can maintain competitive accuracy while identifying relevant input regions.

  3. The statistical analysis is relatively careful, with multiple seeds, significance tests, and attribution faithfulness evaluations. These make the empirical results more credible.

Critical weaknesses
  1. Novelty is somewhat limited. Differentiable fuzzy systems and gradient-based explanations are already well studied. I am not fully convinced that FLARE goes much beyond combining these existing ideas.

  2. The parsimony claim is not well supported in the main experiments. Rule centers and widths are fixed, so the entropy regularizer does not actually affect training on BUSI and SST-2.

  3. The benefit of fuzzy rules for attribution remains unclear. Most comparisons are against random baselines. A conventional linear head with the same Integrated Gradients procedure might achieve similar results.

  4. Interpretability is still rather indirect. The rules operate on latent features without clear semantic meanings, and the paper provides limited evidence connecting individual rules to understandable input patterns.

Questions for authors
  1. What is the main methodological difference from existing differentiable neuro-fuzzy models, especially Zhang et al. (2026)?

  2. Could the authors evaluate trainable rule antecedents and entropy regularization on BUSI or SST-2, rather than only on the smaller tabular experiments?

  3. How does FLARE attribution compare with Integrated Gradients applied to a standard linear head using the same frozen representations?

  4. Can the authors show a few rule-level explanations that directly connect specific fuzzy rules to meaningful input regions?

4 comments
AuthorsOpenReview ↗

W1 and Q1: Novelty and relation to prior work

We sincerely thank the reviewer for their thoughtful evaluation and for recognising the relevance of the interpretability problem and the care of our statistical analysis. We agree with all four weaknesses. To address them, we ran the requested controls. They show that FLARE matches the accuracy of a linear head but does not improve attribution. The rule structure adds something different: attribution decomposes exactly into per-rule contributions, and with more than two classes the rules align with the classes. They can be inspected through the samples they regulate. These results support a narrower paper, and we report them, including those that do not favour our original framing. Below, W1-W4 refer to the weaknesses and Q1-Q4 to the questions in the review.

Setting of the new experiments. Unless stated otherwise, all new comparisons are paired: we fit every head on the same frozen representations of the submitted models and evaluate them on the same data splits and test samples, over 10 seeds, with the primary analyses specified before running the experiments. Square brackets indicate 95% confidence intervals. Integrated Gradients (IG) uses 1,024 steps on BUSI and, on SST-2, a padding-token baseline (128 steps) instead of the submitted zero baseline (see "Corrections"); values with the submitted IG are given in parentheses.

W1 and Q1. Novelty and the difference from Zhang et al. (2026).

A. We agree that the head is not new. A first-order Takagi-Sugeno (TS) layer with Gaussian antecedents and normalised firing can be seen as an ANFIS (Jang, 1993), equivalently a normalised Gaussian RBF network with local linear models (Jang & Sun, 1993; Hunt et al., 1996), or a Gaussian-gated mixture of linear experts (Jacobs et al., 1991; Xu et al., 1994). Researchers have proposed fuzzy heads on CNN features with saliency maps (Yeganejou et al., 2020; 2023), and the rule-usage entropy term inverts known load-balancing losses (Shazeer et al., 2017; Cui et al., 2020).

B. Zhang et al. (2026) use the TS system as the representation learner on raw or hand-crafted features (i.e., antecedents fixed by enhanced soft subspace clustering, consequents learned by an unrolled iterative shrinkage-thresholding algorithm, ISTA), with no deep backbone, input attribution, or rule-usage regulariser. FLARE shares the TS form and fixed antecedents, but uses a frozen deep representation as a head, with input attribution through the composed predictor (backbone and head).

C. We also tested three properties that a rule-structured head could offer and a linear head could not. The unnormalised rule firing, used to detect out-of-distribution inputs, is outperformed by a Mahalanobis detector on the same representations (AUROC 0.69 vs 0.95 on BUSI; 0.80 vs 0.85 on SST-2). With the entropy term, rules applied at every spatial location or token, before pooling, localise worse than the corresponding single-rule control; on BUSI, this is exactly CAM (IoU −0.043 [−0.075, −0.010]; SST-2: −0.032 [−0.058, −0.009]). Mixing class probabilities instead of logits changes neither accuracy nor attribution materially (at most 0.009 IoU).

D. Our contribution is thus empirical: a controlled study of a rule-structured head on deep representations. The revised manuscript states this in the abstract, contributions, and Section 2.

AuthorsOpenReview ↗

W2-W3 and Q2-Q3: Learned antecedents and linear-head IG

W2 and Q2. Trainable rule antecedents and entropy regularisation on BUSI and SST-2. The reviewer is correct, and the gap was wider: in the submission, all neural runs used λ_ℓ1 = λ_H = 0. We therefore fitted learned-antecedent heads (K ∈ {2, 4, 8, 16}, 12-cell ℓ1 × entropy grid) on the frozen representations, and also trained the full model end-to-end:

A. BUSI: at K = 16, entropy lowers the effective number of rules, K_eff, on held-out data from 8.21 to 3.00 (Δ −5.21 [−5.91, −4.60]), without collapse to a single active rule at K ≥ 4 or a significant accuracy cost (0.931 vs 0.927 for the submitted head); consequent sparsity is driven by ℓ1, not by entropy. The configuration chosen by our pre-specified selection rule (K = 16, λ_H = 0.05) localises better than the unregularised learned head (IoU +0.020 [+0.010, +0.029]) and the submitted head (+0.023), but not better than a linear head (−0.007 [−0.014, −0.0001]).

B. SST-2: entropy collapses every K to one rule at λ_H ≥ 0.05 (macro-F1 unchanged at 0.946), so the selected learned head is a linear head with a rule label; all learned heads localise like the submitted one (differences within ±0.017). End-to-end training gives the same collapse at unchanged accuracy.

C. BUSI, end-to-end (40 runs): with a standardisation layer inside the network, accuracy is 0.7-1.3 points below two-stage training and localisation does not improve, although entropy still lowers K_eff (K = 16: from 6.48 to 4.14); without it, accuracy matches the submission and BUSI IoU is the highest of any IG map in our study (0.229 vs 0.155 with the layer; +0.073 [+0.051, +0.096]), but the rule base collapses to about one rule (K_eff 1.0-1.7).

D. With three or four classes, the rule base does not collapse on frozen representations: K_eff is 3.71 [3.16, 4.28] on BUSI (normal/benign/malignant), 4.18 on SST-3, and 6.33 on AG News (ModernBERT), with rules aligned with the classes and macro-F1 equivalent to a linear head; trained end-to-end on three-class BUSI, the rules collapse again.

E. The objective therefore works on learned representations, but its outcome depends on how many distinct regimes the representation supports and on standardisation (frozen binary BUSI without it: K_eff 1.3).

W3 and Q3. IG applied to a standard linear head on the same frozen representations. All results are new. The reviewer's expectation holds (FLARE minus a linear head trained with the same procedure on the same standardised representations):

Pointing-game accuracy (FLARE − linear)IoU (FLARE − linear)Accuracy, FLARE / linear
BUSI−0.088 [−0.136, −0.040]−0.030 [−0.038, −0.022]0.927 / 0.927
SST-2+0.000 [−0.009, +0.009] (+0.004)−0.000 [−0.004, +0.005] (+0.002)0.947 / 0.946

A. On BUSI, FLARE localises worse than the linear head, and worse than the linear classifier trained jointly with the backbone (the submission's comparator); on SST-2, they are equivalent under both IG variants.

B. The BUSI gap exists only on standardised representations. Standardisation leaves accuracy unchanged but worsens BUSI attribution for every head: without it, FLARE, the learned-antecedent head, and the linear head all reach IoU 0.218-0.220 (vs 0.158-0.189 with it; each p ≤ 0.004), matching the jointly trained linear classifier (0.217). The revised manuscript therefore computes attributions on unstandardised representations and reports the standardised setting as an ablation (Table 2). A further control on linear heads shows that the difference between standardised and unstandardised representations comes from centring the head's input under our fitting procedure, not from its scale: it persists on the post-ReLU representations (+0.027 [+0.019, +0.036]), shrinks to +0.006 with a converged logistic regression, and is absent on SST-2.

C. Randomising the fuzzy consequents does not reduce localisation on BUSI, and reference maps that do not depend on the head are competitive: the channel-mean feature map (IoU 0.259) and a fixed centre prior (0.194) exceed FLARE's 0.158, and a sentiment lexicon exceeds FLARE on SST-2 (0.381 vs 0.259). On unstandardised representations, randomising the consequents does reduce localisation (IoU, trained minus randomised: +0.021 [+0.011, +0.034]).

D. FLARE's IG is still more faithful than these reference maps (deletion: BUSI 0.79 vs 0.89; SST-2 0.624 vs 0.670), and on SST-2 the trained head outperforms randomised consequents (IoU +0.050 [+0.032, +0.068]; with the submitted IG: −0.002, inconclusive). The attribution is thus specific to the trained predictor, but not to its fuzzy structure.

E. The head comparisons hold on all test images that contain a lesion, including misclassified ones, and under 9 alternative thresholds for the SST-2 sentiment spans (3 seeds). The localisation in Table 2 is therefore dominated by the representation, and we no longer attribute it to the fuzzy structure.

AuthorsOpenReview ↗

W4 and Q4: Rule-level explanations; corrections and revisions

W4 and Q4. Rule-level explanations. We agree; the submission shows none for neural models. The following results are new.

A. Because IG is linear in the explained function and the margin is a sum of per-rule terms, the attribution decomposes exactly into per-rule maps (error 0.0). On BUSI, about two rules each carry > 10% of the attribution, and the gate switches the dominant rule along the integration path.

B. Exploratory: the dominant rule's map localises lesions better than the whole-predictor map (submitted head IoU 0.190 vs 0.158, +0.031 [+0.020, +0.044]; selected learned head +0.015 [+0.010, +0.021]), reaching the linear head's level; unregularised learned heads show no such gain. A pre-registered replication on new splits and seeds confirms the gain (submitted head +0.022 [+0.015, +0.029], p = 0.002; selected learned head +0.018 [+0.011, +0.026]); the pointing-game gain does not replicate (p = 0.13). Heads fitted on unstandardised representations localise better still (W3), so the decomposition shows which rule carries the evidence rather than improving localisation.

C. For the submitted and the selected learned SST-2 heads, the decomposition is degenerate (77-100% of the attribution in one rule).

D. With three classes on BUSI, every class has its own rules, and the two rules within the benign class separate lesions by echogenicity, a BI-RADS descriptor (lesion-surround contrast: AUC 0.83; grey level inside the lesion: 0.74), not by size, position, or shape. Two-cluster k-means on the same representations recovers the same axis (AUC 0.86), so the rules make this structure explicit. We also explained rule membership itself (the IF part) in input space. Changing only the lesion's grey level moves images between the two benign rules in the expected direction (standardised effect 0.66 [0.56, 0.76], 8 seeds). Still, the membership maps do not concentrate on the lesion more than the class maps, and two-cluster k-means on the same representations does at least as well. The structure therefore belongs to the representation. On AG News, every topic has its own rules (class purity 0.90 vs a majority rate of 0.25).

E. A deeper evaluation of the Iris rules showed that the centres misdescribe the rules: 88% of trained centres lie off the data (vs 0.7% of their k-means initialisations). We therefore implemented a linguistic rendering of each rule computed from the samples it governs; the revised manuscript shows rule-level maps with the top-3 exemplars per rule (Figure 5). Because the rules act on latent representations, their semantics remain only partly grounded.

Corrections. Upon re-examining the submission, we found the following errors, which the revised manuscript corrects, along with a few minor errors in the tables and appendices.

A. The submitted IG was inaccurate. On BUSI, 12 steps leave a median completeness gap of 67% (< 5% with ≥ 256 steps), although localisation is unchanged (IoU 0.159 at 12 and 2,048 steps). On SST-2, the zero-embedding baseline is degenerate for ModernBERT: the submitted 16-step IG has a completeness gap, and refining it drives localisation towards chance (3 seeds: pointing-game accuracy from 0.449 to 0.204; random 0.182). IG with a padding-token baseline converges (median gap 1.3% at 128 steps) with similar localisation (IoU 0.259 vs 0.273 submitted).

B. On Iris, ℓ1 alone contributes 41-61% of the K_eff reduction that Section 5 of the submission attributes to entropy.

Revisions. In the updated PDF, new and revised parts are in red. We reposition FLARE as a first-order ANFIS/normalised-RBF/Gaussian-gated mixture-of-experts head; narrow the claims in the abstract, introduction, contributions, and discussion; extend Table 2 with the head controls and reference maps; state the λ values and correct IG; remove the single-cell experiment, which explains the base Transformer rather than FLARE; and report all new experiments in Appendix E.

We thank the reviewer again for the encouraging and critical feedback, which has substantially changed how we position this work.

I appreciate the authors' thorough and transparent response, particularly the additional controlled experiments and corrections; the results confirm that FLARE provides no clear advantage over a matched linear head in predictive performance or attribution quality, so I maintain my original score.

Reviewer_K12gOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FLARE, a first-order Takagi–Sugeno fuzzy prediction head for learned representations. The head uses Gaussian antecedents, normalized rule firing, and local affine consequents. The authors additionally introduce an ℓ1\ell_1 penalty on consequent slopes and an entropy penalty on aggregate rule usage, and use Integrated Gradients (IG) to attribute the output of the composed representation model and fuzzy head back to the input. Experiments cover image, text, single-cell, and tabular data, with BUSI and SST-2 serving as the two main attribution case studies. The paper is clearly written, technically organized, and unusually candid about several limitations. The localization and perturbation analyses are more careful than purely qualitative heatmap evaluations, and the Grad-CAM++ controls usefully illustrate that localization against an external annotation does not necessarily imply predictor-specific faithfulness. However, I do not think the current experiments establish the paper's central combination of novelty, rule-level grounding, and parsimony on learned representations.

Critical strengths

This article provides a structurally clear differentiable fuzzy prediction framework that links the first-order Takagi Sugeno rule layer, rule complexity regularization, and input attribution. The experiment covers tables, images, text, and single-cell data, and reports the results through multiple random seeds and paired statistical tests. The paper simultaneously evaluates agreement between attribution and independent reference annotations and perturbation-based faithfulness, especially through a Grad-CAM++comparison, to illustrate that localization consistency is not equivalent to predictor-specific faithfulness. The authors also clearly distinguish the fixed-prototype and learned-antecedent regimes, identify the attribution target in each experiment, and discuss limitations concerning predictive comparisons and dataset splits, making the scope of the conclusions transparent.

Critical weaknesses
  1. The main attribution experiment comparing FLARE with random sorting can demonstrate that attribution contains meaningful information, but lacks a comparison of applying the same attribution method to ordinary linear or MLP heads under the same representation and evaluation process, making it difficult to determine how much of the attribution effect comes from fuzzy decision structures.
  2. The main neural network experiment fixed the center and width of the rules, only optimizing the consequent, so rule-usage entropy does not affect the optimization. Learnable antecedents and concentrated rule usage have mainly been validated on Iris and Digits, and these results are not yet sufficient to demonstrate the effectiveness of this mechanism in deep representation.
  3. Rules are defined on latent coordinates that lack direct semantics, while Integrated Gradients explains a complete predictor, not a single rule. The existing results support the connection between overall prediction and input evidence, but provide limited validation for the semantics of specific rules and their corresponding evidence.
Questions for authors
  1. How do the authors distinguish the specific contribution of FLARE from ANFIS and related RBF/MoE formulations?
  2. Do existing experiments or supplementary materials include attribution comparisons for FLARE and linear or MLP heads under the same frozen representation, input transformation, and attribution protocol? If so, please indicate the corresponding results.
  3. Section 3.2 states that the rule-usage entropy is constant and does not affect optimization in fixed-prototype systems. Please clarify which regularization terms were used when fitting the consequent parameters in the main neural experiments, and whether consequent sparsity was measured in these experiments.
  4. How does FLARE compare with a linear or RBF/MoE head trained on the same frozen standardized representation?
3 comments
AuthorsOpenReview ↗

W1, Q2, and Q4: Head controls on the same representations

We sincerely thank the reviewer for their careful reading, constructive critiques, and helpful suggestions. We are particularly grateful for the precise questions on head controls, regularisers, and the relation to ANFIS and RBF/MoE models, which led directly to the new controls reported below. We agree with all three weaknesses. The new controls show that FLARE matches the accuracy of a linear head but does not improve attribution. The rule structure adds something different: attribution decomposes exactly into per-rule contributions, and with more than two classes, the rules align with the classes. They can be inspected through the samples they regulate. Thus, we narrow our claims accordingly. Below, W1-W3 refer to the weaknesses and Q1-Q4 to the questions in the review.

Setting of the new experiments. Unless stated otherwise, all new comparisons are paired: we fit every head on the same frozen representations of the submitted models and evaluate them on the same data splits and test samples, over 10 seeds, with the primary analyses specified before running the experiments. Square brackets indicate 95% confidence intervals. Integrated Gradients (IG) uses 1,024 steps on BUSI and, on SST-2, a padding-token baseline (128 steps) instead of the submitted zero baseline (see "Corrections"); values with the submitted IG are given in parentheses.

W1, Q2, and Q4. FLARE vs linear, MLP, RBF, and MoE heads on the same frozen standardised representations. The initial submission had no such comparison: a random ordering was the only IG reference, and Appendix C's controls covered Grad-CAM++ only. We therefore compared FLARE with a linear head trained with the identical procedure (FLARE minus linear head):

Pointing-game accuracy (FLARE − linear)IoU (FLARE − linear)Accuracy, FLARE / linear
BUSI−0.088 [−0.136, −0.040]−0.030 [−0.038, −0.022]0.927 / 0.927
SST-2+0.000 [−0.009, +0.009] (+0.004)−0.000 [−0.004, +0.005] (+0.002)0.947 / 0.946

A. Attribution (BUSI): FLARE localises worse than the linear head, than the linear classifier trained jointly with the backbone (the submission's comparator), and, in a secondary test, than an MLP head. Attribution (SST-2): FLARE is equivalent to all three under both IG variants.

B. Randomising the fuzzy consequents does not reduce localisation on BUSI (IoU, trained minus randomised: −0.008 [−0.022, +0.007]), and reference maps that do not depend on the head exceed FLARE on IoU: the channel mean of the last convolutional block (0.259) and a fixed centre prior (0.194) vs 0.158 on BUSI, and a sentiment lexicon (VADER) on SST-2 (0.381 vs 0.259), although FLARE's IG is more faithful under deletion. On unstandardised representations, randomising the consequents does reduce localisation (IoU, trained minus randomised: +0.021 [+0.011, +0.034]).

C. The comparisons with the two linear heads hold across all test images that contain a lesion, including misclassified ones (IoU −0.026 [−0.034, −0.020]), and under 9 alternative thresholds for the annotated SST-2 sentiment spans (3 seeds; |Δ| ≤ 0.014).

D. Accuracy: the linear head reaches 0.9267 vs 0.9272 for FLARE on BUSI (99% identical predictions) and 0.9460 vs 0.9469 on SST-2 (99.7%). On BUSI, the MLP head reaches 0.932, and the zero-order normalised RBF network (FLARE's centres, A_k = 0) reaches 0.908. The latter localises better than FLARE (IoU 0.207 vs 0.158). A Gaussian-gated MoE (learned-antecedent FLARE, K = 2) reaches 0.924 on BUSI (Δ −0.003 [−0.008, +0.002]) and macro-F1 0.946 on SST-2; with equal widths, its gates include every linear-logistic gate, so we did not run a separate MoE with linear gating.

E. Standardisation leaves accuracy unchanged (≤ 0.008 for every head) but worsens BUSI attribution for every head. Without it, FLARE, the learned-antecedent head, and the linear head all reach IoU 0.218-0.220 and pointing-game accuracy 0.68-0.69, against 0.158-0.189 and 0.51-0.60 with it (each p ≤ 0.004), and match the jointly trained linear classifier (0.217 / 0.695). The revised manuscript therefore computes attributions on unstandardised representations and reports the standardised setting as an ablation (Table 2). A further control on linear heads shows that the difference between standardised and unstandardised representations comes from centring the head's input under our fitting procedure, not from its scale: it persists on the post-ReLU representations (+0.027 [+0.019, +0.036]), shrinks to +0.006 with a converged logistic regression, and is absent on SST-2.

F. FLARE therefore matches these heads in accuracy, with no gain in accuracy or attribution. The localisation in Table 2 reflects the composed predictor and is dominated by the representation.

AuthorsOpenReview ↗

W2, Q3, and W3: Regularisers, sparsity, and rule semantics

W2 and Q3. Regularisers and consequent sparsity in the neural experiments.

A. None: all four neural experiments fitted the consequents with cross-entropy only (λ_ℓ1 = λ_H = 0). Appendix B omits this, and consequent sparsity was not measured, so the parsimony objective was never active on a learned representation.

B. New: we fitted learned-antecedent heads with K ∈ {2, 4, 8, 16} over a 12-cell ℓ1 × entropy grid and measured the effective number of rules, K_eff, on held-out data. On frozen standardised BUSI representations at K = 16, entropy lowers K_eff from 8.21 to 3.00 (Δ −5.21 [−5.91, −4.60]), without collapse to a single active rule (K ≥ 4) or accuracy cost (0.931 vs 0.927). ℓ1 raises consequent sparsity only slightly (+5.0 points; at most 9% of consequent entries have |A| < 10⁻²). The configuration chosen by our pre-specified selection rule localises better than the unregularised learned head (IoU +0.020 [+0.010, +0.029]) but slightly worse than the linear head (−0.007 [−0.014, −0.0001]). On unstandardised representations, entropy lowers K_eff from 12.16 to 1.31, about one rule.

C. BUSI, end-to-end (40 runs, all stable): with a standardisation layer inside the network, accuracy is 0.7-1.3 points below the submitted head and localisation does not improve, although entropy still lowers K_eff (K = 16: from 6.48 to 4.14); without it, accuracy matches the submission and BUSI IoU is the highest of any IG map in our study (0.229 vs 0.155 with the layer; +0.073 [+0.051, +0.096]), but the rule base collapses to about one rule (K_eff 1.0-1.7).

D. SST-2: entropy (λ_H ≥ 0.05) collapses every K to one rule at unchanged macro-F1 (0.946), i.e., to a linear head, both on frozen representations and with end-to-end training.

E. With three or four classes, the rule base does not collapse on frozen representations: K_eff is 3.71 [3.16, 4.28] on BUSI (normal/benign/malignant), 4.18 on SST-3, and 6.33 on AG News (ModernBERT), with rules aligned with the classes and macro-F1 equivalent to a linear head; trained end-to-end on three-class BUSI, the rules collapse again. The parsimony objective therefore works on learned representations, but its outcome depends on how many distinct regimes the representation supports.

W3. Semantics of individual rules. We agree; the submission validated no rule from a neural experiment. The following results are new.

A. Because IG is linear in the explained function and F_c = Σ_k w_k f_{k,c}, IG of the whole predictor splits exactly into per-rule attributions (exact up to floating-point error), so the rule structure can enter the explanation.

B. On BUSI: about two rules each carry > 10% of the attribution, and the gate switches the dominant rule along the integration path. In an exploratory analysis, the dominant rule's map localises better than the whole-predictor map (submitted head: IoU 0.190 vs 0.158, +0.031 [+0.020, +0.044]); a pre-registered replication on new splits and seeds confirms it (+0.022 [+0.015, +0.029], p = 0.002), though not for pointing-game accuracy (p = 0.13). Unregularised learned heads show no such gain. Heads fitted on unstandardised representations localise better still (W1), so the decomposition shows which rule carries the evidence rather than improving localisation.

C. For the submitted and the selected learned SST-2 heads, one rule carries 77-100% of the attribution, so the decomposition is degenerate.

D. With three classes on BUSI, every class has its own rules, and the two rules within the benign class separate lesions by echogenicity, a BI-RADS descriptor (lesion-surround contrast: AUC 0.83; grey level inside the lesion: 0.74), not by size, position, or shape. Two-cluster k-means on the same representations recovers the same axis (AUC 0.86), so the rules make this structure explicit. We also explained rule membership itself (the IF part) in input space. Changing only the lesion's grey level moves images between the two benign rules in the expected direction (standardised effect 0.66 [0.56, 0.76], 8 seeds). Still, the membership maps do not concentrate on the lesion more than the class maps, and two-cluster k-means on the same representations does at least as well. The structure therefore belongs to the representation.

E. On Iris, 88% of the trained centres drifted off the data (vs 0.7% at initialisation), so a rule described by its centre can be misleading; the revised manuscript describes each rule by the samples it governs (Appendix A).

F. The semantics of rules defined on latent representations therefore remain only partly grounded. The revised manuscript shows the rules of a seed chosen in advance, with per-rule maps and exemplars (Figure 5).

AuthorsOpenReview ↗

Q1: Relation to ANFIS, RBF, and MoE; corrections and revisions

Q1. Relation to ANFIS and RBF/MoE formulations.

A. The head is not a new model class. A first-order Takagi-Sugeno (TS) layer with Gaussian antecedents and normalised firing can be seen as a first-order ANFIS (Jang, 1993), functionally equivalent to a normalised Gaussian RBF network with local linear models (Jang & Sun, 1993; Hunt et al., 1996) and to a mixture of linear experts with normalised-Gaussian gating (Jacobs et al., 1991; Xu et al., 1994). Fuzzy heads on CNN features (Yeganejou et al., 2020; 2023) and log-space firing (Cui et al., 2021) exist, and the rule-usage entropy term inverts MoE load-balancing losses (Shazeer et al., 2017; Cui et al., 2020).

B. In our neural experiments (K = 2, centres fixed at the class means, shared width σ = 2), the gate is a fixed logistic function along c₁ - c₂, and the head contains the linear head (A₁ = A₂, b₁ = b₂) and the zero-order normalised RBF network (A_k = 0) as special cases.

C. We also tested three properties that a rule-structured head could offer and a linear head could not. The unnormalised rule firing, used to detect out-of-distribution inputs, is outperformed by a Mahalanobis detector on the same representations (AUROC 0.69 vs 0.95 on BUSI; 0.80 vs 0.85 on SST-2). With the entropy term, rules applied at every spatial location or token, before pooling, localise worse than the corresponding single-rule control; on BUSI, this is exactly CAM (IoU −0.043 [−0.075, −0.010]; SST-2: −0.032 [−0.058, −0.009]). Mixing class probabilities instead of logits changes neither accuracy nor attribution materially (at most 0.009 IoU).

D. Our contribution is therefore empirical: a controlled study of a rule-structured head on deep representations, covering accuracy, parsimony, an exact rule-level decomposition of the attribution, and what attribution can and cannot show. The revised manuscript cites these works and rewrites Section 2 and the contributions.

Corrections. Upon re-examining the submission, we found the following errors, which the revised manuscript corrects, along with a few minor errors in the tables and appendices.

A. Besides λ = 0 (W2), the submitted IG was inaccurate. On BUSI, 12 steps leave a median completeness gap of 67% (< 5% with ≥ 256 steps), although localisation is unchanged (IoU 0.159 at 12 and 2,048 steps). On SST-2, the zero-embedding baseline is degenerate for ModernBERT: the submitted 16-step IG has a completeness gap, and refining it drives localisation towards chance (3 seeds: pointing-game accuracy from 0.449 to 0.204; random 0.182). IG with a padding-token baseline converges (median gap 1.3% at 128 steps) with similar localisation (IoU 0.259 vs 0.273 submitted).

B. On Iris, ℓ1 alone contributes 41-61% of the K_eff reduction that Section 5 of the submission attributes to entropy.

Revisions. In the updated PDF, new and revised parts are in red. We position FLARE as a first-order ANFIS/normalised-RBF/Gaussian-gated mixture-of-experts; narrow the claims in the abstract, introduction, contributions, and discussion; extend Table 2 with the head controls and reference maps; state the λ values and correct IG; remove the single-cell experiment, which explains the base Transformer rather than FLARE; and report all new experiments in Appendix E.

In summary, the evidence does not support that the rule structure links decisions to input evidence. It supports that, on frozen deep representations, a TS head replaces a linear head at no accuracy cost; its parsimony objective works on deep representations that support several rules and collapses to a linear head when they do not; and an exact per-rule decomposition gives a rule-level view of the attribution. We thank the reviewer again for the constructive feedback, which has substantially changed how we position this work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.