FLARE: A Differentiable and Parsimonious Fuzzy Rule Layer for Grounding Learned Representations

Neural networks learn representations that support accurate prediction, but the decision structure built on these representations is typically opaque. Attribution methods can identify input features associated with individual predictions, while rule-based models expose explicit decision logic; connecting these two forms of interpretability remains challenging. We introduce FLARE, a differentiable and parsimonious fuzzy rule layer for learned representations. FLARE replaces a conventional prediction head with a first-order Takagi–Sugeno system whose Gaussian antecedents define fuzzy regions of representation space and whose local linear consequents produce the prediction. An explicit regularisation objective encourages concentrated rule usage and sparse consequents. Because the complete predictor remains differentiable, its decisions can be traced through the representation model back to the input domain. Across tabular, image, and text experiments, FLARE achieves predictive performance close to conventional predictors while providing an explicit rule-structured decision layer. On breast ultrasound image classification and sentiment analysis, attribution of the fuzzy predictor consistently concentrates on independently defined lesion masks and sentiment annotations across multiple localisation measures and random seeds. FLARE therefore connects three levels that are usually studied separately: learned representations, explicit rule-based decisions, and the input evidence associated with those decisions.