A new class of interpretable models: the GAM tree

The space of interpretable models consists mostly of linear models and their generalizations: generalized additive models (GAMs) and decision trees. We propose a further, natural generalization that so far has not been fully explored: the GAM tree, a decision tree having a GAM in each leaf. The GAM tree inherits the benefits of both models and alleviates some of their limitations. Through the hierarchical structure of the tree, it preserves the ability to explain decisions and to segment the data into populations. And, while a single GAM imposes a univariate function of each feature globally, the leaf GAMs make such functions locally adaptive to each population. Crucial for the success of the GAM tree is to learn the tree and the GAMs jointly in a scalable way, which we achieve using a variation of tree alternating optimization. In our experiments with classification and regression, GAM trees consistently exceed both trees and GAMs in accuracy, as expected, but the improvement is often surprisingly large. Further, the entire set of GAMs can be visualized in a single plot, which gives a global picture of the model and at the same time contrasts the differences across populations.