Hinging Hyperplanes Tree: Transforming ReLU Neural Networks into Equivalent Decision Trees

ReLU neural networks define continuous piecewise-affine functions, but this function-level structure is represented implicitly through neuron activations. We introduce the Hinging Hyperplanes Tree (HHT), which converts a fixed ReLU network into a tree of affine comparisons with affine output rules at its leaves. HHT is constructed from locally extracted affine functions and refined through leaf-wise linear-programming containment checks. We prove finite convergence of the sample-level construction and show that, once all leaves are certified, the resulting HHT is functionally equivalent to the source network over the prescribed bounded domain. Unlike a forward pass, which reveals the affine rule only at a queried input, a certified HHT provides a reusable polyhedral representation of the network's piecewise-affine function. Experiments demonstrate exact reconstruction on the studied ReLU MLPs and characterize the structural and computational trade-offs relative to existing network-to-tree representations.