$\delta$-Approximate Structural Transformation of Decision Trees into Fixed-Depth Neural Networks

Neural networks (NNs) are effective decision-making models, but their decision logic is often difficult to inspect and verify. A large body of research has focused on converting NNs into interpretable models to facilitate their analysis and modification. Decision trees are an attractive representation since they expose their input space partitions and can therefore be used to understand the decisions within each region. However, decision trees can be inconvenient to deploy in standard neural inference pipelines and are not directly compatible with the ecosystem of tools developed for NN verification. A reverse transformation is therefore desirable to move between interpretable and neural models while preserving their behavior. We address this problem by focusing on the transformation from decision trees to NNs. We first discuss a constructive exact transformation from decision trees to rectified linear unit (ReLU) NNs, showing that every decision tree admits an equivalent neural realization, but also that exactness can cause substantial growth in the size of the resulting neural model. We address this issue by introducing a $\delta$-approximate transformation method that maps a decision tree to a fixed-depth NN while relaxing exact equivalence. Specifically, our approximation guarantees exact agreement of the neural model with the decision tree outside a $\delta$-dependent neighborhood, thereby confining possible prediction discrepancies to this neighborhood. We evaluate the transformation on neural control policies on two benchmarks from the OpenAI Gym environment. Across policies obtained from multiple NN architectures and independently trained models, we evaluate the fidelity of the converted NN with respect to its source decision tree, closed-loop performance, and preservation of formally verified properties. We further evaluate a combined pruning-and-conversion pipeline, demonstrating a tunable trade-off between policy fidelity and the size of the resulting network. Overall, our method enables a practical workflow in which policies can be analyzed and modified as interpretable decision trees and then deployed as compact NNs.