How should Language Models See Tree Models? Adaptive Representations for Reasoning

Language models increasingly reason over compact artifacts produced by machine learning pipelines rather than raw data. For tabular problems, a fitted decision tree or boosted tree ensemble is a natural such artifact, yet the language model sees only a textual representation of it. Prior work typically fixes this representation, making its effect difficult to separate from the rest of the pipeline. We introduce a representation controlled framework that varies only the tree to language representation, holding the fitted model, the language model, and the evaluation pipeline fixed. Across two tasks (rule generation and budgeted counterfactual search), five public datasets, and three language models, representation choice has a large and highly heterogeneous effect. No representation is consistently best across datasets or models, and on some datasets the spread exceeds sixty recall points. This variability motivates studying the representation as part of an iterative inference process. We first study fixed-view iteration, which keeps the tree representation unchanged across rounds, and then introduce DRALoop, which re-points the exposed information toward the current unresolved state while keeping both models frozen. Iteration alone provides substantial gains over one-shot reasoning, while DRALoop adds beyond this matched control in regimes with residual headroom. On rule generation, it exceeds the best one-shot fixed representation on all five datasets for all three language models. On counterfactual search, its strongest configuration reaches 92.0% of the attainable validity bound, compared with 81.6% for matched fixed-view iteration. These results suggest that when one learned system reasons over another, the representation connecting them can be treated as a controllable and adaptive component of inference.