A Pre-Registered Audit of Hierarchical Retrieval: Cost, Utility, and Assignment Confidence

Evaluating hierarchical retrieval requires measuring what its structure costs, whether it improves answers, and what its stored confidence can support. We audit these questions inside one RAPTOR-style pipeline. A capacity-constrained (truncated) variational DPMM (Dirichlet-process mixture model) serves as a confidence-bearing audit instrument rather than a proposed method: its fit stores a posterior responsibility per node–cluster pair, and the index keeps a scalar derived from it. In pre-registered comparisons on NovelQA (60 novels, 1,526 questions), the hypothesis that this index outperforms both the GMM+BIC and the Leiden index is not supported under either writer–reader backbone, and the registered comparison with flat dense retrieval does not establish a semantic-tree advantage. We report associations, not causal claims beyond this pipeline. The DP index stores 6,205,729 summary tokens against 1,602,734 for fixed-block contiguous grouping, without an established paired accuracy advantage (0.5770 against 0.5775). Rebuilds under 4 further construction seeds yield median within-book accuracy standard deviations of 0.0368 to 0.0480 over 54 books, at the scale of the registered design target. On an eighteen-book development sample, outside the registration, maximum a posteriori reassignment under sampled variational parameters stays close to the fitted partition (mean adjusted Rand index 0.97), whereas a refit on row-permuted embeddings lands farther (0.72). A fixed confidence reranker lowers book-mean accuracy from 0.5770 to 0.5589, a descriptive result under the registered testing sequence. These results motivate reporting utility beside construction footprint and rebuild variation, and testing stored confidence against both the variation it is meant to track and the decision it is meant to inform.