- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Hilbert Split, a geometry-aware data splitting method designed to improve the reliability of generalization assessment. The method first identifies response-relevant features through sensitivity analysis, then uses Hilbert curve ordering to construct local strata. Within each stratum, test samples are selected by maximizing train–test separation under a locally weighted distance metric. The paper provides theoretical results on concentration bounds and distributional discrepancy for randomized Hilbert stratification. Experiments on synthetic and real regression datasets demonstrate improved distributional coverage, reduced train–test similarity, and better generalization performance in model selection.
Critical strengths
The paper addresses an important problem in generalization assessment by balancing distributional representativeness and train–test separation. The proposed method is supported by theoretical analysis and experiments on synthetic and real datasets, demonstrating its potential to improve the reliability of model evaluation.
Critical weaknesses
Greater train-test separation does not always lead to better generalization and may introduce overly conservative evaluation. Such motivations are not well-grounded.
Selecting one sample from each stratum will not exclude similar samples. On the other hand, the optimal solution global-wise may not be obtained by the greedy method and its efficiency has not been carefully examined.
Questions for authors
Could the authors please share the rationale for maximizing train-test separation for better generalization?