When Reuse Requires New Data: Deciding What to Measure and When to Stop

Reusing past data can reduce the need for new data collection, but discovering how to use it in a new environment can itself require costly measurements. We study this as a decision problem whose objective is to minimize the combined burden of prediction error and measurement cost, rather than pursue accuracy regardless of the cost of obtaining it. We develop a procedure that uses selected measurements from the new environment to choose among predictors trained on past data and correct their predictions for other, unmeasured cases. Known relationships between cases guide the choice of what to measure. Before collecting more, the procedure considers possible results and calculates which predictor it would choose and how it would correct the predictions, without retraining the source models. It continues only when the expected improvement justifies the cost, then updates the choice and corrections using the actual result. The decision to acquire more is thus based on the prediction improvement expected from using the results, rather than uncertainty reduction alone. We evaluate using existing records from two sources: the ChEMBL bioactivity database, recording how strongly molecules bind to proteins, and the Hardware-adaptive Efficient Latency Predictor (HELP) study, recording how long neural networks run on devices. In one molecular example, the procedure uses 28.6% fewer measurements than a reference that uses all candidate measurements for the same predictor selection and correction, with an observed increase of about 0.7% in log-scale mean absolute error. In one hardware example, it uses 89.1% fewer measurements than the corresponding reference, with an observed increase of about 26% in that error measure. Mechanism analyses examine how three elements interact within the procedure: relationships that let measurements inform other cases, corrections that match the errors that remain, and judgments of whether further measurement is worth its cost.