Compose Then Scalarize: Joint-State Valuation for Data Allocation

Allocating a limited data budget across domains requires estimating the value of candidate additions from limited training evidence. Independent scalar-gain valuation based on single-domain pilots predicts each domain’s task-score gain independently of allocations to other domains. When the task score aggregates multiple metrics through a nonlinear utility, however, a candidate's marginal value can change with the joint allocation. We propose Compose-Then-Scalarize (CTS), a joint-state valuation method that reuses fixed single-domain pilot evidence to reassess candidate additions as the joint allocation changes. CTS fits signed metric-response curves, additively composes their predictions across domains into a joint metric state, and applies the utility to obtain state-dependent candidate values without additional pilot training. Experiments on autonomous driving and satellite image classification show that CTS achieves the highest mean task score at every evaluated allocation size. Compared with the strongest of the seven baselines at matched budgets, CTS improves the composite driving score by 0.71–1.61 points and worst-region accuracy by 0.93–3.21 percentage points. Matched ablations support the benefit of joint-state valuation, while controlled retraining shows that CTS more accurately predicts background-dependent candidate preferences than a matched control that values candidates at their own-domain states. Code is available at https://anonymous.4open.science/r/ICLR-CTS.