D²S: Dual-Perspective Dynamic Data Selection for Supervised Fine-Tuning

Effective data selection is crucial for supervised fine-tuning under limited training budgets. Existing data selection methods can be broadly categorized into data-centric and model-centric approaches. Data-centric methods rely on reusable static scores but do not directly capture the evolving learning needs of the model, whereas model-centric methods reflect the current learning needs but incur substantial costs when repeatedly evaluating large data pools. To address these issues, we introduce Dual-perspective Dynamic Data Selection (D${}^2$S), a framework that first performs coarse selection from the data perspective and then refines the candidates from the model perspective. The resulting model-perspective feedback is used to guide subsequent data-perspective selection, forming an iterative selection process which adapts to the evolving learning needs of the model while restricting expensive model evaluation to a small candidate set. To provide fine-grained model-perspective feedback, we further introduce Model-aware Scoring (MaS), which distinguishes between important and unimportant response tokens based on the model's prediction probabilities, applies different evaluation criteria to each group, and aggregates the resulting scores into a sample-level score. Experiments on three data pools and two base models show that our framework achieves an average relative improvement of 3.6% over the best baseline and 16.4% over full-data training, while reducing average runtime by 6.1% and 66.2%, respectively.