- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PODS, a dynamic data-volume scheduling method that periodically alternates between low- and high-ratio training phases to improve training efficiency under a prescribed cumulative data budget. The authors explain the strategy through the trade-off between gradient stochasticity and optimization fidelity and derive an analytical scheduling rule. Experiments across image classification, object detection, and LLM fine-tuning demonstrate that PODS can preserve or improve model performance while reducing training costs.
Critical strengths
- The method is simple and practically useful, PODS requires no additional scheduling parameter search and can be integrated with existing data selection methods. Its simple design makes it easy to incorporate into existing training pipelines.
- Extensive experimental evaluation. The paper evaluates the effectiveness of PODS across diverse tasks and model architectures.
Critical weaknesses
-
Prior studies, such as SCAN (ICCV 2025) [1] and Music (Pattern Recognition 2024) [2], have already explored dynamically adjusting the data selection ratio during training. Although the two-phase oscillatory schedule in PODS has some distinctive characteristics, the paper does not sufficiently discuss its differences from these existing approaches.
-
Proposition 3.1 derives the effect of the selection ratio on gradient covariance based on the average gradient of a uniformly sampled subset. However, the actual method employs mini-batch optimization with a fixed batch size and loss-based hard mining. Under uniform sampling, the marginal covariance of an individual mini-batch gradient does not directly depend on the size of the intermediate subset. Therefore, the current theoretical analysis does not sufficiently explain how oscillatory data-volume scheduling induces the claimed stochastic regularization effect in the actual training procedure, nor does it establish a direct connection between this effect and improved generalization.
[1] Guo, Y., & Kankanhalli, M. (2025). SCAN: Bootstrapping Contrastive Pre-training for Data Efficiency. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 3662–3672.
[2] Yin, Z., Pu, J., Wan, R., & Xue, X. (2024). Embrace Sustainable AI: Dynamic Data Subset Selection for Image Classification. Pattern Recognition, 151, 110392.
Questions for authors
- What is the core novelty of PODS compared with existing dynamic selection-ratio scheduling methods such as SCAN? What are the advantages of the specific oscillatory schedule?
- How does Proposition 3.1 relate to the actual mini-batch optimization procedure and loss-based hard mining? Can the authors further explain the mechanism by which the selection ratio affects gradient stochasticity and model generalization in practice?
4 comments
Looking forward to the reply
Dear Reviewer NtKB,
Thank you again for your time and constructive feedback. Following your suggestions, we have conducted additional experiments and further discussed related work. We have also revised Sec. 2 of the manuscript to better highlight the distinctions of PODS.
As the discussion period is approaching its end, we would greatly appreciate your feedback on whether our responses have adequately addressed your concerns. If there are further concerns or questions, we are willing to address them.
Thanks again for taking the time to review our work and provide insightful comments.
Authors of Submission 3402
Thanks for the detailed response. My concerns have been mostly addressed, and I've decided to change the rating to weak accept.
Dear Reviewer NtKB,
Thank you very much for your positive feedback and recognition of our work. We are pleased that our responses and revisions have addressed your concerns. We sincerely appreciate your constructive suggestions, which have helped strengthen our work.
Best regards,
Authors of Submission 3402
Dear Reviewer NtKB,
Thank you for the careful review and constructive suggestions. We appreciate your recognition of our work's strengths, e.g., practical effectiveness, simple and tuning-free design, seamless integration with existing training pipelines, and extensive empirical validation.
We provide responses to address the comments below:
Q1/2: Differences from SCAN [a] and MUSIC [b].
A1/2: Thanks for suggesting additional related works. We would like to clarify that while SCAN employs iterative dataset mutation with cosine-based pruning for contrastive pre-training, and MUSIC adaptively determines subset ratios through performance-driven feedback for image classification, PODS explicitly investigates oscillatory data-volume allocation under a fixed cumulative training budget.
Our key advantage lies in its principled, lightweight, and broadly applicable scheduling design: PODS periodically alternates between selective training and broader data coverage, without auxiliary optimization, feedback-driven ratio adjustment, or dataset-specific tuning. Moreover, PODS decouples data-volume scheduling from sample importance estimation, enabling direct integration with existing advanced selectors. Experiments in Sec. 4 further demonstrate its broad applicability across vision and language tasks. Thus, PODS advances dynamic data selection through a simple, budget-constrained oscillatory scheduling framework with independently validated effectiveness and broad applicability.
Following your suggestion, we have included both works [a,b] in Section 2 of the revised manuscript. Specifically,
Our experiments provide direct empirical support. As shown in Fig. 5, measured gradient variance during low-ratio phases is 2.0×/2.6× higher than during high-ratio phases. Importantly, suppressing this variability consistently degrades performance, supporting its contribution to the observed performance gains. Moreover, Table C-1 below demonstrates that oscillatory scheduling improves performance even with random selection, establishing its effectiveness independently of hard mining.
We have clarified the theoretical scope in Sec. 3.2 of the revised manuscript. Together, these results connect stage-wise data-volume control to practical optimization dynamics, with controlled experiments demonstrating the effectiveness of oscillatory scheduling.
Table C-1: Effectiveness of PODS with random selection strategies.