- Rating
- 1
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces Infer-ZO, a system that separates zeroth-order optimization algorithms from backend execution. It represents compatible perturbations and accumulated updates using low-rank factors, reducing repeated dense-weight writes and allowing optimization evaluations to run through an inference engine alongside ordinary serving requests.
Critical strengths
Infer-ZO improves the efficiency of ZO optimization by representing perturbations and accumulated updates as low-rank adapters, reducing costly dense weight writes and enabling inference-engine optimizations.
Critical weaknesses
There are system improvement in decomposing the ZO system, but the algorithmic novelty is limited. A system-level optimization for a non-production-useable method ZO has very limited contribution. The experiments are done on SST-2, BoolQ, and CB. These datasets are older datasets and covers very limited capabilities of modern LLMs. The author reported the performance on dev set and validation set, and therefore does not fully establish whether performance are on-par with earlier methods. The model experimented are mainly small models with very limited training steps. It's unclear if the result will transfer to test set in more realistic production setting.
Questions for authors
Could you provide GPU hours vs evaluation results of the proposed Infer-ZO compared to gradient decent methods? What are the performance of the proposed infer-ZO perform after trained to convergence on more recent tasks?