- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper explores a modality-aware curriculum in the context of the memory-augmented video agent. To solve the issue of long-tail modality distribution in VideoQA caused by text-heavy questions, this paper proposes a curriculum-based training environment named VideoGym that adapts VideoAgent’s training distribution online. The authors conducted experiments on five long-video benchmarks. The results show that the proposed VideoGym improves performance by a large margin.
Critical strengths
-
The exploration of modality-aware curriculum in video agents is technically sound and shows great sample efficiency in training.
-
This paper is well-organized and easy to read. The authors provide code, experimental details, and computing costs, which would help researchers follow the work.
-
The experimental results on multiple long-video qa benchmarks validated the effectiveness of the proposed VideoGym.
Critical weaknesses
-
The claim that "performance gap tracks the imbalance in the training distribution" needs more evidence to support it. In Table 2, the results of the training-free method also show a performance gap. Therefore, the performance gap may not be caused by imbalance in the training distribution. The authors should provide more experiments on the distribution of data to validate the claim.
-
The curriculum algorithm is based on EXP3, and the contribution is limited. In addition, the composite reward formulation (Eq. 4) is different from Fig. 2.
-
There are some typos. For example, "fine-gained frames" in Fig. 1.
Questions for authors
After reading this paper, my bigger concern is that the motivation of this paper is hard to understand. The authors should give more evidence to support the core claim that "performance gap tracks the imbalance in the training distribution" in the Introduction.