- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The manuscript addresses the challenges of multimodal evidence reliability and scoring in automated speech assessment and proposes an agent-based evaluation framework, AIMPACT. The framework generates preliminary scores through multimodal evidence extraction, temporal evidence memory, and reliability-guided multi-agent collaboration. The authors further introduce RC-MEPO, which combines supervised fine-tuning with online preference optimization to align model-generated scores with the score distributions of multiple human experts and improve adaptation to different evaluation criteria. Experimental results show that the proposed framework outperforms existing multimodal large language models.
Critical strengths
-
The research problem has clear practical relevance.
-
The proposed AIMPACT framework is reasonably designed and comprehensive.
-
The experimental results provide strong evidence for the effectiveness of AIMPACT.
Critical weaknesses
-
The authors should further verify whether the performance gains arise from structured multimodal features and expert supervision, or from multi-agent collaboration.
-
The novelty of RC-MEPO appears limited, as it mainly lies in the combination of different reward designs.
-
During SFT, the authors introduce 30% expert correction, but the manuscript lacks a detailed explanation or analysis of this design choice.
-
MEPA contains only 35 independent speech samples. The dataset size is limited, and evaluations across multiple dimensions should not be treated as independent samples.
-
The authors are encouraged to provide additional experimental analysis of computational overhead.
Questions for authors
The manuscript excessively compresses the vertical spacing around tables. Please refer to the weaknesses above for the remaining concerns.
Thank you for the additional experiments and detailed clarifications. The fine-tuned baselines and same-backbone comparisons substantially address my concern about comparison fairness, and the accuracy-latency analysis better justifies MACG for offline assessment. However, despite the reassuring overfitting analyses, the small number of independent presentations still limits the evidence for generalization, so I maintain my recommendation.