- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes a method to evaluate music alignment as well as its music quality in text-to-music generation problem by using a comparative DuetJudge approach. The comparative preference scores of two audio clips are expressed in terms of probability scores that are used as reward for training RL to optimize the policy. The experimental data and procedures are clearly described and the results are slightly better in comparison to other baselines. They provide a feasible method that deal with the difficulty of the quantize the music quality and alignment of text-to-music generation problem.
Critical strengths
The paper proposes a comparative duetjudge concept as well as a RL framework into training of the evaluation model on music generation in terms of text-alignment and music quality. The experimental data and procedures are carefully and well presented and the results are slightly better than previous baselines. The method provide an alternative music evaluation method to deal with otherwise difficult to assess the specification alignment of prompt text and generated music quality.
Critical weaknesses
The correctness of RL policy learning relies on accurate reward models. converting comparative probability to a scalar reward score tends to be weak in theoretical justification. Is the converting mapping linear or non-linear (I suspect) is not clear. Intuitively, high probability indicates the high reward (or preference), low probability represents low reward. But the corresponding function is not clear by converting probability into actual reward for RL optimization. It can be sensitive if the mapping functions differ. How many comparisons are enough to collect the sufficient data for constructing the accurate reward score for all correct judgement of music quality is not addressed. The DuetJudge reward score concept somewhat borrows from Bradely-Terry model , it also inherits the weakness of the model such as the cyclic preference comparison (transitivity might not hold) problem, context sensitivity problem. In the sense, the novelty of the paper adopting Duet-Judge approach is not much difference than BT model. But the paper adopts the probability reward score and then refine with RL to obtain slightly better performance against DuetJudge-BTRM as shown in the ablation study results in Table 4 of the paper.
The comparative preference scoring of audio clips is based on 10 seconds which is a local score. How well can local preference rewards on text alignment or music quality can be accumulated into overall evaluation on a global preference concept on the generated music is still a concern. Can RL optimization model really learn the global concept quality by accumulating the local preference reward is not clearly justified. Because RL is based on dynamic programming principle, where global optimality score relies on accumulation of local optimality score while global music properties or quality might not follow. When the local scores are assessed according to the local preference with respect to certain global metric, it may not reflect accurately that cause bias. Therefore the accumulate local scores cannot reflect correct global metric score on the whole music sequence evaluation. For example if supposed a global uniformity is to be evaluated, assuming clip uniformity x == y > z, if one sequence consists of alternating clips x, y, x, y, x, y while another sequence consists of z, z, z, z, z, z, then apparently each local uniformity comparison clip wise no matter x or y is greater than z, however the global uniformity score the latter sequence is greater than former one.
To deal with Tie, there is extension called Bradley-Terry- Davidson model that assign probability score to TIE situations. The paper did not discuss with this extension.
Questions for authors
Table 2 The tie tends to be higher than other base-line models, does that imply the proposed model tend to avoid ambiguities so that the performance tend to be better (like recall and precision tradeoff)? Also as the paper writes: “We predict Tie if |P(A)−P(B)|<0.05 or P(Tie) >0.25; otherwise, we choose A or B according to which has the higher probability.”, the threshold parameter of TIE can affect the performance evaluation of the model. It is not justified how the threshold is decided (optimally?)