- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LaDex, a method that evaluates response diversity in latent space. It computes latent distances among the responses within a group at stage level. Then weights them by correctness through a Latent Classifier Network (LCN), and uses the result as an additional reward to guide learning. The method gives consistent improvements across multiple benchmarks and settings.
Critical strengths
The method is clearly structured, the problem is well defined, the experiments are thorough, and the appendix provides plenty of additional results.
Critical weaknesses
I do have some concerns:
- I’m not convinced that computing distances in latent space is necessary. The paper does not show whether a simple token-level distance would also work.
- The paper should briefly explain how temporal resampling is done. It directly addresses how stages are divided when responses differ in length, so it deserves a clear description. I'd also like some other details clarified. For example, how is splitting stages by “\n\n” guaranteed to work? The model’s use of this delimiter for segmentation is not consistent, and how the model produces the content of each stage is itself a form of diversity.
- Computing the diversity value as the average within-group distance is quite simple. Simple methods can of course be effective, but further explain how this distance behaves under different distributions, for example, when all responses in a group converge to the same paradigm, or when the distances between different paradigms are uneven...
- The paper shows that the average latent distance increases, but under the designed objective this is almost an inevitable outcome. It does not show a direct link between latent distance and actual diversity. Some examples, such as a clear increase in the number of distinct response paradigms, would help.
Questions for authors
See weaknesses above.