- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the "constrained-option scoring" scenario in LLM evaluation, where a prompt presents the model with a multiple-choice question and the task is to forecast whether the model will answer it correctly. The authors find that the forecast ranks the model's correctness at chance level, analyze the answer distributions, and find that prefilling an answer stem increases the success rate.
Critical strengths
None
Critical weaknesses
The paper is overall incomprehensible, and I can't make much sense of what it is trying to do after reading it several times. The scope of this paper is very small (and I would even say trivial or irrelevant). It focuses on the niche (and clearly outdated) scenario where the model must choose from a fixed set of answers.
This so-called "constrained-option scoring" is essentially a classification problem. In real-world scenarios, we would almost certainly use an embedding or reranking model for them. If it's a reasoning-intensive classification problem (e.g., a problem in GPQA, or a medical diagnosis / financial decision in the real world), then it's the reasoning process that actually matters, not the answer.
The only scenario that I can think of where "constrained-option scoring" could be useful is language model evaluation on simple benchmarks (like the example in Figure 1). However, instruction following is also one aspect of language models' capabilities to be evaluated. If we explicitly asked the model to answer with one of A, B, C, D and the model generated something else, then it means the model is not strong enough and doesn't have much real-world use. Trying to extract one answer from the model's logits doesn't achieve anything in this case.
An additional note on the example in Figure 1 (and the complete one in Appendix B): this example doesn't make any sense. Based on my understanding, the prompt says explicitly that the answer to a problem is A, and asks the model to predict how many future readers will answer it correctly, without providing any information on who these future readers are. So this question is unanswerable.
Questions for authors
- I shall ask this one single question: is there any real-world implication of "constrained-option scoring"?
Answer to Author's Rebuttal
I would like to thank the reviewers for their rebuttal and I would like to apologize for the late reply. While I very much appreciate that the authors took the effort of rewriting large parts of the paper, it is still extremely hard to read and follow. Further, I am not convinced by the practicality argument of the authors. I believe that many benchmarks can be improved by prompting the model differently or reading out answers in a different way which is inherently connected to how the model was posttrained. Still, I do not think that the contribution is significant enough to meet the standards of ICLR. I am keeping my score. Thank you again for spending time on rebottling and communicating to the reviewers.