- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies whether reweighting LLM-simulated categorical responses improves over direct estimation with the same real human sample that is used for reweighting. Theoretical and empirical analyses demonstrate that reweighting does not have substantial advantages over direct estimation.
Critical strengths
None
Critical weaknesses
The writing in this paper is extremely difficult to follow.
-
Many technical terminologies and notations are used without definitions or explanations on how they relate to the problem (e.g., , , "plateau", "recovery-screen" in Section 3.3). Moreover, a large number of terminologies that appear later in the text are referred to in much earlier sections, without any explanations on their purpose. For example, Section 3 refers to experiment details in Section 5 (e.g., "human-target studies" and "LLM-teacher studies" in lines 142-143, and "The Chinese and English threshold studies" in line 180) without giving any explanations.
-
The theory in Section 4 is presented with barely any motivations. The theorem statements themselves are also hard to understand. In particular, the results in Section 4.3 are basically stated in words, with a number of terminologies unexplained. The paper offers almost no explanations and interpretations of the theorems. It is completely unclear to me how the theorems relate to each other, and contribute to the conclusions of the paper.
-
The experiments in Section 5 have similar presentation issues. It reports numerous experiment results without clearly explaining the experiment goals and setups, interpretations of the findings, and how they support the paper's conclusions.
Questions for authors
None