- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper propose a decomposition of the squared error of LLM's action (conditioned on sequential actions) with three parts, variability (for different repeated predictions), epistemic (for error between the average prediction and the true posterior), and aleatoric (the entropy of posterior itself). The authors give a good and detailed discussion on estimation of these components. They also demonstrate this framework on three real-world datasets and give interesting interpretations on suggested interventions based on the decomposition results.
Critical strengths
- The paper is technically sound, offering definitions and proof for every statement. I didn't find any error in it. It is also fun to read and easy to follow.
- The idea of decomposition is useful for identifying the reason of prediction error. The demonstrations show interesting use case and practical takeaway.
Critical weaknesses
- I didn't understand the section 4.1 completely. It's a little hard to see the connection between this section and the big picture of the paper. Therefore, I'm not sure how the proposed decomposition is related to a sequence of predictions, compared to just a single prediction.
- A miscalibrated proxy could make the framework useless. In figure 2c and d, we wouldn't know whether it is because the average prediction by LLM is bad or the label is uncertain by itself. It would be great if the authors can include some results between the calibration of the proxy and the quality of the decomposition in the main texts.
Questions for authors
- What is the usage of the section 4.1 and its connect to VEA decomposition? And how does it make the framework related to sequence of predictions?
- How would the authors interpret the results in 5.3 given that the proxy is not calibrated (or truthful)?
2 comments
Thank you again for your constructive feedback. We have included additional analyses and clarifications in our rebuttal in response to your comments. We know you have a busy schedule and would appreciate any further feedback if you have a chance to review our response. We would be happy to address any remaining concerns.
We thank the reviewer for the positive assessment of the paper and for two insightful and constructive questions, which have helped us improve the paper greatly.
Weakness 1 and question 1
Section 4.1 and its connection to VEA. The VEA decomposition is developed for the total squared error, but it does not tell practitioners when, and how often, V and E lead to a very large error in a single run. We are interested in the agreement between one probability prediction pi and the true replication probability πi. We show (page 4, lines 184-185) that the sum of V and E, the key ingredients of the VEA decomposition, equals E[(pi−πi)2]. This connection lets us study the probabilistic behavior of the agreement. It motivated the two bounds (Freedman and empirical Bernstein) on the deviation ∣pi−πi∣, which describe the chance that the deviation is unduly large and identify the factors that control that chance. This knowledge is useful for designing suitable interventions. We have revised the opening of Section 4.1 (page 6, lines 309-316) to state this connection.
Sequential versus a single zero-shot prediction. The actions are naturally sequential, in a divide-and-conquer spirit, which gives more insight into the profiles of variability, uncertainty and error in a complex decision process. In a designed chain of actions, we can identify the "trouble-makers" that contribute unduly large errors or uncertainties, so that interventions can be applied at the right place and at the right time. A chain may cost more tokens and computation than a single zero-shot prediction, and more actions may introduce more errors. Nevertheless, in important scientific studies, knowing the detailed sources of uncertainty is of great interest for reproducibility and replicability. The opening of Section 4 now says this (page 6, lines 281-286).
Weakness 2
A toolbox to check the calibration of the proxy. We agree on the importance of a well-calibrated proxy for an appropriate evaluation of uncertainty. At a minimal level of assurance, we propose the Tier-3 diagnostic as a toolbox to check the calibration of the proxy. The observed mean squared error minus the estimated V and E terms, n−1∑iR−1∑r(pi(r)−Yi)2−n−1∑i(V^i+E~i), estimates the mean aleatoric term n−1∑iπi(1−πi), and it agrees with the plug-in π^(1−π^) only when the proxy is calibrated in aggregate. The rationale of the check rests on a sufficient condition: a proxy calibrated in aggregate passes it, so a failure flags the proxy. Figure 4 in Appendix E shows the check for each dataset, Appendix E adds a discussion of the results, and Section 3.2 now refers to both (page 5, lines 246-247). RPP passes; CB and SSRP do not. Section 5.4 now also identifies the results that do not depend on the proxy at all (pages 9-10, lines 485-489).
Question 2
We reiterate that the Tier-3 diagnostic checks whether the proxy (the logistic prediction model) is adequately calibrated. When the check fails, as on CB, we do not suggest stopping the analysis.
Some results in Section 5.3 for CB and SSRP deserve special attention: the small V values, the dominant action-specific E values, and the closeness of the scoring action to its proxy. Because the proxy fails the calibration check on both datasets, results that involve the proxy at the scoring action should be interpreted with caution. There are two exceptions, where the miscalibration of the proxy does not affect the estimates.
The variability estimates, because they do not depend on the proxy.
The epistemic components of the extraction and statistical-review actions. These actions are upstream of, and separate from, the scoring action, and their references are the dataset's recorded values (N,p,r) and the statistical power computed from the original sample size and effect size, rather than the proxy, so their epistemic components can be evaluated appropriately.
Section 5.4 now states this (pages 9-10, lines 485-489). For the scoring action on CB, Section 3.2 already notes that a large epistemic component there measures the disagreement between two imperfect predictors, and that the LLM predicts the CB outcomes better than the proxy does (page 5, lines 250-253).
We are grateful for your encouraging assessment that the paper is technically sound and easy to follow. Your two questions led us to explain how Section 4.1 connects to the VEA decomposition and to make the calibration check of the proxy visible in the main text. We would be happy to answer any further questions.