- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper implements a number of paradigms that various other researchers in the field have created, on a range of different model sizes and families. It tests privileged self access, through measuring whether other models can predict self-report labels as well as the model itself, and generalization of fine-tuning on one introspection task to others, on the logic that failure means the training taught shortcuts. They authors report that models mostly do not outperform external observers, and that training mostly does not transfer across tasks. The authors conclude that the evaluations cannot establish introspection.
Critical strengths
The paper adopts a sound definition of introspection, and critical appraisals of reported results in the field are welcome. It includes an impressive breadth of tests and a diverse set of models.
Critical weaknesses
Such a paper's conclusions can only be as strong as the paradigms it uses, and there are far too many for me to validate either their logic as valid tests of introspection or the correctness of their implementation. In general these paradigms are not widely used "benchmarks" as the paper claims, but are custom made for specific investigations.
I take it as given that there are numerous overclaims about introspection in the literature, but this paper seems to take the opposite tack, either shrugging off or ignoring the four positive self-access tasks, the numerous off-diagonal transfers, and the fact that Qwen-27B in particular looks to be relatively robust in terms of transferring to other tasks, except for steering.
Questions for authors
You suggest that Qwen-27B's "large gain may reflect answer-format learning" but say you didn't test it. Why not? If you want to claim that "training on introspection tasks strengthens task-specific rationalization rather than privileged access" it seems important to settle whether the gains are indeed task-specific.
3 comments
I appreciate the authors' commitment to truth, but the fact that they discovered a coding error that substantially changed some results in response to my question does not allay my concerns about my (and anyone's) ability to validate such a large number of paradigms. Therefore, I maintain my score.
We thank the reviewer for the reply. We would like to clarify that the issue was a configuration setting rather than a coding error, affected only the zero-shot baseline rows of the Qwen models, and did not change the conclusions of the paper, but in fact strengthened them.
Regarding verifiability, could the reviewer suggest concrete steps that would make the implementation easier to validate? We have already released all the code, which follows the original implementation of each paradigm, and Appendix C maps every reported number to the command that produces it. We would like to make sure that we don't face a similar criticism in future revisions.
We thank the reviewer for the critical reading and for appreciating both our definition of introspection and the breadth of our evaluation. Below, we address the questions and concerns. If those address or mitigate your concerns, we would appreciate it if you could update the score accordingly.
[W1] R: "there are far too many paradigms for me to validate either their logic as valid tests of introspection or the correctness of their implementation."
We understand that the number of tasks makes them hard to check within a review. Regarding implementation, each task is taken from the literature and follows the original data and label definitions. Appendix C maps every reported number to the command that produces it, so that any single paradigm can be checked independently.
Regarding the logic of the paradigms, our experiments take existing introspection tasks and ask whether each one, as defined in the paper that introduced it, can be solved without privileged access.
R: "In general these paradigms are not widely used "benchmarks" as the paper claims, but are custom made for specific investigations."
We agree that these tasks are not widely used "benchmarks", and now call them "evaluation paradigms" throughout the paper to reflect the reviewer's suggestion. They are, however, the paradigms on which published introspection claims rest, which is why we test them.
[W2] R: "I take it as given that there are numerous overclaims about introspection in the literature, but this paper seems to take the opposite tack, either shrugging off or ignoring the four positive self-access tasks, the numerous off-diagonal transfers, and the fact that Qwen-27B in particular looks to be relatively robust in terms of transferring to other tasks, except for steering."
Our aim is not to argue that models cannot introspect, but to identify which positive results current paradigms can attribute to privileged access. We agree the submission discussed the positive evidence too briefly, and the revision adds a section, "What positive evidence survives?" (Sec. 5.3), which examines it. Below we discuss the positive evidence the reviewer listed:
Four positive self-access tasks. On 1st char, 1st word, among, and ethical stance, the self-reporter keeps a residual advantage over external reporters (+4.4 on average), although external reporters still learn these tasks well without privileged access. Our test cannot attribute this advantage to introspection rather than self-simulation or familiarity with one's own outputs. The submission noted this only briefly (abstract and Sec. 5.1). The revision discusses it explicitly (Sec. 5.1 and 5.3) and corrects the Conclusions, which mentioned only self-simulation, to state that a small privileged component remains possible.
Qwen3.5-27B. Its broad transfer came from an error with the thinking mode in the baseline of Qwen models (see Q1 for details). With the corrected baseline it no longer stands out.
Off-diagonal transfers. With the corrected baseline, none of the seven significant off-diagonal gains of the submission remains significant (App. B.9). The clearest remaining transfers are between tasks that share their target (e.g., predicting the first word fixes the first character), which follows from learning regularities of the model's outputs that external reporters also learn (Sec. 5.1). The largest remaining gains (Ministral-3-14B, into confidence reporting tasks) are a dataset-specific response bias: they come from the same few confidence datasets whichever task was trained, while on other datasets the same adapters fall below the majority class.
[Q1] R: "You suggest that Qwen-27B's "large gain may reflect answer-format learning" but say you didn't test it. Why not?"
Thank you for spotting this outlier and pressing this point. Re-testing it revealed an error in our zero-shot baseline, which we have now corrected. The script for computing the zero-shot baseline did not disable Qwen's default thinking mode, while the training script correctly did. Therefore, the untrained Qwen models (i.e. the zero-shot baselines rows in Tables 1 and 35) ran out of tokens while reasoning before giving an answer, and scored 0% on most tasks. This made off-diagonal gains look big but they actually are not. We also re-checked all other model configurations and no other model is affected (Gemma-4 defaults to thinking off, Llama and Ministral have no thinking mode). All the other numbers in the tables (e.g. fine-tuned results and the cross-model test) are also correct and unchanged. We have corrected Tables 1 and 35 and App. B.9.
With the corrected baseline, the off-diagonal gain disappears: after training on most other tasks, Qwen3.5-27B performs like the untrained model, and no Qwen transfer remains a significant gain (App. B.9). The in-task gains remain, so the gains are task-specific.