- Overuse and inconsistent use of specialized terminology
As written, readers of this manuscript must connect the starting answer, advice correctness, outcome names, “same-state” versus “cross-state,” and several scores while reading the experimental setup and results. The introduction and contributions already use terms such as "validity discrimination," "adoption criterion," and "beneficial correction." There are brief explanations, but the fuller account of the measurements comes later in Section 3.2. I think the manuscript would be much more readable and valuable to the reader if ideas are expressed in plain language wherever possible, minimizing specialized phrasing. For terms that must remain, perhaps introduce the plain descriptions prior to the technical exposition, the latter point at which, the paper can show how they fit together. Perhaps, the paper could keep technical names for its important measurements and use plainer, more consistent language elsewhere. Below are specific examples that caused me to struggle in reading the paper.
-
For example, "baseline-wrong" and "initially wrong" refer to the same retained starting group: questions where the model chose the same wrong option in all three runs without advice. The paper also uses "baseline-correct" and "initially correct." I found switching between the names unnecessarily confusing. Helpful corrections are labeled A, while following wrong advice is labeled I_W or I_C depending on whether the model started wrong or correct.
-
The opening paragraph does explain "beneficial correction," "harmful regression," and "same-state invalid following." However, "benefit–harm separation" is mentioned in the introduction on page 2 before its full definition in Section 3.2 on page 4. There, benefit–harm separation (S_BH) and the cross-state criterion (c_BH) both use beneficial correction (A) and harmful regression (I_C), from initially wrong and initially correct groups. Validity discrimination (D_V) and the same-state adoption criterion (c_W) instead use A and I_W on the same initially wrong group. These definitions are provided, but I found it difficult to keep connecting the outcome names, comparison names, and score names while reading.
-
Another specific example is "same-state." In "same-state invalid following," the answer goes from wrong to another wrong answer. But the "same-state adoption criterion" uses both correct- and wrong-advice following on initially wrong questions, so it also includes wrong-to-correct answers in A. It would help to state clearly that "same-state" in this comparison refers to the shared starting group, rather than every answer remaining wrong. A table linking the three outcomes to the two comparisons and their scores would make these connections easier to follow before the equations.
-
I also found it harder to track the setup when the results combined source and role into "source–role pairs." Source means human or AI, while role means peer or supervisor. I understand that these are different choices, but showing the combined labels alongside the original choices would make the results easier to read.
The paper does give short definitions. I would prefer one consistent name for each starting group. My concern is how the measurements are explained, rather than whether they are needed.
- The results need clearer explanations of what has been learned.
Section 4, especially Section 4.3, should highlight what each result means for the model's behavior and connect that conclusion to the scores. As written, the current results section does little more than repeat the information in the tables and figures themselves. An analysis of what broader ideas can be learned from these results would make for a much more valuable research contribution. This would help readers see what each experiment achieved and what remains uncertain.