- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies gradient interference under different losses in reinforcement learning.
Critical strengths
None
Critical weaknesses
The manuscript as presented does not satisfy basic scientific standards of communication.
- It is incredibly dense writing.
- The text does not clearly establish what it attempts to do or why.
- The text uses various terms without defining them. This starts in the title. I don't know what "the limits of kappa" means. This lack of definition includes terms such as retention which seems to be the main object of study.
- Mathematical notation is not properly introduced.
- Uncontextualized numbers are scattered everywhere (starting in the abstract). These may be referring to experiments that are not described or referenced.
- The text references tools/settings that are never described such as a "mirror bandit".
- The text states something about withdrawing claims but those claims never appear in the text (something about abelian groups).
- The experiments are missing proper descriptions as to what the experiments are trying to do and how. The experiments seem largely to provide a collection of measurements of gradient norms.
- As far as I understand, the math seems to be elementary vector cancellations, obfuscated simply by complex writing.
My review is not particularly detailed as the text is not readable. I believe extensive revisions are needed in terms of writing before it can be properly reviewed
Questions for authors
None
2 comments
Response to Reviewer pURj: a reading guide and a simplified revision
A short update to my earlier comment: that comment was written before the revision and described the plan; the revised PDF is now uploaded.
The revised paper:
- replaces the vague "Limits of Kappa" in the title with "When Capacity Pays";
- simplifies the abstract and adds a reader's map and a notation paragraph;
- states the decision rule in one place: c/n_mean > 2, effect above the calibrated gate, and a task-relevant condition; retention is a diagnostic, not a gate;
- adds §4.5 validation-selected capacity, §4.6 DICES, and §4.7 switching. The earlier references to "limits of kappa" and "abelian groups" are no longer in the paper; direction dependence is stated as K-way simplex geometry.
We have not added a full glossary in this revision; it is planned for the next version. The earlier comment remains for transparency; this note supersedes its section map and revision plan. No score request.
Response to Reviewer pURj: a reading guide and concrete revision plan
We thank Reviewer pURj for the frank assessment. We agree that the paper is too dense, that several terms are used before they are defined, and that the abstract contains numbers without enough context. We will not argue with that; the revision will fix it. To make the current submission reviewable now, here is a short guide.
What the paper is trying to do, in five sentences.
Definitions that were missing or too late.
Section map for the current PDF. §2 definitions and measured fields; §3 theory (Props. 1-4); §4.1 exact bandit; §4.2 protocol audit; §4.3 Overcooked/510K; §4.4 capacity; §4.5 DICES; §4.6 switching; §6 what the audit does and does not claim; §7 limitations.
On the math. The mirror-bandit algebra is intentionally elementary. The contribution is not a difficult theorem; it is the measurement argument: what a retention score can and cannot identify, and which protocol controls are needed to measure it. The complexity is in the interpretation, not the algebra. We will say this explicitly in the introduction.
Concrete revision plan.
We will upload the revised PDF before the end of the public discussion period. We are not asking for a score change. We would be grateful for a pointer to the first section where the reader loses the thread; that will help us target the rewrite.