- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This is a difficult paper to read, despite reading it multiple times to understand what it is about. With ChatGPT's help, I think the paper aims to study when an AI agent does not have access to, or is unable to perform certain actions, why some systems would fail completely while others experience relatively gradual performance degradation. The paper suggests the answer depends on whether the system's function depends on a small number of critical decisions (so-called bottleneck systems) or otherwise (substitutable systems). The paper also shows that removing random agent actions is less harmful than targeted action removal (targeted means that some identified important actions are systematically removed). On this end, the paper proposes a concept called Critical Decision Capacity (CDC), which measures how vulnerable the system is to losing available actions to guide which parts of the system should be protected and reinforced.
Critical strengths
None identified.
Critical weaknesses
The paper may contain interesting or novel contributions. However, the lack of clarity, coupled with an overwhelming number of theorems, equations, assumptions, and propositions, makes it difficult to follow the underlying research narrative. I find it hard to appreciate the significance of the proposed contributions. The introduction is too brief to motivate the research. The manuscript reads more like a rebuttal to a previous submission (App. A.13 - Consolidated answers to reviewer questions) + an extension of Kujur 2025 (heavily cited as methodological foundation) than a standalone paper. The repetition and defensive iteration of the arguments from the 1st page (CDC is not merely a name for a failure threshold) and the extensive mathematical justifications without sufficient motivation and intuitive explanation fail to convincingly communicate what the authors aim to establish.
The authors should properly introduce and define all key quantities used throughout the paper. E.g., terms and notations such as (f_c), CAC, and the various CDC measures (structural, value-based, and normalised) are introduced abruptly, even in the abstract, as though they were already familiar to the general readership. Their meanings, interpretations, and distinctions are not adequately explained in the manuscript. Simply citing Kujur 2025 and expecting readers to consult the prior work for essential definitions is not an acceptable substitute for a self-contained presentation. This approach places an unnecessary burden on readers and further undermines the accessibility and clarity of the paper.
Problem setting should also be introduced properly; the paper appears to work on a poker game environment: Kuhn and Leduc poker but the game rules are not properly introduced, Contribution 3 starts right with Leduc-N \in {5,6,7} and mentions a non-existent appear to be development file (algo_multiseed.json). The appendix has many references to development files but no supplementary files included with the submission.
Suspected sign of autoresearch. L132-134: Do not conflate them: CDCstruct and CDCnorm coincide only under Assumption 1 with uniform ρδ on the cut; otherwise report which estimator and why (Prop. 1; Section 6).
Questions for authors
The authors are strongly encouraged to rewrite the manuscript. Jumping directly into mathematical proofs without properly introducing and motivating the underlying research problem is unlikely to convince readers of the significance of the work.