- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces InterviewLite, a synthetic structured-interview environment for auditing hybrid RL controllers under a proposal-execution distinction. It evaluates TD3-style and PPO controllers with optional memory, reliability conditioning, nuisance consistency, and proposal-cost optimization, and argues that executed safety alone can hide either shield-corrected unsafe proposals or degenerate no-follow-up behavior. The empirical study reports 60 trained policies and compares proposal safety, executed safety, intervention, return, overlap, and useful clarification outcomes. The paper’s main conclusion is deliberately scoped: in this synthetic setting, joint reporting of proposals, executions, and utility exposes failures that would be hidden by executed safety alone.
Critical strengths
- The proposal-versus-execution accounting is a useful evaluation contribution. Equation 1 defines cost on unsafe proposals, including proposals blocked by the shield, while Section 5.1 clearly explains why executed unsafe rate alone cannot identify whether a policy is intrinsically safe or merely corrected by intervention.
- The hybrid-action formulation is more explicit than is common in small applied RL studies. In particular, Equations 3 to 6 specify how the continuous action and seven-way categorical action enter the TD3-style critic and actor update. Enumerating exact one-hot categories in Equation 6 is preferable to passing a fractional categorical representation into a critic that is only queried at discrete actions. The authors also appropriately acknowledge the residual surrogate-to-mode mismatch.
- The experimental accounting is careful. Section 6 explains that repeated evaluation seeds do not create additional independent subjects, aggregates at the subject level, preserves within-subject pairing across methods and seeds, and avoids treating dependent turns as independent samples. Equation 10 makes the paired estimand explicit. The paper also correctly separates pooled event rates from subject-averaged rates and explains why their denominators differ when policies induce different episode lengths.
- The main tables communicate the central diagnostic well.
Critical weaknesses
- The scientific contribution is constrained by a highly transparent, largely hand-specified fixture, and the paper does not demonstrate that its audit protocol changes conclusions outside this particular construction.
- The main paper does not provide enough empirical evidence for the broad “60 fitted policies” study it describes.
- The claimed exploration-pressure diagnosis is not fully specified or quantitatively established in the main paper.
- The TD3 and PPO optimization descriptions remain underspecified in ways that matter for the comparison.
Questions for authors
- For the TD3-style controller, what exact behavior-policy exploration is used during data collection for the continuous and categorical action components? Please distinguish this from the target smoothing noise in Equation 3. Is the stated 20% exploratory rate applied to all TD3 training runs, only to the always-proceed control, or both?
- Can the authors provide a compact summary of all six component settings across the nominal, dropout, and slow conditions? At minimum, this should include return, useful probes, proposal unsafe rate, executed unsafe rate, and total probes. The current Tables 1 to 3 only establish the full-versus-bare contrast.
- Would the authors consider evaluating the audit protocol on at least one environment beyond InterviewLite, with a separately declared proposal-level safety predicate?