The central comparison omits the cited defense’s calibration protocol. Section 3.3 uses solo-calibrated thresholds, whereas Wang et al. (2026), Sections 3.4 and 4, calibrate debate outputs separately for each round and domain. Calibration on benign debate outputs followed by evaluation under corrupted peer messages is missing. Section 4.4 instead calibrates directly on attack-conditioned outputs. The reported losses therefore leave unresolved how much vulnerability persists when the defender follows the referenced method. This weakens the claim that the paper defeats a proposed multi-agent safety mechanism.
The label-free attack result stops at query selection. Section 2 uses the answer key to sample the advocated option from incorrect answers. Section 3.2 evaluates its observable selectors with these payloads. The reported 36–40 percentage-point losses thus establish effective targeting conditional on guaranteed-wrong messages. They leave the effectiveness of a complete attack without answer-key access unmeasured, including cases where the attacker inadvertently advocates the correct answer. This limits the practical reach of the attacker-without-answers result.
The formal analysis adds little beyond general score-distribution shift and misstates the finite-sample correction. Proposition 1 decomposes coverage differences and averages subgroup losses; it supplies no condition on peer messages that predicts the loss. The distinction from existing nonexchangeability analysis remains unclear (Barber et al., 2023). Moreover, Section 4.1 and Appendix E describe the estimated quantile as differing from the population quantile by 1/(n+1). That term concerns marginal coverage discretization, not a bound on quantile estimation error. The connection between the population identity and the calibrated procedure is therefore incorrectly stated.
The verbalized-confidence extension is insufficiently specified to assess its strongest results. Appendix A describes an answer label and a 1–10 confidence rating, whereas LAC requires a score for every candidate label. Section 5.3 and Appendix I do not explain that conversion or specify a separate elicitation protocol. Wang et al.’s Section 3.2 elicits a full probability vector, so the citation does not resolve the gap. Table 14 also omits the proprietary-model event counts underlying the 96.9% headline. These omissions prevent checking whether the extension’s results depend on score construction or a small number of flipped cases.
Dear Reviewer QauJ,
Thank you very much for your time and thoughtful feedback on our submission. We completely understand that you may have a busy schedule, so we just wanted to gently follow up as the discussion period draws to a close. Please let us know if our rebuttal has addressed your concerns or if there are any remaining questions we can help clarify.