- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes VStress-CA, a stress-testing and certification framework that evaluates binary verifier aggregation under correlated corruption with reliable risk guarantees.
Critical strengths
I don't find any strength.
Critical weaknesses
The novelty is almost entirely preempted. The paper makes numerous overclaims. Its core "discovery"—that correlated errors invalidate majority voting—was already independently reported in at least three other papers by 2026.
CGES (Confidence-Guided Early Stopping for Efficient and Accurate Self-Consistency) covers the stopping/abstention rules proposed in VStress-CA.
"Adaptive-K via design effect" corresponds to the sequential sampling approach touted as a key contribution of VStress-CA.
"Convergent Adaptive Inference" addresses the thresholding mechanism used in VStress-CA.
Kohli, Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels covers the very conclusion reached by VStress-CA.
Blind to the Pivotal Vote
Evaluator Ensembles Under Reward Hacking: Covariance Geometry
FUSE: Ensembling Verifiers with Zero Labeled Data
SEVRA: Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning
Adaptive Test-Time Compute Allocation via Learned Heuristics
Denisov-Blanch et al., Truthfulness Does Not Scale Like Reasoning: Why Polling Fails as a Proxy Verifier
When Self-Consistency Backfires
Consensus is Not Verification
these paper should be discussed.
The bibliography contains many references completely unrelated to the topic—likely hallucinated by generative AI—along with numerous citation errors. Meanwhile, the paper fails to discuss or compare itself against a wealth of genuinely relevant literature. Furthermore, the formatting does not adhere to ICLR requirements.
The experimental evidence fails to support the paper's claims (the experiments are of very poor quality). This submission warrants a desk rejection.
Questions for authors
see above