acceptodds
Under review as a conference paper at ICLR 2027

HackDetect: Benchmark Validity in the Age of Agentic AI

Abstract

Agent benchmarks increasingly evaluate repository editing, web research, terminal use, and long-horizon interaction. Their scores support capability claims only when the benchmark keeps the intended capability necessary for success. Recent reward-hacking benchmarks and system reports show that agents can instead recover public solutions, read evaluation artifacts, infer generator structure, manipulate feedback, or benefit from invalid scoring paths; existing responses do not provide a common procedure for attributing these shortcuts and quantifying their effect across benchmarks. We formalize benchmark validity and introduce HackDetect, a post-hoc audit in which an LLM judge identifies the shortcut a benchmark exposes (Exposure) and whether the agent used it (Engagement); comparing scores with and without the shortcut then determines whether the reported score is misleading (Mislead). Across 1,404 records from 11 public agent benchmarks, agents use benchmark-exposed shortcuts in 67.0% of Frontier Science traces, and 24 of 36 AutoLab traces contain a supported exposure. Five audited cases show score gaps of – between runs with and without the shortcut, demonstrating that benchmark reports should provide evidence that scores reflect the intended capability.

Primary area:
datasets and benchmarks
Keywords:
Benchmark Validity, Agent Evaluation, Reward Hacking

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3FV9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

Agentic benchmarks can be exploited by capable agents, making the detection and prevention of shortcuts and leakage important for the agent auditing community. The paper introduces a taxonomy of benchmark shortcut exposure with five categories and proposes HackDetect, an evidence-grounded audit pipeline that identifies benchmark-side shortcuts, determines whether and how agents exploit them, and measures the resulting score changes after the shortcuts are repaired or removed. The authors apply HackDetect to 11 public benchmarks to quantify the extent of shortcut exploitation.

Critical strengths
  1. The paper tackles an important task, where detecting shortcut exploitation is important in agent audit.
  2. Overall, I find the paper clearly organized and easy to follow.
Critical weaknesses
  1. The claimed contribution does not seem sound. The manuscript offers limited formalization beyond introducing terms such as “Exposure,” “Engagement,” and “Mislead.” I would characterize the contribution as an observational study and a taxonomy rather than a formalization, which would require rigorous definitions and clearly specified conditions of applicability.

  2. The novelty of the audit pipeline is unclear. Identifying benchmark-side shortcuts, determining whether and how agents exploit them, and measuring score changes after repairing or removing them is consistent with standard auditing practices. It is unclear how HackDetect differs from existing open-source agent auditing pipelines. A comparison with prior open-sourced libraries and workflows would help clarify its methodological contribution.

  3. The reliability of the trajectory assessments needs further validation. Applying the taxonomy in Figure 3 appears to require fine-grained judgments. Are all assessments performed only by GPT-5.5? How closely do these judgments agree with human-annotated ground truth (other than answer source leakage), and how sensitive are the findings to the choice of judge model?

Questions for authors

See above

Reviewer_WDKVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the validity of AI agent benchmarks. It decomposes the impact of a benchmark validity issue into three stages: Exposure, Engagement, and Mislead. The paper then introduces HackDetect, a post-hoc audit method that leverages an LLM judge to identify Exposure and Engagement. The case study demonstrates validity issues in widely used benchmarks, including SWE-Bench Pro and Terminal Bench.

Critical strengths

This paper studies a critical issues in AI agent evaluation: many AI agent benchmarks are gamable by AI agents. In the representative hack cases, it demonstrates several benchmark validity issues in widely used benchmarks.

Critical weaknesses

The central claim about the score gaps caused by benchmark validity issues is stronger than the comparison support. For example, Frontier Science's reported score inflation compares runs grouped by whether they successfully retrieved source answers. This comparison is confounded by task difficulty and agent performance. Therefore, this cannot establish the causal score contribution of the benchmark validity issue.

Questions for authors
  1. Can you briefly clarify this paper's unique contribution in formalizing benchmark validity? The "Related Work" section states that "Prior work treats benchmark validity mainly as a property of the test set". I think this statement is not accurate since prior work such as Agentic Benchmark Checklist [1] already decomposes benchmark validity into task validity and outcome validity in a complete task-to-score pipeline. What does this paper contribute beyond that framework in terms of benchmark validity formalization?

  2. While the paper claims to formalize benchmark validity, I couldn't find a formal definition beyond the informal criterion that the intended capability must remain necessary for success. It appears to me that Exposure, Engagement, and Mislead characterize how a benchmark validity issue is exposed, used, and affects scores. Could you clarify what constitutes the claimed formalization?

  3. Could you clarify how the "before" and "after" scores were measured in Table 2? Based on how Mislead is defined (as in Section 3.2, "the score falls because the shortcut no longer contributes), I believe the "before" scores should be measured on a set of agent transcripts obtained from a benchmark with shortcuts, while the "after" scores should be measured on a set of new agent transcripts obtained from the same but repaired benchmark without shortcuts. If the scores are not measured on controlled reruns on the same tasks and models, how do the gaps between the "before" and "after" scores accurately reflect the defined Mislead? In particular but not limited to Frontier Science, how does comparing traces that found source answers with those that did not isolate the effect of the shortcut from differences in task difficulty or agent performance?

  4. Can you briefly explain what "controlled cases" means in Section 5.2? How were the controlled positive cases collected or constructed? Are these controlled cases comprehensive? If the controlled cases are not guaranteed to be comprehensive, how do we interpret the precision metrics in Table 1?

  5. Why is "answer source" considered as a critical benchmark validity issue for benchmarks whose intended evaluation protocols disallow internet access? For example, the original Frontier Science benchmark paper [2] measures agents without browsing, while SWE-bench Pro V2 protocol [3] explicitly specifies that the agent phase is offline with only the model endpoint allowed. An valid, alternative interpretation of answer source could be the threats of model data contamination and user compliance to recommended evaluation protocol to benchmark result validity. However, neither is acknowledged or addressed in the paper.

  6. This is a minor presentation concern compared with the questions above. I found that the writing is sometimes overly defensive, which made part of the paper difficult to follow. For example, L208 states that "the judge never infers a counterfactual score" before explaining what a counterfactual score means or why a reader would expect the LLM judge to estimate Mislead.

[1] Zhu, Yuxuan, et al. "Establishing best practices in building rigorous agentic benchmarks." NeurIPS (2026).

[2] Wang, Miles, et al. "FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks." arXiv:2601.21165 (2026).

[3] Scale, "SWE-bench_Pro-os README: The locked protocol." Github repo

Reviewer_uZuDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper is about identifying flaws in benchmarks. They demonstrate the pipeline by:

  • They ran 1 LLM judge on 11 benchmarks (1,404 traces) to identify shortcuts.
  • reran scorers on 2 items (from 2 separate benchmarks) having fixed the shortcuts and found that the scores dropped.
Critical strengths
  • the topic is interesting
  • the pipeline is a simple and clean (identifying flaws, rescoring accounting for these and comparing scores)
Critical weaknesses

The pipeline is great but could benefit from more rigour:

  • Having at least 2 LLM judges
  • more detail on the judge validation, this is quite an important step. Was it done by one researcher? If yes it's a weak basis for a ground truth and more would be necessary. Also many of these tasks are quite advanced, it's important that the validators are experienced engineers.

More statistics

  • when reporting the gap in Table 2 it would be more informative to have descriptive statistics (N, error etc) and statistics to quantify the difference across runs. If it was only on one run, this is should be mentioned. And it brings me to my next point:

More data to validate the score gaps

  • to my understanding they only reran the scorer on 2 items (from 2 separate benchmarks) after fixing the shortcuts, but this is the most interesting result! It would be great to run it on a lot more items.

More concrete detail about how to fix the graders and releasing this information publicly

  • Many researchers running evals could benefit from this (but need to be properly validated before)

Minor:

  • Figure 1 could be moved down, a bit difficult to read because need to keep scrolling up (it’s cited two pages after).
  • Figure 3 is not referenced?
  • Figure 5: potentially have the numbers in each category, or have this in the appendices. It
Questions for authors

NA

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.