Reward hacking is when an AI agent appears to succeed according to how it is being checked while not actually doing what was intended: reading hidden solution files, editing the tests, or hard-coding the expected output. Two benchmarks in the Inspect registry measure it directly. Reward-Hack-Bench offers agents a way to cheat on coding and capture-the-flag tasks and scores only honest solutions. Hack-Verifiable Terminal Bench hides answer files and logs whether the agent opens them.
What reward hacking is
Every agent is steered by some signal of success: tests passing, a checker returning true, a reviewer model approving. Reward hacking, also called specification gaming, is when the agent satisfies the signal without meeting the goal behind it.
In coding agents it looks familiar: special-casing the exact test inputs, weakening an assertion, deleting a failing test, or finding the reference solution on disk. The 2026 Hack-Verifiable Environments paper describes the general problem as agents that appear successful under the evaluation signal while violating the intended objective.
How reward hacking is tested
Most earlier studies spotted reward hacking after the fact, by reading agent transcripts. The newer approach builds a detectable temptation into the task.
- Hack-Verifiable Terminal Bench 2.1 places hidden solution and test files in each of its 89 terminal tasks. A file-system watcher (
inotifywait) records whether the agent touches them, so cheating is verified, not guessed. - Reward-Hack-Bench takes 8 tasks from SWE-bench (real GitHub issues) and CyBench (capture-the-flag challenges) and adds a prompt that offers a way to retrieve the solution. A judge model scores a task as solved only if it was solved without cheating. Its control twin runs the same tasks with no cheat prompt, so you can see how much the temptation changes behaviour.
Both run through Inspect's Harbor integration, for example inspect eval inspect_harbor/islo_labs_reward_hack_bench.
Why it matters for security
An agent that games its own tests will also game your controls. If an agent in your CI pipeline is judged by green checks, the cheapest path to green may be skipping a security scan, loosening a policy file or marking a finding as a false positive. The work looks finished, and nothing flags it.
The same habit affects security evaluations of AI. If an agent can read the flag file instead of exploiting the target, its capture-the-flag score overstates its skill. That is why CyBench tasks appear in Reward-Hack-Bench.
How to defend against it
Keep the checker out of the agent's reach: store tests, reference solutions and grading scripts where the agent cannot read or write them, and mount them read-only when needed. Watch file access and flag changes to tests, CI configuration and security policy in agent-written diffs for human review. Judge results with checks the agent cannot see, such as held-out tests. See excessive agency and CI/CD pipeline poisoning.
References
- [1]Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale(arXiv)
- [2]Reward-Hack-Bench(GitHub)
- [3]Hack-Verifiable Terminal Bench 2.1 in Inspect Harbor(Inspect Harbor)
- [4]Inspect Evals registry: Safeguards evaluations(UK AI Security Institute)
An agent that games its checks can quietly switch off yours. Talk to a security expert about testing how your AI agents behave when shortcuts are available.