HumanEval
OpenAI's 2021 benchmark of 164 hand-written Python function problems with unit tests; the first standard code-generation eval, now saturated.
HumanEval established pass@k and the function-from-docstring format. Codex scored 28.8% pass@1 at release; frontier models exceed 95%, so it no longer separates them. Its successors moved toward whole-repository tasks (SWE-bench), multi-language exercises (Aider polyglot) and terminal tasks (Terminal-bench).