Blog

AI Coding Tools Glossary

Developer productivity metrics

The measures used to evaluate AI coding tools: task completion time, throughput (PRs, tasks), cycle time, defect rate, and self-reported satisfaction.

No single metric works. Acceptance rate is easy but weak; completion time needs controlled trials (METR, GitHub's HTTP-server RCT: 55.8% faster; Cui et al.'s field RCTs: 26% more PRs); throughput and cycle time come from telemetry (Faros, Uplevel: no significant change); quality needs bug and security rates. Reading any AI productivity claim starts with asking which of these it measured and on whom.

Source: The Effects of Generative AI on High Skilled Work — Cui et al., SSRN 4945566

See the numbers behind this term: AI Coding Tools Statistics 2026.