A public benchmark has measured a 1.9x gap: the amount of code generated by the median model on SWE-bench Verified is about 1.9 times that of human fixes; on the newer DeepSWE benchmark, however, it is close to 1x. The next test for AI coding is therefore not only whether it can solve a problem, but also how much maintenance burden it leaves behind.
What this is
Parsimony is an open-source benchmark for comparing the complexity of code generated by AI models and their execution tools—the harness, or workflow, that enables a model to invoke terminals, edit files, and run tests. Rather than counting lines of code, it measures syntactic units such as statements, function calls, and naming, while excluding comments and tests. Every AI-generated patch is compared with the human fix for the same issue.
The preliminary data supports two conclusions. First, greater model capability does not necessarily produce more concise code: on SWE-bench Verified, smarter models generated slightly more code. Second, the execution tool may have a larger impact than the model itself. With the same model but different tools, code volume ranged from 0.80x under Codex to 0.99x under mini-SWE-agent and OpenClaw.
Industry view
AI coding evaluations have long focused on problem-solving rates and inference costs, rarely penalizing overengineering or redundant code. Code that runs is not necessarily suitable for long-term maintenance. These extra structures raise the costs of understanding, modifying, and handing over code, eventually creating technical debt.
That said, we do not interpret 1.9x as direct proof that newer models are worse. Different benchmarks, model versions, and task sets are not fully comparable; public patches are also incomplete, and some results require independent validation. The more defensible conclusion is that a model-tool combination must be evaluated as a whole. Model leaderboards alone are insufficient, and a single data point should not be treated as an industry verdict.
Impact on regular people
For enterprise IT: When procuring AI coding tools, organizations should validate fix success rates, code volume, maintainability, and test coverage together. Otherwise, cheaply generated code may simply defer costs to later maintenance.
For individual professionals: Developers will need code review and architectural judgment even more. AI has increased the speed of output, but it has also accelerated the spread of flawed implementations. The ability to generate code is becoming less scarce.
For the consumer market: We see no immediate reduction in software functionality, but simpler code can lower iteration costs over time, help small teams ship products faster, and potentially give consumers more stable applications with less maintenance debt.