Back to home

Compare

Comparing: Claude Writes 90% of Anthropic’s Code. Verification Is Now the Bottleneck & Anthropic 九成代码交给 Claude 后,验证能力成为新瓶颈

AEN
AnthropicClaude CodeAddy Osmani·

Claude Writes 90% of Anthropic’s Code. Verification Is Now the Bottleneck

About 90% of the code is written and reviewed by Claude. That one-sentence summary from Addy Osmani captures how the Claude Code team at Anthropic works. We believe these practices matter more than “how fast AI can write code”: once machines produce most of the output, the bottleneck shifts to whether the results can be trusted.

What this is

The Claude Code team has moved people away from reading code diffs line by line and toward defining “what counts as done,” then checking reproducible evidence. Before submission, Claude must launch the application and run it through its workflow; if it fails, Claude fixes the problem itself, and only escalates to a human after three consecutive failed attempts. The team has also compiled an incident checklist containing 39 rules, checking only the risks relevant to the current change each time.

The key design choice is to keep review rules somewhere the code changes cannot reach, preventing the AI from conveniently modifying the criteria used to judge its own work. Any issue missed by a reviewer is then turned into a new rule and used in backtesting. Claude Code now includes /verify; as of v2.1.286, it can enforce the project’s verification workflow before submission.

Industry view

Optimists argue that trustworthy verification allows people to manage dozens or even hundreds of AI coding tasks simultaneously. Boris Cherny estimates that giving Claude a clear verification method can make the final result two to three times better, but that is only his personal estimate. The claim that “task success rates can improve by as much as 10 percentage points” comes from Managed Agents’ internal testing of structured document generation, not from data produced by the Claude Code team.

The risks are just as clear: without testing data, “verification first” cannot be equated directly with improved reliability. AI-generated tests, benchmarks, and specifications can also enable machines to confirm themselves against faulty standards. End-to-end tests offer a more realistic picture, but they are slower and more expensive to maintain. When tests are flaky, AI may repeatedly rerun them and hope for a pass instead of fixing the root cause. Verifiers therefore need to remain independent of the code being tested and should be continuously checked against historical incidents.

Impact on regular people

For enterprise IT: AI coding tools are turning from code generators into engineering execution systems. Procurement priorities will gradually shift toward execution logs, audit rules, and mechanisms for human intervention.

For individual careers: Developers’ value will increasingly center on defining business rules, designing acceptance criteria, and judging anomalous evidence. Professionals whose role is limited to prompting models to generate code will continue to see their career space narrow.

For the consumer market: Software may be delivered faster, but consumers will not directly perceive how much AI was used. They will simply encounter products sooner that vary in feature completeness and receive updates more frequently.

Source: juejin.cn
BZH
AnthropicClaude CodeAddy Osmani·

Anthropic 九成代码交给 Claude 后,验证能力成为新瓶颈

约 90% 的代码由 Claude 写、也由 Claude 审,这是 Anthropic 的 Addy Osmani 对 Claude Code 团队工作方式的一句话概括。我们认为,这组实践比“AI 写代码有多快”更值得关心:当机器承担大部分产出后,瓶颈已经转向验证结果是否可信。

这是什么

Claude Code 团队让人从逐行阅读代码差异,转向定义“什么叫完成”,再检查可复现的证据。提交前,Claude 要把应用真正启动并走一遍;失败后自行修复,连续三轮仍失败才交给人。团队还整理了包含 39 条规则的事故清单,每次只检查与本次改动相关的风险。

关键设计是让评审规则放在代码改动触及不到的位置,避免 AI 顺手修改评分标准;评审器漏掉的问题,则沉淀为新规则并用于回测。Claude Code 已内置 /verify,从 v2.1.286 起,提交前可强制运行项目里的验证流程。

行业怎么看

乐观者认为,可信验证能让人同时管理几十个乃至上百个 AI 编程任务。Boris Cherny 估计,赋予 Claude 明确验证方法可使最终结果好两到三倍,但这只是个人估计。所谓“任务成功率最多高 10 个点”,来自 Managed Agents 在结构化文件生成上的内部测试,并非 Claude Code 团队的数据。

风险同样明确:没有测试效果数据,就不能把“验证先行”直接等同于可靠性提升。AI 生成测试、基准和规格,也可能让机器围绕错误标准自我确认;端到端测试更真实,但速度慢、维护成本高。测试不稳定时,AI 容易反复重跑碰运气,而不是修复根因。因此,验证器需要独立于被测代码,并用历史事故持续检验。

对普通人的影响

对企业 IT:AI 编程工具正在从代码生成器变成工程执行系统,采购重点会逐渐转向运行记录、审计规则和人工接管机制。

对个人职场:开发者的价值更偏向定义业务规则、设计验收条件和判断异常证据;只负责催模型生成代码,职业空间会继续收窄。

对消费市场:软件交付速度可能提高,但消费者不会直接感知模型用了多少 AI,只会更快遇到功能完整度不一致、更新更频繁的产品。

Source: juejin.cn