返回首页

对比阅读

对比阅读:AI Code Is Done: Don’t Ship Yet—Cut Rework With This 与 AI 写完代码别急着交付:这套方法如何帮你少返工

AEN
形式化验证AI 编程独立开发者·

AI Code Is Done: Don’t Ship Yet—Cut Rework With This

Monday, 10 a.m.: I stared at a customer’s screenshot, palms sweating

Last month, I delivered a small outsourced website. After the customer clicked “Submit order,” the page sometimes went blank. I had tested it locally ten times, and everything worked. The customer stayed calm and asked one question: “So, can you guarantee it now?” That was when I realized how fast AI could write code—and how incapable I was of identifying the hidden problem. That was also when I started learning about formal verification. It isn’t asking AI to declare, “No problem.” You write down the rules the system must follow, then make a computer prove the code follows them.

Not another code generator, but a gate before delivery

Formal verification isn’t as mysterious as it sounds. First, you describe rules such as “an order must not be charged twice” or “the amount must never be negative.” Then a tool tries to prove, using logic and mathematics, that the program always follows them. This differs from asking AI to run tests or review the code, because those approaches can still miss cases nobody thought of. Formal verification gets closer to logically ruling out specified errors.

Stanford professor Clark Barrett, a co-founder of &Truth, helped develop the cvc5 reasoning tool. Amazon uses it for specification analysis and test generation in the Kiro development environment, with billions of calls every day. Harmonic is another company in this space, while Theorem and Axiom Math are also exploring AI-assisted formal verification.

I also got this step wrong at first: seeing the words “mathematical proof,” I assumed I had to complete an entire university logic course before I could try it. Later, I realized a small team can usually start by verifying its single most critical, highest-risk rule. There’s no need to prove the entire system on day one.

What would it cost to try today?

For an individual experiment, I calculated that the replication cost could start at $0 in cash: public materials and open-source tools are enough for an initial trial. If I wanted a commercial team plan, I’d reserve about $200 per month, though I’d need to confirm the current price with the vendor.

In terms of time, choosing one business rule, preparing the verification environment, and getting a first result would take roughly 5–10 hours. The technical barrier is manageable: I need to understand the basic business rule and be willing to turn a vague requirement into an explicit “if… then…” condition. I don’t need to become a mathematics expert.

The first step isn’t installing complicated software. It’s opening the &Truth, Harmonic, or relevant open-source project website and finding the demo, trial, or documentation. If I don’t know where to begin writing rules, I can start with one scenario—duplicate order submissions must not trigger duplicate charges—and put the normal flow, failure flow, and expected result on paper.

This tool isn’t essential for everyone. If my product is tiny and the risk is low, trying it now may not be worth it; solid manual testing and a rollback plan may matter more right now.

Match the next step to my team’s size

If I’m just starting, focused on a personal brand or one-off projects, I’d skip full verification for now. I’d turn the one rule I worry about most into an acceptance checklist, have AI generate the tests, and review them manually. It’s cheaper and more realistic.

If I already have one or two paying customers, I’d choose a rule that directly affects payment, privacy, or delivery. Then I’d try a small formal verification exercise and keep side-by-side records of what looked true before and after verification.

If I’m scaling and multiple people are now changing the code, I’d add formal verification to the release checklist. Every change would include its rule-verification result, and engineers would explain anything they couldn’t prove or had temporarily excluded.

The point isn’t to pursue “proof of everything.” It’s to let the team know which critical promises have actually been checked—and which ones we’re still merely trusting. I’m still mapping the boundaries, and I’ll keep recording the real time, costs, and mistakes.

来源: latent.space
BZH
形式化验证AI 编程独立开发者·

AI 写完代码别急着交付:这套方法如何帮你少返工

周一上午十点,我盯着客户刚发来的截图,手心全是汗

上个月,我接的一个小型网站外包已经上线。客户点“提交订单”后,页面偶尔空白,我本地测试十次都正常。对方很平静,只问了一句:“所以现在能保证吗?”那一刻我才发现,AI 写代码很快,但我根本没能力判断隐藏问题出在哪。也是从那以后,我开始了解形式化验证:不是让 AI 自己说“没问题”,而是把系统必须遵守的规则写清楚,再让计算机证明代码符合这些规则。

它不是另一个代码生成器,而是一道交付前的检查关

形式化验证的核心并不神秘:你先描述“订单不能重复扣款”“金额不能为负”这类规则,再由工具尝试用逻辑和数学证明程序一直遵守它们。它与让 AI 运行测试、复查代码不同,因为这些方法仍可能漏掉没被想到的情况;形式化验证更接近在逻辑上排除指定错误。斯坦福教授 Clark Barrett 是 &Truth 联合创始人,他参与开发的 cvc5 推理工具已被亚马逊用于 Kiro 开发环境中的规格分析和测试生成,每天被调用数十亿次。另一个相关玩家是 Harmonic;Theorem 和 Axiom Math 也在探索 AI 辅助的形式化验证。不过我也得承认,这一步我最初搞错过:看到“数学证明”四个字就以为必须先学完大学逻辑,后来才发现,小团队通常只需先拿一条最关键、最怕出事故的规则做验证,不必一上来验证整个系统。

今天想试一遍,花多少钱、时间和多少技术门槛?

以个人试水为例,我算过的复刻成本是:公开资料和开源工具可以先用,现金预算按 0 元准备;如果要购买商业产品的团队方案,先预留每月约 200 美元,具体价格要向厂商确认。时间上,选一条业务规则、准备验证环境并跑出第一轮结果,大约需要 5—10 小时。技术门槛是:要能读懂基本业务规则,并愿意把模糊需求写成“如果……那么……”的明确条件;不需要成为数学专家。第一步不是安装复杂软件,而是打开 &Truth、Harmonic 或相关开源项目官网,找 Demo、试用或文档入口。如果连写规则都不知道从哪里开始,可以先拿“重复提交订单时不能重复扣款”这种单一场景,把正常流程、异常流程和期望结果列在纸上。这工具不是所有人都需要;如果你的产品很小、风险很低,现在不试也没事,先做好人工测试和回滚方案更重要。

按你现在的人数,先做不同的事

如果你刚起步、主要做个人 IP 或一次性小项目,我会先不碰完整验证,只把最担心的一条规则写成验收清单,交给 AI 生成测试,再人工复核;成本低,也更现实。如果你已经有 1—2 个付费客户,我会挑一条会直接影响付款、隐私或交付的规则,尝试用形式化验证工具做小范围验证,并保留验证前后的对照记录。如果你正在扩规模、团队开始多人同时改代码,我会把它纳入发布检查流程,让每次改动都附上规则验证结果,同时要求工程师解释无法证明或暂时排除的部分。这里的重点不是追求“全部证明”,而是让团队知道:哪些关键承诺已经检查过,哪些仍然只是相信。我也还在摸索边界,后续会继续记录真实投入和踩坑过程。

来源: latent.space