Back to home

Compare

Comparing: AWS Uses Semantic Diffing to Review AI-Generated Policies & AWS 用语义比对检查 AI 生成策略 — 关键不只在生成

AEN
AWSOpenShellCedar·

AWS Uses Semantic Diffing to Review AI-Generated Policies

AWS has made public a method for semantically comparing formal policies: first identify the common structure between two policies, then isolate the parts that differ in actual behavior. We find that it comes closer than ordinary text diffing to the genuinely difficult part of code review—judging whether a change alters meaning.

What this is

Formal policies translate requirements such as “only the owner, with approval, may modify a resource” into rules that systems can enforce and verify. AWS’s open-source engine uses anti-unification—finding the most specific common pattern between two expressions—and an e-graph, which records equivalent expressions and the rules under which they apply, to calculate semantic differences.

A typical scenario is an LLM translating natural-language requirements into policy languages such as SMT-LIB, Cedar, Dogwood, OpenShell, or CEL. A model may produce a different result on each run, while manual line-by-line review is both slow and prone to missing critical changes such as replacing “and” with “or.” Semantic diffing can present the stable portions first, then route the differing portions to human review.

Industry view

The value here is not in automatically approving AI-generated code for enterprises, but in providing an auditable verification layer. AWS explicitly notes that the LLM generates the formal result and does not calculate the differences; this separation of generation from verification is well suited to environments with demanding permission, security, and compliance requirements.

But it cannot serve as a direct guarantee of correctness. The engine can only prove that policies are equivalent under the given rules; if the rules are incomplete or the search space is insufficient, unrecognized equivalences may remain. Differences may also arise because the rules themselves are wrong, rather than because the model made an error. Semantic diffing can therefore reduce review burden, but it cannot replace domain experts, testing, or clear allocation of security responsibility.

Impact on regular people

For enterprise IT: Enterprises can more confidently let LLMs participate in permission rules and policy configuration, but they still need to retain human approval and include rule changes in audit records.

For individual professionals: People who understand business requirements but not necessarily formal languages will increasingly be responsible for confirming whether AI-generated rules match the intended outcome; the technical barrier falls, while responsibility boundaries become clearer.

For the consumer market: This approach is unlikely to become a mass-market product in the short term, but it could reduce permission-configuration errors and compliance incidents in enterprise applications, enabling AI features to enter sensitive systems such as financial and healthcare services more cautiously.

BZH
AWSOpenShellCedar·

AWS 用语义比对检查 AI 生成策略 — 关键不只在生成

AWS 公开了一套形式化策略语义比对方法:先找出两份策略的共同结构,再保留行为上真正不一致的部分。我们注意到,它比普通文本差异检查更接近代码审查中真正困难的一环——判断变化是否改变含义。

这是什么

形式化策略是把“只有所有者且获得批准才能修改资源”之类要求,写成可由系统执行和验证的规则。AWS 的开源引擎使用反合一(寻找两个表达式最具体的共同模式),并借助电子图(记录等价表达式及其适用规则)计算语义差异。

典型场景是大模型把自然语言需求转成 SMT-LIB、Cedar、Dogwood、OpenShell 或 CEL 等策略语言。模型每次生成的结果可能不同,人工逐行检查既慢,也容易漏掉把“且”改成“或”这类关键变化。语义比对可以先呈现稳定部分,再把差异部分交给人工复核。

行业怎么看

这件事的价值不在于替企业自动批准 AI 生成代码,而在于提供一层可审计的检查机制。AWS 明确指出,大模型负责生成形式化结果,并不参与计算差异;这把生成与验证分开,适合权限、安全和合规要求较高的环境。

但它不能直接充当正确性保证。引擎只能证明策略在给定规则下等价;若规则不完整、搜索范围不足,仍可能存在未识别的等价关系。差异也可能来自规则本身有误,而不是模型写错。因此,语义比对能减少审查负担,却不能取代领域专家、测试和安全责任划分。

对普通人的影响

对企业 IT:企业可以更放心地让大模型参与权限规则和策略配置,但仍需保留人工审批,并把规则变更纳入审计记录。

对个人职场:懂业务要求、但不一定懂形式化语言的人,更可能承担“确认 AI 生成规则是否符合意图”的工作;技术门槛下降,责任边界反而更清楚。

对消费市场:这类方法短期内不会成为大众产品,却可能减少企业应用中的权限配置错误和合规事故,让 AI 功能进入金融、医疗等敏感系统时更审慎。