返回首页

对比阅读

对比阅读:Agent 走出实验室撞上的第一道墙:不是用户,是安全 与 Agent 走出实验室撞上的第一道墙:不是用户,是安全

AEN
AgentPrompt InjectionLLM Security·

Agent 走出实验室撞上的第一道墙:不是用户,是安全

When Agents move into production, they don't collide with users—they collide with hackers. A new security guide splits risk into three layers, with the most insidious—indirect prompt injection (hiding malicious instructions in the data the Agent reads)—rated as critical severity. The core judgment: intelligence itself is the attack surface.

What This Is

Agents (AI assistants capable of autonomously executing tasks) face an old problem when entering real scenarios—security. But this time the trouble is different: traditional software has clear security boundaries, while Agents must parse natural language, invoke tools, and touch real data. "Every capability is an attack surface."

Developer communities have recently used a three-layer framework to organize the risks:

  • Input layer: Malicious instructions hide in user conversations, web pages, and documents. The most insidious is "indirect prompt injection"—the attacker doesn't talk to the Agent directly, but buries poisoned instructions in the external data the Agent will read.
  • Execution layer: Agents are induced to delete files, leak data, or invoke tools they shouldn't—once these operations execute, the damage may be irreversible.
  • System layer: System prompt leakage, API key exposure, and Agent-to-backend communication interception.

Old methods (firewalls, input validation, permission control) are far from enough. The new three principles: least privilege (grant only the minimum tool set required for the task), human-in-the-loop (high-risk operations must get human approval), and full observability (a trail for every step).

Industry View

Supporters see this as a sign of Agent maturation—the demo phase only cared about how flashy things looked; now talking about security means they've actually entered serious production environments. A good signal.

But there's calm opposition. One view: it's a bit early to talk about Agent security—most Agent projects haven't even shipped, so security frameworks built now would have no users. A sharper take: many "Agent security issues" are infrastructure problems in disguise. People who can't run a traditional WAF (Web Application Firewall) won't be able to do prompt injection detection either.

We lean toward both sides having a point. Security-first matters more in the Agent era than ever, but over-engineering is also real—mid-size companies should probably first decide whether they even need an Agent before deploying one.

Impact on Regular People

For Enterprise IT: Companies considering Agent deployment should budget for security as a separate line item—every Agent action and tool invocation should have an audit log.

For Professionals: New roles like "Agent Security Engineer" and "AI Red Team" (teams that simulate attacks to find vulnerabilities) are emerging. People who know both traditional security and LLMs will command a premium.

For the consumer market: When using Agent products, ask one more question—"How do you defend against indirect prompt injection?" Products that can't answer—use with caution.

来源: juejin.cn
BZH
AgentPrompt InjectionLLM安全·

Agent 走出实验室撞上的第一道墙:不是用户,是安全

Agent 走向生产环境后撞上的不是用户,而是黑客——一份新出的安全指南把风险拆成三层,最阴险的「间接 Prompt Injection」(把恶意指令藏进 Agent 读的资料里)被评为极高危险级别。核心判断是:智能本身就是攻击面。

这是什么

Agent(能自主执行任务的 AI 助手)走向真实场景时,撞上老问题——安全。但这次的麻烦不一样:传统软件安全边界清晰,Agent 要读懂自然语言、要调用工具、要碰真实数据,「每个能力都是攻击面」。

开发者社区最近用三层框架梳理风险:

  • 输入层:恶意指令藏在用户对话、网页、文档里。最阴险的是「间接 Prompt Injection」——攻击者不直接和 Agent 对话,把毒指令埋进 Agent 会读的外部资料里。
  • 执行层:Agent 被诱导删文件、泄露数据、调用不该用的工具——这些操作一旦执行,伤害可能不可逆。
  • 系统层:系统提示词泄露、API 密钥外泄、Agent 与后台通信被截获。

老办法(防火墙、输入校验、权限控制)远远不够。新的三条原则:最小权限(只给完成任务必需的最小工具集)、人在回路(高风险操作必须人工点头)、全程可观测(每一步留痕)。

行业怎么看

支持方认为这是 Agent 走向成熟的标志——Demo 阶段只看炫不炫,现在谈安全,恰恰说明开始进入正经生产环境,是好信号。

但也有冷静反对。一种观点:现在谈 Agent 安全有点早——多数 Agent 项目连上线都没上线,安全框架搭好也没人用。更尖锐的看法:很多「Agent 安全问题」是基础架构问题换个马甲,传统 WAF(Web 应用防火墙)做不好的人,做 Prompt 注入检测一样做不好。

我们倾向认为两端都有道理。安全前置在 Agent 时代比以往都重要,但「过度设计」也真实存在——中型企业上 Agent 前,可能更该先想清楚到底要不要上。

对普通人的影响

对企业 IT:考虑部署 Agent 的企业,应把「安全预算」单独列项——Agent 每一步操作、每一次工具调用,都该有审计日志。

对个人职场:「Agent 安全工程师」「AI 红队(专门模拟攻击找漏洞的团队)」这类新岗位正在冒头,懂传统安全又懂 LLM 的人会有溢价。

对消费市场:用 Agent 类产品时多问一句「怎么防间接 Prompt Injection」——答不上来的产品,谨慎用。

来源: juejin.cn