Since 2025, at least three domestic and international frameworks and guidelines have listed agent risk management separately, marking a shift in AI safety governance from drafting principles to engineering practice. We have noticed that once an agent can plan autonomously, call tools, and run for extended periods, input-output review alone can no longer cover the real risks.
What this is
An agent is an AI system that can plan autonomously, invoke software tools, and continuously execute tasks. Version 3.0 of the AI Safety Governance Framework was released alongside an Agent Risk Management Framework as an annex. China Academy of Information and Communications Technology (CAICT) has published a deployment guide, while the US National Institute of Standards and Technology (NIST) has launched an AI agent standards initiative. Regulatory attention is expanding from what a model says to what a system can do, what it remembers, and whether it can be stopped promptly when something goes wrong.
These systems introduce three main categories of risk: behavior paths can change dynamically as tools return results; incorrect information may be written into long-term memory and cited repeatedly; and permissions can spread outward through databases, email, payment interfaces, and sub-agents. Governance capabilities therefore need to cover the entire lifecycle, from design and development to deployment, operation, and retirement.
Industry view
Our view is that the key change in 2025 is not the addition of a few more documents, but the convergence of governance language with product and operations language. Risks must be definable, operations must be observable, and anomalies must be intervenable. Only then can enterprises translate responsibility into procurement, deployment, and incident-handling processes.
But the risks are also more immediate. Having more standards does not make implementation easy. Different frameworks do not fully align on risk classification, permission boundaries, or accountability. Companies that simply copy checklists may end up with “complete compliance documentation but uncontrolled real-world permissions.” Another counterargument is that imposing excessive approval and logging requirements too early can significantly slow deployment and raise costs. The hard part of agent governance is not proving that a system is safe, but finding the minimum control threshold commensurate with the business risk.
Impact on regular people
For enterprise IT: Budgets may shift from standalone model evaluation toward foundational capabilities such as permission management, behavior logging, rollback, and human approval. Teams that can integrate agents with existing access-control and operations systems will be more likely to deploy at scale.
For individual work: Agents will first expand permission boundaries in document organization, email handling, and cross-system operations. Employees need to understand which instructions should not be executed automatically, but enterprises should not shift the full burden of security responsibility onto individuals.
For the consumer market: Assistants that can get things done will interact more frequently with orders, accounts, and personal preferences. Consumers will receive a more continuous automated experience, while also becoming more concerned about why data is retained, whether mistakes can be reversed, and who is responsible for erroneous actions.