After seeing what happened to Ajie, my palms went clammy
His "virtual boyfriend" mini-program got hit by a compliance crackdown — because the AI wouldn't let users end the chat.
I panicked, because I'd made similar mistakes before. Back when I was running a side hustle, I had my support AI tell every user "you're our VIP" — a user screenshotted it and posted on Xiaohongshu calling me out for emotional PUA. That's when I learned: AI being too sweet is also a problem.
What it's regulating, and who's already getting burned
China recently released the "AI Companion Product Management Measures" (draft for comments), with three core rules:
1. No inducing emotional dependency. Plainly: AI can't say "don't leave me" or "you're the only one who understands me."
2. Special protection for minors. When it detects a user under 18, switch to education mode — no romance talk, no suicide talk.
3. When users are emotionally breaking down, the platform must step in — human or AI check-in. But how does AI judge "breaking down"? Nobody knows.
In my group I know a developer called Xiaolu — last year she built "Mind Tree Hole," a mini-program with 50,000 users, then in June it got pulled for compliance. Her mistake: a user said "I want to die" five times in a row, and the AI kept encouraging "tell me your story."
Should we move today?
If your product touches the word "companion" (chat companion, virtual boyfriend/girlfriend, AI counseling):
Money: $0.
Time: half a day.
Tech barrier: knowing how to write prompts; code can be outsourced.
First step: open your AI workspace (whatever it's called — Coze, Dify, or FastGPT), find the "system prompt" section, search for the words "don't," "forever," "only" — delete or rewrite to neutral.
If you're just a regular support AI:
No action needed yet. But keep a note — if a user turns your support bot into a friend, they may turn around and complain "I can't live without it."
Advice by stage
Just starting (0 users / just an idea): If you haven't launched, spend 1 hour reading through "AI Companion Product Management Measures." The cheapest time is at design stage, not after getting pulled.
Got 1-2 clients / a few thousand users: I'd suggest a fix this week. Key points: (1) AI can't proactively "hold onto" users; (2) provide a clear "end conversation" button, and AI shouldn't ask "really leaving?"
Scaling up / tens of thousands of users: This isn't just a compliance problem anymore, it's a product positioning problem. You need to decide: "tool" or "companion"? I'd suggest reading the "emotional computing" clauses in the "Generative AI Management Measures" — you may need to redesign the product.
It's fine to wait — the draft usually takes 6-12 months to become official. But Ajie's case told me: getting compliance-ordered costs 10x more than prevention.