What This Is

Two of Alibaba's internal AI systems are now running in production on RocketMQ (a message middleware): the coding agent Qoder handles single tasks spanning multiple days, while an inference gateway schedules millions of tenants. This forced a redesign of the middleware for "multi-day AI tasks" — and LiteTopic was born.Traditional message queues rest on three assumptions — millisecond-level processing, pre-defined channels, and interchangeable messages within a queue — all of which break down in AI Agent (AI programs that autonomously execute multi-step tasks) scenarios: tasks run from minutes to hours, long sessions span days, and each user needs an isolated channel. The old architecture produces three concrete problems: backlog spikes, head-of-line blocking where one stuck user holds up the entire queue, and wasted GPU cycles when interrupted tasks restart from scratch.LiteTopic's approach: refine channel granularity to lightweight channels dynamically generated per session at runtime. A two-layer structure: a Parent Topic governs permissions and quotas, while LiteTopic handles session-level isolation (ordered, exclusive, replayable). At the storage layer, a RocksDB key-value store replaces file-based indexing to consolidate millions of small files. At the delivery layer, event-driven dispatch replaces long polling to cut idle compute.

Industry View

Supporters see this as a sign of AI infrastructure maturing. Over the past year, Agent projects have repeatedly stalled at deployment — one root cause is that underlying messaging systems don't support long sessions, isolated execution, or failure recovery. These problems are now being absorbed into the middleware layer. Alibaba is feeding production-validated capabilities back to the open-source community — a direct win for mid-sized vendors building their own multi-Agent systems.Skeptical engineers raise two points. First, LiteTopic pushes session state down into the messaging system, binding business semantics more tightly to infrastructure. If session models evolve — say, to multi-Agent collaborative calls — protocol-layer change costs could exceed the gains. Second, the design assumes "session" is the right isolation granularity, but in practice cross-session context sharing is common (think: a user's history across multiple tasks). In those cases, "one session, one channel" becomes a new form of coupling. And overseas alternatives like AWS SQS and Kafka are evolving in similar directions — this is not an Alibaba-exclusive path.

Impact on Regular People

For enterprise IT and technical decision-makers: when evaluating self-built AI Agent systems, "whether the message middleware supports session-level isolation" should be on the procurement checklist. MQ products purchased for traditional workloads may need to be upgraded.For professionals: as Agent task durations stretch to hours or even days, "waiting for results" becomes the norm — work rhythm may shift from "kick off, wait a few minutes, done" to "kick off, do something else, check back tomorrow."For consumer markets: the AI products you're already using (customer service bots, coding assistants, AI search) almost certainly run on this new type of infrastructure. The differences in service stability and conversational experience depend heavily on this underlying capability — not on how smart the model itself is.