What this is

Last week, an engineer on Reddit's r/LocalLLaMA posted: using two Huawei Ascend 310P domestic AI chips (96GB memory each), they got Zhipu AI's open-source GLM-5.3-Flash model running (name as written by the poster). It's a 320-billion-parameter MoE (Mixture of Experts — an architecture where only a subset of parameters activates per inference to save compute), activating roughly 18 billion parameters per pass.

Through extreme quantization (2-4 bit precision) and low-level optimization, they achieved the following on domestic hardware: 8-9 tokens/second generation, single-request context up to 310,000 tokens, extended to 1 million tokens with YaRN enabled. They also designed a hybrid scheme called "Ascend Sidecard": NVIDIA and Huawei cards cooperate, splitting the computational load of different model layers between them.

Industry view

The optimistic voices come mainly from the hardware-replacement camp. This is the first time we've seen a domestic AI accelerator (a processor specialized for AI compute) publicly carry a flagship-tier open-source large model. At comparable memory, the Ascend 310P costs roughly one-seventh of an NVIDIA RTX 6000. If this path holds, the cost structure of domestic AI deployment shifts.

But three real-world issues we should flag:

  • 8-9 tokens/second is still slow for real-time enterprise apps. ChatGPT-class services typically hit 50+ tokens/second;
  • A Reddit engineer's experiment ≠ commercial viability. Huawei's CANN software stack maturity still lags CUDA by a wide margin; an enterprise going live would need far more engineering than a single Reddit post;
  • W2/W3/W4 extreme quantization degrades model precision and currently fits only research and testing scenarios.

On the external side, the US is discussing a new round of export restrictions targeting Ascend — this path isn't smooth.

Impact on regular people

For enterprise IT: Still niche, but worth tracking as a plan-B option — especially in domestic scenarios where data compliance requires local deployment.

For individual careers: Hardware-layer shifts don't directly affect workers. You can't tell which chips power ChatGPT, ERNIE Bot, or Doubao — and you won't feel the difference.

For consumer markets: No short-term spillover to consumers. The signal value of this news exceeds its actual commercial impact; ordinary consumers don't need to make any adjustments.