A fact we're tracking this week: Zhipu AI's open-source model GLM-4.5-Air — an MoE (Mixture of Experts: the model splits internally into multiple smaller experts invoked per query) architecture with 106B total parameters but only 12B activated per inference — unlocked MTP (Multi-Token Prediction: lets the model output multiple tokens at once) acceleration in llama.cpp (the most widely used open-source local inference tool). The result is a visible boost in generation speed, with the community already validating it on consumer-grade cards like the RTX 3090. Our editorial judgment: the bar for running big models locally just got quietly lowered another notch.
What this is
GLM-4.5-Air is an open-source model Zhipu AI released last year, positioned as the "big model, small footprint" approach — large in total capacity, lightweight per inference. The core of this week's update is integrating MTP acceleration into llama.cpp, alongside multiple creative-writing-tuned variants uploaded to Hugging Face. Simply put: on the same GPU, it now generates tokens faster.
Industry view
The upside: Chinese open-source models continue tilting the performance-versus-deployment-cost balance toward local viability. For data-sensitive enterprises reluctant to move to the cloud (finance, healthcare, government), this is a potential option that could reshape their dependence on cloud APIs over the long term.
But a cool head is warranted: an RTX 3090 costs several thousand yuan per card, and multi-GPU setups may not pencil out cheaper than calling cloud APIs; what's driving this is the Reddit geek community, not enterprise IT departments; today's mainstream use cases are still concentrated in creative writing and roleplay, far from enterprise-grade deployment. A model that can run locally doesn't mean you should run it yourself.
Impact on regular people
For enterprise IT: Local LLMs just took another half-step from tech demo toward a genuinely evaluable option, but in the short term they don't displace cloud APIs.
For individual professionals: Unless you do content creation or AI development, this barely concerns you — cloud products remain the better choice.
For the consumer market: Consumer-grade AI assistants remain cloud-dominated; local deployment will stay the territory of geeks and privacy-hardcore users for the next 1–2 years.