What This Is
A set of benchmarks posted to r/LocalLLaMA shows Zhipu's GLM-5.3-Flash hitting 73.5 tokens/sec on the new TensorSharp framework—exactly double llama.cpp's speed. This deserves our attention because it could redraw the cost curve for local LLM deployment.
GLM-5.3-Flash uses UD-Q2_K_XL quantization (compressed to 2-bit precision, runnable on ordinary GPUs). The key to TensorSharp's 2x decoding speed is "graph caching": the compute graph for each token is built once and replayed repeatedly, while llama.cpp rebuilds it for every token. Prefill speeds are nearly tied, with the gap within 3%.
Industry View
Two camps emerged on Reddit.
Supporters call this a genuine engineering breakthrough. GLM-5.3-Flash's structure—deep and fragmented compute graphs per token—is precisely the scenario where graph caching pays off the most. If domestic model local deployment costs really halve, SMBs running private AI are the direct beneficiaries.
But the objections are clear: first, the benchmarks were run by the submitter themselves (/u/fuzhongkai), with no third-party reproduction yet; second, the tests used CUDA 12.8 + SM 120, this generation's new GPUs—performance on older cards is unknown; third, TensorSharp is a brand-new project, its toolchain and ecosystem nearly blank, still a long way from production.
Impact on Regular People
For enterprise IT: if the results hold up, self-built LLM inference clusters could see hardware spend drop by 30–50%—the same cards serving twice the concurrency. But TensorSharp is closer to a tech demo today than to an enterprise-grade product.
For working professionals: minimal direct impact. When you use Doubao, Kimi, or Ernie, you can't feel which engine is running on the backend. But if enterprise compute costs really do drop, AI feature pricing has room to fall further.
For consumer markets: local AI devices (AI toys, edge boxes, in-vehicle AI) could theoretically respond faster, but TensorSharp needs at least another year of ecosystem work before reaching consumer-grade hardware.