Our editors spotted an experiment report on r/LocalLLaMA this week. An independent engineer used an ordinary gaming PC — RTX 4070 Ti GPU + 32GB RAM + two NVMe SSDs — to shoulder Moonshot AI's Kimi K3.
Kimi K3 has 2.8 trillion parameters and a 711GB model file — currently the largest publicly disclosed flagship MoE architecture (MoE = Mixture of Experts, a design where only a small subset of "expert" sub-networks activates per inference). But the author is candid: "It runs" doesn't mean "It's usable."
What This Is
This is a custom inference engine called CRANE V2, purpose-built to solve the "model is far bigger than VRAM" problem. The core idea isn't cramming the model into VRAM — it's keeping it on disk and loading on demand.
Key engineering moves: treat two SSDs as a single logical storage pool with parallel reads; load precise expert slices on demand; buffer through host RAM to overlap storage and GPU transfers; benchmark different cache replacement policies.
Results across three models: Kimi K3 standard speed 0.014 tok/s (0.014 tokens per second); DeepSeek V4 Flash Q3 (Q3 quantization: parameter precision compressed to 3 bits to shrink size) jumped from 1.14 to 8.08 tok/s; Qwen3.5-122B standard 1.89 tok/s.
Industry View
Supporters see hope: large-model decentralization is becoming technically feasible. For data-sensitive industries — healthcare, law, finance — running models locally, never leaving the corporate network, is a real demand.
But we're pouring cold water. Those prettier numbers — Kimi at 9.3 tok/s, Qwen at 57 tok/s — come at the cost of quality: altered routing strategies, lowered precision, skipped shared-expert layers. The output devolves into multilingual gibberish that can't even answer "Paris is the capital of France." The author kept "speed-wall experiments" and "quality-preserving experiments" in separate ledgers, deliberately — so the numbers couldn't be misread.
Another overlooked risk: 711GB of storage remains astronomical for most users. The whole rig runs over RMB 10,000, and what comes out is still "one or two tokens per second." This isn't a consumer product — it's a geek toy.
Impact on Regular People
For enterprise IT: The inflection point for on-premises AI deployment is approaching, but three gaps still separate "it can run" from "it's commercially viable": speed, stability, and ops cost. Unless strict compliance demands it, most enterprises should still default to cloud services.
For individual professionals: Over the next 2–3 years, your work laptop may be able to run "small versions" of large models under 10B parameters. Full versions will still require cloud APIs. This has practical value for lawyers, consultants, and journalists who travel often or work offline.
For the consumer market: DIY enthusiasts will drive the first wave, much like the self-built NAS era of the past. Mainstream consumers will likely have to wait until 2027–2028 for a "plug-and-play" local AI device — and only if there's a fundamental breakthrough in model architecture.