We spotted a help thread on r/LocalLLaMA this week: a hardware enthusiast tried to run a quantized Deepseek model locally (~120GB; quantization is the technique of compressing a model to fit in smaller VRAM), with a 48GB Blackwell 5000 and a 24GB 3090 on hand. Running it solo hit 16 tokens/sec (how many characters the model generates per second), but adding the 3090 dropped it to 11–12 tokens/sec—after three days of struggle.
What This Is
Deepseek is an open-source MoE-architecture model (Mixture of Experts: the model contains thousands of "expert" modules, only a small subset activates per inference, theoretically more resource-efficient). In theory MoE should split automatically across multiple GPUs, but in practice it doesn't—you have to manually specify which expert layers go on which card; misallocation makes it slower than a single GPU.
This isn't an isolated case. Over the past six months, similar help threads on r/LocalLLaMA have been piling up, showing that "running frontier LLMs locally" is still hobbyist territory.
Industry View
The "local AI is the future" camp argues: cloud APIs charge per call and require data to go to the cloud, which doesn't pencil out for SMBs or data-sensitive scenarios; advances in open-source models (Deepseek, Qwen, Llama) and hardware upgrades (Blackwell, consumer AI PCs) will keep pushing down local deployment costs.
But the warning side deserves attention: this user is at hardware-enthusiast level, and three days on Reddit didn't resolve it—proof that current local-deployment barriers run higher than the marketing suggests. If you're an IT lead at an ordinary company, expecting "install software on employee machines and run Deepseek locally" is unrealistic today. Another rarely-discussed fact: there's a vast gap between open-source models' "runnable" and "usable"—16 tokens/sec is passable for chat but far short for productivity tooling.
Impact on Regular People
For enterprise IT: local LLM deployment won't be a mainstream option in the near term; cloud APIs remain the more economical choice unless you have hard data-compliance or scale requirements.
For individual professionals: if you hear pitches like "open-source Deepseek, anyone can run it locally," remember "runnable" and "usable" are two different things—confirm your workflow genuinely needs on-prem before committing.
For the consumer market: "AI PCs" and "AI phones" with "built-in LLMs" are essentially doing the same thing—"squeezing LLMs onto devices"; but vendor scenarios are marketing concepts while enthusiast scenarios are productivity tooling—the implementation difficulty isn't even in the same league.