What this is
This week, a developer on r/LocalLLaMA ran DeepSeek V4 Flash 284B — a ~284-billion-parameter Mixture of Experts (MoE) architecture large model (the model internally splits into multiple "expert" sub-networks and activates them on demand) — on a single RTX PRO 6000 (96GB VRAM). The 144GB model wouldn't fit on one card, so 21 layers of experts were placed on the GPU and 19 layers in memory, then stacked with a technique called DSpark "speculative decoding": a small model guesses several tokens first, and the main model verifies them in batch. Best coding-scenario speed hit 31 tok/s (smallest generated language units per second) — roughly 15–17% faster than without the technique.
The most interesting finding: that 10GB auxiliary small model placed in regular DDR5 memory ran 4.4% faster than in expensive VRAM. The reason is that VRAM pays off most when reserved for the "main model" — the small model is only called occasionally, while the main model is needed at every step.
Industry view
Supporters see this as a milestone for open-source LLMs going local: a single workstation GPU can now run near-frontier models, meaning enterprises building their own AI no longer need to stack GPU clusters, and cloud providers' compute pricing power is quietly being eroded.
But a word of caution: this is a single-card, single-workload (coding), single-platform test — it does not represent stable, production-grade performance. 31 tok/s is barely adequate for a single user; multi-concurrent scenarios will degrade rapidly. For local deployment to truly replace the cloud, it still needs to solve the old problems of high availability, operations, security, and cost amortization.
Impact on regular people
For enterprise IT: The hardware threshold for self-hosted LLMs is dropping, but we're not yet at "no longer need the cloud" territory. We recommend watching another 6–12 months of real production deployment cases before deciding whether to launch a PoC (proof of concept, a small-scale pilot).
For working professionals: No direct short-term impact — a single card still sits in the $4,000–5,000+ range, and both configuration and maintenance require professional expertise. The indirect effect: pricing leverage for cloud AI services will gradually expand.
For the consumer market: In the long run, falling AI service costs will eventually transmit to SaaS (software subscription service) pricing — but over the next year or two, the change felt by C-end (consumer-side) users will remain limited.