The core of this post is a four-year hardware timeline: in 2023, gaming GPUs to run Llama 1; in 2024, a 6-card 3090 mining-rig frame for experiments; from 2025 into early 2026, progressive upgrades to 4× RTX 6000 Pro Max Q paired with 4× 3090. Total system draw sits near 2kW, roughly 300W per card. The author ran his own numbers: tracked from January 2026 onward, he generated only 30 million tokens and 2 billion prompt tokens in half a year — priced at cloud API rates, electricity plus hardware depreciation never comes close to breaking even.
What this actually is
"Local LLM deployment" means running open-source models like Llama or Qwen on your own hardware instead of calling OpenAI, DeepSeek, or Anthropic. The binding constraint is VRAM — a single 3090 gives 24 GB, a 6000 Pro Max Q gives 96 GB, and running anything above 70B essentially demands multi-card setups. This configuration can comfortably run quantized models in the ~120B range.
The motivation is stated plainly in the post: privacy, keeping data in-house, and the process itself being fun to tinker with. He calls out a specific pain point: unstable PCIe buses, a miswired PSU that fried an adapter card, and a near-miss that could have burned the house down. This is the part most media never mention — the engineering pitfalls of multi-card local inference go far deeper than "just install Ollama and run it."
Industry view
In favor: In Reddit's LocalLLaMA community, local deployment has never been the cost-effective choice — it's a DIY orientation. You pay a premium for data sovereignty and not getting locked into a vendor. For enterprises, on-prem is a hard requirement in finance, healthcare, and government — not optional.
Against and the risks: The author himself concedes that the cloud is far cheaper than his rig. This extends the industry consensus — once open-source models closed the quality gap with closed-source (DeepSeek, Qwen3), cloud token prices fell over 10× in a year, and the economics of local deployment now only justify "hobby" or "compliance" reasons. On top of that, multi-card system stability is a hidden barrier: the "GPU dropping off the bus" he describes is unacceptable in production. Enterprises that genuinely need on-prem typically buy turnkey white-box-supported systems from Dell or HPE.
Impact on regular people
For enterprise IT: On-prem LLMs are no longer a geek toy. Finance, government, and hospitals are already requiring data to stay inside the network. If your company has these compliance constraints, this post serves as a budget reference — 4× 6000 Pro plus peripherals runs roughly 150,000–200,000 RMB.
For individual professionals: If you only use AI for emails and summaries, just use ChatGPT, Claude, or Tongyi App — do not build your own rig. Only consider this path when the data you handle cannot leave any server.
For the consumer market: The RTX 6000 Pro Max Q 96 GB retails around 80,000 RMB per card, and used mining-card prices swing wildly — the author explicitly says "I wouldn't buy at today's prices." Consumer cards (4060/4090) are enough for sub-30B models. Don't let posts like this push you into hoarding GPUs.