This week, what's worth paying attention to isn't a product launch—it's a real budget calculation on Reddit: a local LLM enthusiast sitting on three RTX 5090s is still debating whether to add an AMD 9700 AI Pro to push VRAM to 128GB, just to fully run the new version of DeepSeek (an open-source large language model). His alternative plan: buy a 5070 Ti for gaming and keep the 5090s dedicated to AI.
Our take on this: the local LLM community is transitioning out of the "more GPUs is always better" beginner stage and into a new phase where "VRAM is the real bottleneck." Three top-tier cards not being enough means running 70B (70 billion parameter) class models on consumer hardware remains unfriendly.
What This Is
What this player faces is a concrete hardware trade-off: use an AMD card to round out 128GB of VRAM and tolerate the hassle of mixing CUDA with ROCm (AMD's GPU computing platform, analogous to NVIDIA's CUDA), or accept that "VRAM is king" and settle for partial degradation. The backdrop is the rapid expansion in parameter counts of open-source models like DeepSeek (the more learnable variables inside a model, the more capable—and VRAM-hungry—it becomes), making local inference on home hardware increasingly painful. In the post's comment section, some suggest cloud endpoints, others recommend splitting the workload across two machines, and a few bluntly tell him to "face reality."
Industry View
The pro-stacking-hardware camp argues that VRAM capacity directly determines whether model weights (the parameter files) can be loaded at all; slower speeds are tolerable, because "if it doesn't fit, it doesn't fit." The opposing view is more worth listening to: a veteran player points out that hybrid multi-GPU setups deliver poor RPC acceleration (using network protocols to chain GPUs across machines) on MoE (Mixture of Experts) architecture models—they only work well on dense models (where all parameters participate in every computation). Another commenter states flatly that "waiting for the next generation of consumer GPUs is the rational choice; spending money now is paying for anxiety." We've noticed one overlooked risk: electricity and cooling are hidden costs—three 5090s push a full system's power draw close to that of a small air conditioner.
Impact on Regular People
For enterprise IT: the rising hardware cost of local deployment is forcing companies to re-evaluate "build vs. call cloud APIs"—hard-spending on 128GB of VRAM doesn't pencil out for small teams.
For individual professionals: white-collar workers who want to run AI locally need to recognize this as a hobby costing tens of thousands with uncertain returns, not a work skill.
For the consumer market: second-hand GPU prices may continue to come under pressure. The AI compute arms race won't stop anytime soon, but the dividends flow mostly to cloud providers and chip companies.