A Reddit developer has published a local AI rig built with eight RTX 3090s: it generates at roughly 60 tokens per second for a single agent and around 370 tokens per second in aggregate with 10 agents running in parallel. Our view is that model inference is descending from an expensive data-center capability into local hardware that people with substantial budgets can access—but being able to build it does not mean it is cost-effective to use.
What this is
The system uses eight RTX 3090s with 8GB of VRAM each, dual power supplies, water cooling, and custom PCIe riser cables. A modified NVIDIA driver enables peer-to-peer communication between the GPUs. According to the author, measured communication bandwidth reached approximately 22GB/s. With models such as Qwen Flash, the rig can run multi-agent tasks locally and has room for fine-tuning smaller models. An agent, in simple terms, is an AI program that can call tools autonomously and complete tasks step by step.
Industry view
The notable shift is that developers are assembling consumer GPUs to deliver capabilities that previously required professional hardware. VRAM capacity, model-loading efficiency, and parallel throughput remain the key factors determining whether this approach can handle real workloads; the number of GPUs alone cannot compensate for poor software optimization.
We also need to consider the objections and risks: the system draws about 300W while idle, while electricity and cooling costs rise further under full load. Water cooling, extension cables, dual power supplies, and driver modifications also make troubleshooting more difficult. Even when estimated at the author's earlier purchase prices, eight used 3090s cost approximately £4,800 to £5,600. For most organizations, renting cloud capacity on demand is generally more flexible. This is better understood as an experiment by a technically capable team with a generous performance budget than as a broadly adopted deployment model.
Impact on regular people
For enterprise IT: Privacy-sensitive document processing, code analysis, and small-model fine-tuning now have another local deployment option. Before procurement, teams should compare cloud service costs with hardware depreciation and ongoing operations and maintenance expenses.
For individual professionals: People familiar with Python, Linux, and model deployment can use a system like this to build a private work assistant. Users without that technical background should still prioritize mature cloud or desktop products.
For the consumer market: Stacking GPUs can provide more powerful local compute, but high power consumption and cost also mean manufacturers will not turn this into an ordinary home computer in the short term. AI acceleration will remain centered on cloud service subscriptions.