A user with two 16GB RTX 5060 Ti cards is running a quantized Qwen model at roughly 70 tokens per second. He is considering buying two more to reach 150 tokens per second, at a cost comparable to a pair of RTX 3090s.

What this is

This is a hardware-upgrade discussion about local large language model inference, or running models on your own PC or server. The user mentioned Qwen3.8 Flash and IQ3_S, referring to the model and its low-bit quantization format. Quantization compresses model parameters to reduce VRAM usage, but usually comes with some loss in output quality.

The key point is that adding more GPUs does not automatically double the speed of each response. Generation speed also depends on whether the model fits entirely in VRAM, data transfer between the CPU and GPUs, memory bandwidth, and batching. If the model still has to move frequently between system memory and the GPUs, additional cards may primarily increase capacity rather than deliver linear speed gains.

Industry view

We believe this case shows that the bottleneck in local AI is shifting from whether you have GPUs to whether the hardware matches the workload. For users who continuously process long contexts, batch jobs, or shared services, adding VRAM and parallel compute capacity may offer real value. For occasional question-answering, however, optimizing software configuration, model quantization, and prompts will usually be more cost-effective than immediately expanding the hardware.

The risks are equally clear. Consumer GPUs generally offer weaker multi-GPU interconnect, cooling, power delivery, and long-term stability than data-center platforms. A four-card setup also increases the costs of PCIe lanes, motherboard expansion, and cooling modifications. Although two RTX 3090s provide more total VRAM, that does not necessarily mean faster inference. The decisive variables remain the actual model, context length, and test scenario—not the number of GPUs.

Impact on regular people

For enterprise IT: If a company plans to run local models across multiple PCs, it should first calculate VRAM requirements, concurrency, and maintenance costs rather than treating consumer GPUs bolted together as a “server.”

For individual professionals: Local models are well suited to handling material that should not be uploaded to the cloud. Before upgrading, users should benchmark their existing devices to identify the real bottleneck and avoid pouring money into idle hardware merely to chase a higher tokens-per-second figure.

For the consumer market: The AI compute advertised by GPU vendors does not equal the speed users will actually get. VRAM capacity, interconnect capability, software support, and sustained power delivery affect purchase value more than any single specification.