Four AMD MI100 GPUs running Qwen3.8 27B decode more than 1,000 tokens per second. This is not a new hardware launch, but the result of a community open-source project repurposing existing GPUs. We believe AI infrastructure cost reduction is shifting away from buying more expensive chips and toward keeping older devices useful for inference.
What this is
The int8-vllm project is a fork of vLLM, an inference framework for deploying large language models. It connects models to servers and handles concurrent requests. The project extends INT8 quantization support to older AMD MI-series, NVIDIA Volta, and Ampere GPUs, while fixing several numerical computation issues.
This update adds support for NVIDIA Nemotron 3.5 Bolt. It also lets models bypass the CPU and load directly from storage devices onto GPUs. According to the developers, Bolt's initial loading time has fallen from roughly seven minutes to two minutes.
Industry view
What matters is that enterprises' rationale for purchasing new GPUs will shift from absolute performance toward cost per unit, compatibility, and availability. If an open-source fork can push existing servers to high throughput, inference operations may extend hardware depreciation cycles, while cloud providers can turn idle capacity into sellable compute.
The risks are equally clear: the project is led by an individual developer, and its updates and stability lack the backing of a formal product team. Its test results also come mainly from the author's own servers; other systems and real production workloads may not reproduce them. Companies deploying community projects in production must also account for maintenance interruptions, dependency upgrades, and security fixes.
Impact on regular people
For enterprise IT: Businesses with limited budgets and existing graphics cards can evaluate open-source inference solutions before expanding their data centers. Before deployment, however, they still need to validate model quality, failure recovery, and long-term maintenance capabilities.
For individual professionals: The performance and cost barriers to local models continue to fall. As organizations use AI for retrieval, writing, and code analysis, they may be more inclined to keep sensitive data on their own infrastructure.
For the consumer market: Older graphics cards will continue to receive software optimization support. For now, however, most of the value lies in reusing enterprise compute rather than delivering an immediate consumer experience of keeping old cards instead of buying new ones.