A developer ran Qwen3.8-Flash-Next on a Valve Steam Machine using an 8GB AMD GPU and 32GB of system memory, reaching roughly 17.6–17.9 tokens per second during long-form text generation. Our judgment is that the barrier to running large models locally is falling: consumer devices are beginning to offer practical value. But “running a large model on an 8GB GPU” still does not mean every computer can reproduce the same experience.
What this is
The test used the Gyro S TQ1_0 quantized model, with 128K context enabled—the maximum amount of text the model can process in one pass—while disabling vision capabilities and multi-token prediction, or MTP, an acceleration mechanism that predicts and generates multiple tokens at once. The machine had 8GB of VRAM on the AMD Navi 33 GPU and 32GB of system memory. The test used an input of approximately 8K tokens and was repeated three times.
Results showed input-processing speeds of approximately 234–242 tokens per second and generation speeds of 17.3–19.8 tokens per second. Even when input exceeded 58,000 tokens, generation held at 17.6–17.9 tokens per second without running out of memory.
The change that actually made a difference was assigning a 64GB memory budget to the inference engine so it could manage expert weights. Other GPU parameter adjustments produced almost no benefit. Enabling MTP did not improve generation speed and reduced input-processing speed by roughly 10%. This shows that hardware specifications are only the starting point: memory scheduling and software configuration determine the experience just as much.
Industry view
We believe this result sends a clear signal: local models are moving from “barely runnable” to “suitable for some everyday tasks.” For people who routinely work with documents, code, or sensitive materials, local deployment can reduce the privacy risks associated with uploading data while eliminating metered usage fees and dependence on connectivity.
But the risks are equally clear. The test covered only one hardware configuration, one quantized version, and inputs of no more than 61,000 tokens. The author did not test a busy desktop environment or more advanced acceleration options. Generation at 18 tokens per second is also clearly slower than leading cloud models. The developer also encountered problems including settings that failed to take effect and processes exiting when the SSH session ended. For ordinary users, the barriers to installation and maintenance remain high.
Impact on regular people
For enterprise IT: Businesses with older gaming GPUs or mini PCs can reassess whether local models could serve as low-cost experimental environments. For customer data, intellectual property, and high-concurrency workloads, they still need to assess the gaps in computing power, maintenance cost, and output quality.
For individual professionals: Local execution is well suited to tasks such as document summarization, code assistance, and research organization where latency is not critical. The balance among speed, model quality, and installation complexity will determine whether it becomes a practical everyday assistant.
For the consumer market: “Running large models on ordinary computers” is moving from a demonstration toward genuine usability, but it will not quickly replace cloud services. Whether device sales increase will depend on whether the software becomes stable enough and whether vendors can hide the complexities of memory and model configuration behind a simple setup.