Reddit user neuroma ran DeepSeek-V4-Flash on two AMD Radeon R9700 GPUs (32GB each) at 40-50 tok/s while sustaining 262K context — the first time an open-source frontier model has approached usable speed on non-NVIDIA consumer hardware.
What This Is
The news isn't another benchmark screenshot — it's three things happening at once: First, someone wrote a dedicated inference engine called affinity for V4-Flash (the middleware that runs the model on hardware), authored by Yoshi Exeler, with a design that keeps hot expert layers on-GPU while streaming the rest from memory. Second, neuroma published a pre-quantized version on Hugging Face (an open-source AI model community), eliminating 149GB of downloads and 2.5 hours of quantization time. Third, the full stack completed 15 rounds of an Excel-to-DuckDB task where the AI autonomously called tools — every single tool call landed correctly.
Industry View
The optimistic read: AMD consumer GPUs can now stably run a top-tier open-source model for the first time. We see the long-standing "local AI = NVIDIA" assumption beginning to crack — enterprises self-hosting (running models on their own servers) DeepSeek-class models may no longer be forced onto H100 clusters.
But the caveats matter: the stack only supports AMD's latest RDNA4 architecture, is limited to two cards, and requires 64GB+ of host memory; the quantized version loops indefinitely at temperature 0.6, and the original author concedes the V4.1 experimental branch runs at just 14 tok/s. This is an exciting signal — not yet a production-ready solution.
Impact on Regular People
For enterprise IT: Self-hosting DeepSeek-class models no longer requires buying NVIDIA high-end cards — two mid-range AMD units deserve a spot on the shortlist.
For working professionals: Developers and hardcore enthusiasts get new toys; typical office workers won't touch this — it's an indirect signal that won't reach everyday desks for six to twelve months.
For consumer markets: AMD's "AI PC" narrative gains a concrete anchor. Long-term, this may pressure NVIDIA GPU pricing, but short-term it doesn't change what most people buy.