This week, an anomaly surfaced in the llama.cpp GitHub issue tracker that kept us staring at our screens: the same quantized Qwen3.5-9B model (quantization is a compression technique that shrinks models for local deployment) runs 32% slower than unaccelerated baseline on an NVIDIA RTX PRO 4000, but nearly twice as fast on an AMD Radeon card. The reporter ran the test over a hundred times, and the results were stable.
What this is
MTP stands for Multi-Token Prediction — the model drafts several candidate output tokens at once, then uses the accepted ones as a speedup signal. It is normally an inference acceleration technique: the more drafts the model accepts, the faster it goes. The strangeness here is that with the identical model file and identical software version, the acceptance rate (the share of drafts the model endorses) diverges wildly across GPUs — 35.8% on NVIDIA, but 91%-92% on AMD. In other words, the technique itself is fine; the underlying compute paths from different GPU vendors (NVIDIA via CUDA, AMD via Vulkan) handle these drafts in opposite directions.
Industry view
The open-source community is split into two camps. One camp argues this is proof that the local-LLM ecosystem is "de-NVIDIA-izing": AMD is becoming increasingly attractive to SMBs on power efficiency and price-performance, and running domestic models like Qwen can actually be an advantage. The counterargument is equally persuasive — the report tested only one NVIDIA card, with no controlled comparison of CUDA versus Vulkan on the same hardware, and the raw data hasn't been released. A more pointed critique comes from a separate test: when MTP is stacked with another draft method called n-gram, the NVIDIA acceptance rate collapses to 3.8% and throughput drops 78%, while AMD jumps 210%. This "compatibility inversion" suggests the problem may not be the GPU itself, but a clear blind spot in llama.cpp's optimization path for the CUDA backend (the driver layer). In other words, this may not be NVIDIA's fault — it may be that the open-source inference framework optimizes more carefully for AMD than for NVIDIA. A counterintuitive, and not particularly flattering, conclusion.
Impact on regular people
For enterprise IT: If your company plans to build an on-prem LLM cluster for compliance, customer service, or code assistance, AMD GPUs have entered the "no longer wait-and-see" shortlist for the first time — provided your technical team can debug low-level tools like llama.cpp.
For individual careers: Technical people who understand GPU scheduling and quantized deployment are becoming a scarce resource. You don't need to write drivers — the people who can tell the boss "buy AMD or NVIDIA" during procurement are getting more valuable by the quarter.
For the consumer market: Consumer users don't need to replace their laptops in the short term, but this signals that the "AI notebook" category may no longer be an NVIDIA-only domain. AMD's window of opportunity in the mid-range is opening.