This week on r/LocalLLaMA, Reddit user RapidRaid released a fork of llama.cpp (currently the most popular open-source tool for running large models locally), implementing a technique called "DSpark PC Tree" speculative decoding (in plain terms: let the AI guess several tokens at once — if the guesses are right, skip ahead; if wrong, correct). On an RTX 5090 running the Qwen3 8B model, generation speed jumped from 94 tok/s to 159 tok/s, a 65%-69% speedup. The "29.5%" in the paper's title was self-added by the author and is inaccurate; a more grounded description is "about 65% speedup."
What this is
Standard inference generates one token at a time (think of a token as a "character"). Speculative decoding uses a small model (a lightweight draft) to predict several tokens in a row, then lets the large model verify them all at once — if the guesses are right, you save time. PC Tree (Parent-conditioned drafting tree) is a new approach to this guessing strategy, letting the draft model make more accurate predictions based on the parent node's context. In this experiment, the k3/n16 configuration (guess depth 3, 16 branches per layer) hit 159 tok/s with a 67.87% acceptance rate.
Industry view
The community reaction is measured, with three main concerns. First, the author himself noted "first shot, no peer feedback" — this is a single data point. Second, when he tested the 27B Q4 quantized version, performance actually got worse, showing the technique doesn't scale linearly with model size. Third — and most critical — across the 11 scenarios in SPEED-Bench, summarization tasks gained +6.56%, but RAG (having the AI look up information before answering) and multilingual tasks actually slightly underperformed the linear approach (-0.5% to -1.4%). In other words, this is not a "total victory" but rather "effective in some scenarios." The fork has not been merged into the official ggml-org repository; it remains a "geek toy" for now.
Impact on regular people
For enterprise IT: this isn't production-ready yet — we don't recommend any action; wait for the official merge and observe stability over 1-2 more versions.
For working professionals: most workers won't feel a direct impact, but we see this as a signal — running 8B-class models locally on consumer GPUs already hits 150+ tok/s, nearly double what we saw a year ago.
For consumer markets: inference costs continue to fall, which will keep pressuring AI app subscription prices and cloud API call fees — good news for consumers over the long term.