Alibaba
$280 Mining FPGA Runs Qwen 9B — Local LLMs No Longer a Rich Person's Game
A tinkerer runs Alibaba's Qwen 9B on a $280 second-hand mining FPGA from eBay; 27B is also planned. The local LLM cost barrier is quietly crumbling.
2 articles tagged with this topic
A tinkerer runs Alibaba's Qwen 9B on a $280 second-hand mining FPGA from eBay; 27B is also planned. The local LLM cost barrier is quietly crumbling.
Karpathy's MicroGPT deployed on FPGA hits 50K tps by storing weights in on-chip ROM instead of external memory. This proves edge AI inference is bottl