On-Device AI

3 articles tagged with this topic

Qwen Crams a 9B Model into a 16GB Phone — China's First Real Say in On-Device AI

Reddit's LocalLLaMA names Alibaba's Qwen 9B as the top model for 16GB phones. On-device AI hits a tipping point — Chinese models earn their first real

6d ago2 min read

QwenRTX 5090

RTX 5090 Can't Run Qwen 27B Smoothly: The GPU Is Local AI's Real Bottleneck

Alibaba’s Qwen leads for local writing and chat, but even an RTX 5090 runs its 27B model too slowly. The real bottleneck is hardware.

Aug 222 min read

MetaMuse Glimmer

Meta Open-Sources 30B Local Agent Model — Big Tech Doubles Down on Putting AI Assistants on Your PC

Meta open-sources 30B-parameter Muse Glimmer, a local AI agent model hitting 20K tokens/sec on a single GPU, rewriting data privacy and cost calculus

Aug 102 min read