This week, Reddit's LocalLLaMA board published a leaderboard: testing every large model that runs on GPUs with 10-16GB of VRAM (consumer gaming-card territory), ranked on "everyday writing and roleplay" criteria. What caught our eye: Alibaba's Qwen 3.6 35B-A3B and Zhipu's GLM-4.7-Flash 31B-A3B—two Chinese MoE (Mixture of Experts, meaning different sub-modules are activated based on question type) models—both cracked the top five.

What This Is

The reviewer used six criteria: generation speed (at least 10 tokens per second, the unit measuring how fast a model outputs text), internal coherence of dialogue/monologue, multilingual handling (Portuguese, Japanese, English), cliché density (the "leaving a long shadow," "smelled like ozone and caramel" style of AI boilerplate), MoE versus full-parameter tradeoffs, and whether the model over-refuses (many have been "de-censored," i.e., abliterated/heretic variants). The verdict: Gemma 4 26B-A4B takes first place, with Qwen and Mistral families close behind.

10-16GB of VRAM means an RTX 4060 Ti/4070-class gaming card can handle these—not an enterprise A100/H100. This pushes "running AI at home" out of the geek-toy bracket into something an ordinary PC gamer can actually touch.

Industry View

The supporters' verdict is clear: open-source small models are already "good enough." Daily writing, chitchat, simple Q&A—a sub-35B model handles these smoothly without demanding hardware. The fact that Qwen and GLM can stand toe-to-toe with Gemma and Mistral means Chinese LLM companies are no longer supporting players in the open-source ecosystem.

But the reviewer himself threw cold water on that: he ran knowledge quizzes on anime, manga, film, and music and found 70B and 120B models badly lacking on the common-sense facts of these domains—"only models above 200B actually carry that knowledge." A 30B model can already "speak like a human," but stacking parameters is still the only path to "actually knowing things."

The more pointed question: models above 200B won't run on an ordinary laptop. The real positioning of "local LLMs" is "fine for light tasks, heavy lifting still requires the cloud."

Impact on Regular People

For enterprise IT: Open-source small models fit "draft-grade" work—first-pass generation for internal knowledge bases, first-round customer service triage, document summaries. But on specialized-domain knowledge, you still need web retrieval or a larger model as a backstop—otherwise the hallucination (AI confidently making things up) cost gets steep.

For individual professionals: For data-sensitive fields like law, medicine, and consulting, a "no internet, no upload" local model is now a genuinely usable option—especially for anyone who doesn't want ChatGPT/Claude/Gemini reading client files.

For the consumer market: "AI running on your own machine" is shifting from concept to consumer product, but expectations need managing: it can draft your emails, polish your copy, plan your trips—but don't expect it to actually understand your professional domain.