This week we look at an open-source project that did something counterintuitive: it crammed a 37GB model into a browser tab on a 24GB laptop—by streaming MoE weights off the SSD. The point isn't speed (about 3x slower than llama.cpp); it's that "running big models locally" has crossed into engineering-usable territory.
What this is
LocalMind, released by NakliTechie, is a static webpage that runs large models using the browser's built-in WebGPU—the interface that lets web pages call the GPU directly. The key trick is "streaming MoE weights." MoE (Mixture of Experts) splits a model into many small "expert" sub-networks and only activates a handful per inference. LocalMind keeps the inactive experts on the SSD and fetches them on demand. The result: a MacBook M4 Pro with 24GB of RAM can run a 37GB Qwen3.6 35B-A3B and a 14.4GB Gemma 4 26B-A4B, with output that closely matches llama.cpp—the de facto standard open-source inference engine.
The trade-offs are explicit: 23.6 tokens/second decode (about 24 tokens per second), versus llama.cpp's Metal build at 70.6 tok/s—roughly 3x slower. First load of the 14.4GB model takes 11.5 minutes; subsequent loads open instantly.
Industry view
Supporters argue this approach "punctures the myth of cloud AI"—large models don't have to keep living in data centers and behind APIs. A laptop browser can host them, which carries real weight for data-sensitive enterprises.
Critics push back just as hard. A 3x speed gap is fatal for real-time applications. WebGPU's cross-browser stability is uneven. The project is maintained by a single person, with no real engineering rigor. Our more sober read: this is a tech demo, not a product form. It proves that "good enough" local AI is possible—but the path to commercialization is still long.
Impact on regular people
For enterprise IT: don't worry about replacing cloud solutions in the short term—the speed gap is right there. But the "sensitive data stays home" requirement from SMBs is likely to be the first real demand such tools actually meet.
For working professionals: technically inclined colleagues can use it to prototype internal tools. Everyone else can't use it yet—they'll need to wait until browsers and models get lighter.
For the consumer market: expect more "large models on phones and ultrabooks" stories, but genuinely usable products are still a way off.