What This Is
This week, a test worth bookmarking surfaced on Reddit's r/LocalLLaMA: a developer ran the 35B-parameter Ornith-1.5 model on an RTX 3070 laptop (8GB VRAM), completing a full agentic coding workflow locally—AI reads code, analyzes it, modifies it, and delivers results—at roughly 32 tokens per second (tokens are the smallest units AI processes, roughly one Chinese character or half an English word).
He used the Q4_K_M quantization (compressing the model to roughly a quarter of its original size, trading a bit of precision for footprint). The architecture is MoE (Mixture of Experts, activating only 3B parameters per pass), with a 128K context window (enough to ingest a 200-page book in one go). He had been running Qwen3.6-35B-A3B long-term and reports Ornith feels smoother on both speed and "overall agent flow handling," with self-rated code quality as "near perfect."
Industry View
Caveat first: this is a personal test post, not a benchmark—replicability from a single user's experience is questionable. The author himself admits "tomorrow a new model might make me eat my words."
The bullish read: local large models have visibly improved over the past six months. 35B-class size plus MoE plus 4-bit quantization has shifted running large models on consumer GPUs from "toy" to "actually useful." For developers and enterprise R&D teams unwilling to upload code and data to the cloud, this is a real option—works offline, data stays in-house.
The caution: a single user's "near perfect" lacks horizontal comparison; a 35B MoE that activates only 3B still trails dense large models on complex reasoning; running 128K long context at 32 tok/s under tight VRAM raises real questions about stability across hardware and driver differences. This is not "AI no longer needs the cloud"—it is "there's now another path."
Impact on Regular People
For enterprise IT: Locally deployed AI coding assistants move from "worth a try" to "worth considering." Data stays in-house, works offline—attractive for finance, healthcare, and government. But the supporting ops, model updates, and compliance review capabilities aren't off-the-shelf; you need a dedicated team.
For working professionals: Non-technical roles don't need to panic—models like Ornith primarily serve coding work. But IT teams and R&D managers should add "local AI coding" to their H2 tool evaluation list.
For the consumer market: 8GB VRAM is now standard in mid-range gaming laptops, and hardware is starting to carry local AI workloads. In future laptops and tablets, "can it run a given open-source large model" may sit alongside "what games can it play" as a selling point.