What this is
Ornith-1.5-35B-A3B triggered a wave of benchmark discussions on Reddit this week: a 35-billion-parameter open-source model hitting 250 tokens/s on Nvidia's consumer-grade 5090 — the first time local AI has approached cloud-level interactive speed.
Two things made it possible: the model uses sparse activation (only ~3 billion parameters fire per inference, sharply cutting VRAM use), and the 5090's compute happens to be just enough. The original poster specifically notes it handles Agent tasks reliably — meaning the AI can break down steps, operate tools, and chain multi-step actions on its own, instead of just doing single-turn Q&A.
Industry view
Supporters see this as a tipping point for local AI: data never leaves the premises, per-generation hardware cost is essentially just electricity, and latency drops to near-imperceptible. For data-sensitive industries — finance, healthcare, legal — these three points are real demand, not marketing talk.
But we'd flag three cold realities. First, the 5090 sells for roughly 20,000–30,000 RMB in China, with supply constrained by export controls — not hardware a typical team can easily stock. Second, open-source models still trail commercial cloud flagships on stability, version iteration, and long-term support; when something breaks, there's no support line to call. Third, multi-step Agent execution on smaller models remains brittle — direct swap-in for mission-critical workflows is premature.
Our read: local AI has moved from "can run" into "basically usable," but it's still 12–18 months from "safe cloud replacement."
Impact on regular people
For enterprise IT: Industries with strict data-compliance requirements now have a cloud-vendor-independent option worth budgeting a local deployment pilot line item for in 2026.
For technical professionals: Engineers and developers get trial-and-error costs down to nearly zero — no more wincing at API quotas per call.
For consumer market: Regular users won't feel it short-term, but subscription AI products will likely start cutting prices within 1–2 years, since open-source local deployments will cap the ceiling on what they can charge.