This week's number worth recording: a developer ran DeepSeek's latest open-source model on an M2 Ultra Mac workstation (around RMB 50,000, roughly $7,000, 192GB of unified memory) at 25.8 tokens/s — roughly 26 Chinese characters per second, smooth enough for real conversation. Notably, he used a method he calls "lossless repacking" to compress the model down to 141GB, smaller than the official quantized version, with no quality loss.
What this is
DeepSeek is a leading Chinese open-source large model company. The "V4 Flash" referenced in the community post belongs to its latest generation of large models, using a MoE (Mixture of Experts) architecture — splitting a large model into multiple "expert" sub-models that are invoked on demand. The model is large in footprint, but the actual inference compute demand stays manageable.
The author's key move: he forked the open-source inference framework llama.cpp and did three things — lossless repacking of the model, using SSD to spill the conversation cache (KV cache, the model's "short-term memory"), and dynamically allocating 8 processing lanes. The result: a total context window of 1 million tokens.
Bottom line: the open-source community can now run a frontier-tier Chinese large model in near-real-time on a pro workstation, fully local, no internet required.
Industry view
The optimistic take: locally deploying open-source Chinese large models has graduated from "barely runs" demo status to actually being useful. For finance, healthcare, and government clients where data can't leave the premises, this is a substantive positive — they no longer have to bet entirely on closed-source APIs.
But we want to pour some cold water. First, the hardware barrier starts at RMB 50,000, and it still requires a developer to hack the underlying code — extremely hard to replicate, far from "IT department plug-and-play." Second, the "V4 Flash" in the community post is not an officially confirmed DeepSeek release; the naming should be treated cautiously. Third, 25 tokens/s is fine for chat, but for batch document processing or running Agent automation (letting AI autonomously complete multi-step tasks), there's still an order-of-magnitude gap.
Impact on regular people
For enterprise IT: the ceiling of open-source large models is being shaved down bit by bit, but if your team lacks someone who understands low-level inference frameworks, the cost-effectiveness still isn't better than calling a closed-source API in the short term.
For working professionals: unless you're buying your own top-spec Mac Studio and willing to wrestle with the command line, this doesn't concern you. The indirect signal is — over the next year, "local AI assistants" will get cheaper and faster.
For consumer markets: no direct impact yet, still a geek-circle experiment.