The client asked: “Can AI run on an offline device?”
Last Tuesday, I was demoing automatic photo organization for a freelance photographer in Beijing. The client interrupted: “Could I buy a device that recognizes people and scenes where there’s no internet?” I froze for a second. An ordinary computer could do it, but it was bulky and power-hungry—clearly not something he wanted in his outdoor photography bag. I felt a little uneasy too: I had never actually costed out the hardware for a request like this.
What DeepX is building—and who might use it
DeepX is a startup building AI chips for on-device use. It recently announced that it had taped out its second-generation chip, with plans to use LPDDR5X and compute-in-memory technology. The team says the chip can support inference for models with up to 100 billion parameters while consuming less than 5W, while targeting roughly a 6.7× performance increase.
What does that look like in practice? Imagine Lin Zhe, a hardware entrepreneur in Shenzhen, adding portrait recognition and subtitle generation to a drone or industrial inspection device. It cannot stay connected to the cloud, and it cannot carry a bulky power supply. DeepX wants to fit large-model capabilities into a smaller, more energy-efficient device. But “taping out” does not mean mass production has begun, and we cannot treat the announced specifications as results ordinary users have already measured.
The cost and technical barrier to reproducing this today
If I wanted to build something similar right now, I would need to prepare millions of dollars, then wait for chip production, receive a development board, and complete software adaptation. A realistic early-stage timeline is at least 6 to 12 months. The technical barrier is high as well: this is not a button-click setup. It usually requires understanding model compression, chip compilation, and embedded deployment. The first step is not buying anything; it is contacting DeepX to confirm the development board, sample timeline, and developer program, then asking a solution provider to assess whether the model can move into its runtime.
One thing needs to be clear: not every independent creator needs this. It is perfectly fine not to try it yet. If an ordinary computer or an existing cloud service already solves the need, switching to a dedicated chip could simply add cost. There is another limit here too: most of the current information comes from the company. Mass-production pricing, real power consumption, and software compatibility still need to be verified later.
How I would choose at this stage
If I had just started and mainly took writing or design jobs, I would stick with my existing computer and usage-based cloud services, avoiding dedicated chips because the costs are usually easier to control. If I already had one or two clients—and one explicitly required offline operation and low-power devices—I would first build a one-page budget covering power consumption, size, and response time, then speak with DeepX or a hardware solution provider about samples. If I was scaling up and preparing to preload the capability into a batch of devices, I would focus on the real sample’s power draw, first-version latency, model migration time, and volume pricing, while keeping a cloud option running in parallel when needed.