What this is
A Juejin technical test published this week reaches a counter-intuitive conclusion: on the Horizon Journey 6M (J6M) chip running YOLOv5x (a common image recognition model), scaling threads from 1 to 16 took FPS from 96.7 to 100.4 — essentially flat — while average per-frame latency ballooned from 10.3 ms to 158 ms, a 15x increase.
The engineer wants to flag two things. First, how fast a model runs does not depend on how many "pipelines" you spin up — it depends on the AI inference accelerator (BPU), which can only process one task at a time. More threads just thinly spreads the same total capacity. Second, model loading time (DDR load took 495 ms) and single-frame inference time (11 ms) cannot be conflated, or you will reach absurd performance conclusions.
Industry view
On the surface this is a technical blog post, but read at the industry level, we see it reflecting what Chinese AI chip companies are scrambling to learn: the developer toolchain. For the past few years everyone competed on raw compute specs (TOPS) and process node. Now more and more customers are discovering that whether a chip can ship at scale and run reliably depends on how good the tools are. Horizon, Huawei Ascend, and Cambricon are all working on this.
A few caveats are warranted, however. First, this is a single-model, single-chip test — the conclusion does not generalize to industrial scenarios with dynamic input or ROI (region of interest). Second, the tooling measures the model's own time; in real deployments, image capture, decoding, and preprocessing are what hammer the CPU. Pretty benchmark numbers do not equal stable production. Third, Horizon's main battlefield is automotive; industrial, security, and other more fragmented markets remain a proving ground.
Impact on regular people
For traditional enterprise IT: smart vision projects are becoming commonplace. If procurement decisions look only at FPS benchmarks and ignore "latency distribution" and "real experience under high concurrency," you will walk into painful traps later.
For working professionals: if your company touches AI vision projects, remember to keep "model inference time" and "end-to-end application runtime" in separate conversations, or you will blow up at delivery acceptance.
For consumer markets: Horizon chips mostly land in vehicles (Li Auto and select BYD models). A more mature toolchain means lane keeping, pedestrian detection, and similar features may become more stable and cheaper going forward — but in the short term what you actually notice is still incremental in-car infotainment updates.