An 11-million-parameter model with 8 labels and a size of roughly 12MB processed 190 scripts in just one minute. This is the captioning and segmentation model trained by Reddit user Pantheon3D on his own 4070 Ti. His tool, CueCaps, runs both transcription and caption labeling locally in the browser, with no data uploaded during the process.

What this is

The tool has two layers. Speech transcription uses onnx-community's whisper-base-timestamped through transformers.js in the browser. Machines with WebGPU take the fp32 GPU path, while the rest use WASM (WebAssembly, a format that compiles code into a browser-executable form). The decoder is further optimized with q8 quantization, which lowers numerical precision to save memory. The author's tests found that software adapters masquerading as WebGPU were actually slower than WASM, so he simply routed them back to WASM as well.

The more interesting layer is the second one: which words to bold, where to break sentences, and how to restore missing punctuation and capitalization. It started with hand-written rules, but was later replaced by a BERT-mini distilled from LFM3-Omni-2B, compressing a large model's capabilities into a smaller one. The model has just 11 million parameters and assigns 8 labels to each word. It can replace hand-written rules because it performs better on held-out data, not because it produces prettier results on the training set.

Industry view

We believe the value here lies not in the model itself, but in the trade-off structure it demonstrates: when transcription accuracy falls short, leave a cloud option available—the author acknowledges that Whisper base performs worse than larger models on noisy audio—while “structural tasks” such as segmentation and labeling can be handled by extremely small local models. The old human-machine collaboration problem has become an engineering division of labor: send the parts that require money and privacy to the cloud, and move high-frequency, verifiable tasks onto the device.

The risks are equally clear. First, the labeler currently supports only English. Chinese segmentation involves entirely different rules for punctuation and rhythm, and the distillation path may not transfer directly. Second, Whisper base has a hard accuracy ceiling. In short videos and multi-speaker conversations, errors in the transcription come first; better labeling afterward only formats those errors more attractively. Third, browser-based performance depends heavily on device differences. The older a user's machine, the more its execution path degrades toward the speed ceiling of WASM, making the overall experience vary from person to person.

Impact on regular people

For corporate IT teams: local inference means sensitive audio and video do not have to leave the internal network. This is a more affordable compliance path than procuring a large-model API, although the technology is still at the maturity level of a personal project.

For professionals: in editing, voice-over, subtitling, and similar work, the cost of repetitive segmentation and labeling is falling, freeing more room for differentiation in content judgment.

For the consumer market: in-browser speech transcription is shifting from “demo” to “usable.” In the near term, we expect to see more small tools for captions, translation, and meeting records that do not upload audio.