4 hours 18 minutes versus 21 minutes. 949 lines versus 1,759 lines. That is the comparison a Reddit developer posted this week. He handed the same ~39,000-line, ~600,000-token C-language shooter codebase to three AI setups and asked each to rewrite it as a single-file HTML/three.js web version.

The codebase exceeds any model's context window (the maximum amount of text an AI can "see" at once), forcing the AI to walk the file itself and judge which functions matter. The three results:

  • Cloud Claude Code Opus 5: 21 minutes, 1,759 lines, output "barely usable"
  • Local Hermes-orchestrated Qwen 3 27B: 4 hours 18 minutes, 949 lines, output "broken"
  • Local codehamr-orchestrated Qwen 3 27B: 1 hour 40 minutes, 1,056 lines, output also "broken"

The local setup used FP8 (a low-precision floating-point format that fits larger models into limited VRAM at some accuracy cost), the vLLM inference engine, and an RTX 6000 Pro 96GB GPU.

Industry view

These numbers surface at least two things worth thinking through carefully.

First, local "cheapness" has a ceiling. Even at the medium 27B scale, running FP8 on a 96GB card burns four hours of electricity and GPU depreciation — not trivial. Cloud's 21 minutes, billed per token, isn't necessarily more expensive per task. The hidden costs of going local — setup, debugging, power — most teams have never properly tallied.

Second, model weights aren't the deciding factor; the harness (the agent framework that lets AI plan across turns, call tools, and edit its own code) is. The original poster's most pointed finding: the same Qwen 27B weights, swapped between two different harnesses, both produced broken results. This tells us medium-scale open models are extraordinarily sensitive to how they are used — thin prompts cannot be rescued by elaborate scaffolding; they only burn more GPU time.

But the counterargument has to be on the table: one run and one prompt is not a verdict. The original author himself wrote, "this isn't a representative test." Claude's 21 minutes in the cloud reflects Anthropic's years of accumulated engineering in tool calling, code comprehension, and long-context handling — precisely what the open-source local ecosystem currently lacks. Treating a single comparison as an "open source vs. closed source" judgment is overreach.

Impact on regular people

For enterprise IT: Local LLM deployment is not "install and save money." On complex coding tasks, cloud APIs' engineering maturity is still a step ahead. Total cost must include toolchain development, debugging time, and labor.

For working professionals: If you are evaluating AI coding tools, right now a cloud subscription is more realistic than buying your own GPU and wrestling with deployment — unless your code or data genuinely must stay on-premises.

For the consumer market: "Download it locally and replace ChatGPT" marketing — take it with a grain of salt. Fast-running, poorly-used open-source models will be the norm for the next year. The model itself is just the entry ticket; the supporting toolchain is where the real divide sits.