A Reddit user ran Cloudflare's newly launched customer-service small model Clef through a basic test question — "should this be refunded?" — and got a confidence score of 0.57 out of 1.0. Swapping in a competing model called Jev, the same question immediately scored 0.99.
This matters: the enterprise calculus of replacing large models with small ones to cut costs may be harder to execute than the spreadsheet suggests.
What this is
Cloudflare (the CDN and security company) recently released a small model called Clef, purpose-built for customer-support ticket routing — determining which team a customer issue belongs to and how urgent it is. The model is small enough to run locally via llama.cpp (an open-source local inference framework). Testing used the Q4_K_M quantization build (a compression method that shrinks model size to roughly a quarter while sacrificing a small amount of precision) — the kind any ordinary enterprise can run.
Industry view
The enterprise market is currently fixated on "small models": lower cost, support for on-premises private deployment (installed on a company's own servers, not traversing the public internet), and data compliance. Cloudflare, Anthropic Haiku, and GPT-4o mini are all competing for this slice of the pie.
The pushback is sharp: when a model is compressed down to a few hundred million parameters, the ceiling on "understanding" gets cut. In testing, Clef did output the correct classification (billing), but with only 0.51 confidence; the urgency judgment was essentially a coin flip — across the three buckets "can wait / today / urgent" the probabilities came out 23%, 42%, and 35%, which is no judgment at all.
Another counter-argument: a single-prompt test is unfair. But we note the user was running Cloudflare's own official example and still got this result — meaning the out-of-box experience is itself failing.
Impact on regular people
For enterprise IT: Privately deploying small models remains, as of now, in the "technical validation" stage — short of production-ready. Don't trade support quality for token savings (the per-call billing unit of AI services).
For individual professionals: Using AI for email triage and ticket routing is a real need, but when picking a model, don't optimize purely for "small" and "cheap" — run it on your own real data before going live.
For consumers: Next time you reach an AI support agent that sounds vague or answers a different question than the one you asked, there's a decent chance a small model compressed to its limit is what's running the show.