What This Is
This week Databend dropped a striking data point: they compared three AI Agents across 1,171 execution steps and found that even when all final tests passed, path length and idle-loop counts varied by several multiples. Jev is the key tool here — developed by TypeSafe, and wired by Databend into its own database.
Unlike general-purpose large models that produce open-ended commentary, Jev scores each step against predefined structured questions: did this step advance the task, repeat prior actions, or touch the root cause? Verdicts are written directly back to the Databend database, and teams keep running off-course rates and model comparisons in SQL.
Industry View
Supporters see this as filling a gap. For the past two years, the industry has been racing on "can it run." Now it's time for "does it run stably, and how many tokens does it burn." Databend ran the numbers: one million execution steps per day, reviewed by general models like GPT-4.1 mini, costs $1,000–$3,500 daily. Jev compresses that cost by more than an order of magnitude.
There's pushback. One view holds that this infrastructure suits companies building their own Agents; most enterprises simply call ready-made products (Cursor, Devin, and the like) and don't need to reinvent the wheel. Another concern: small-model judgment accuracy on more complex tasks isn't fully validated. Open-ended root-cause analysis remains out of reach, and large models still have to catch the fallback cases.
Impact on Regular People
For enterprise IT: If your company is building or heavily customizing AI Agents, factor "path cost" into the books — beyond "pass rate." Two Agents on the same task can burn ten times the tokens.
For individual professionals: When using AI to write code or do research daily, watch the hidden costs across tools. The same task can drive one tool to repeatedly read files and retry, silently inflating your bill.
For the consumer market: No direct near-term impact. But as the industry focuses more on "process visibility," AI assistant products may eventually expose more usage data, making enterprise selection easier.