I almost got fooled by a pretty scorecard
Last Tuesday night, I was still tweaking a client's website in a café in Chaoyang District, Beijing. An AI assistant generated a page version in minutes, and the demo looked smooth. But when I asked it to finish login handling, error messages, and delivery documentation, we spent another two hours wrestling with it. That was when I realized: being able to write a piece of code is not the same as independently completing the whole job.
If I also use AI to take on client work or build small tools, this kind of scorecard may be more useful than watching generation speed every day.
MaxxBench does not crown the highest scorer; it tests real tasks
MaxxBench is an AI coding-agent benchmark released by Un Labs. It focuses on whether agents can complete software tasks that resemble real work, rather than merely solving one exercise or generating code that looks presentable. The paper, codebase, and experiment notes are publicly available on GitHub and the project website, making it relatively accessible to research and test.
I know a friend, Lin Yue, who works in automation. In March, she took on an inventory spreadsheet project in Hangzhou. Rather than trusting the model’s marketing, she tested it against one of her old tasks first: the output not only had to make the right changes, but also explain what changed and whether someone could reproduce the result from those instructions. She found that some tools delivered polished demos while missing edge cases. Now she includes “real-task pass rate” on her tool-selection checklist and no longer asks only which one generates code fastest.
Replicating it today does not require a complex setup
Replication cost: the public materials and a basic test run cost about 0 yuan. If you use a ready-made hosted entry point, there is usually a free allowance, with paid plans starting at roughly $20 per month. Time: allow 30 minutes to understand the test tasks, scoring method, and results table. To test it seriously against your own project, budget around two hours. Technical barrier: knowing how to copy task instructions and run an existing web page or command line is enough; programming knowledge is not required. You do need to judge whether the output meets the business requirements.
First step: open the MaxxBench project page, find the repository or trial entry point, and review one public task’s “task description—actual output—score result” before rushing to register or pay. I got stuck here myself: at first, I focused only on the total score, ignored the failure reasons, and wasted half an hour.
Decide whether it belongs in your workflow based on your stage
If I were just starting out and had not yet found stable client demand, I would test one small page of my own rather than spend more money chasing leaderboard positions. There is no urgency to try now. Benchmark scores are not my orders. If I already had one or two clients, I would use a sanitized real requirement as a comparison, focusing on completion rate, number of rework rounds, and clarity of the instructions. If I were scaling up, I could run different AI coding agents through the same set of internal tasks, record the time spent on manual intervention, and keep the team from being swayed by demo effects.
I treat it as a health check, not a purchasing decision. I validate with a small task first, then decide whether AI can take on more delivery work.