106 Small AI Models Benchmarked — But What Can 150M Parameters Do?

What this is

This week, Hugging Face added a new open dataset called "Open SLM Evaluations." SLM stands for "Small Language Model" — typically under 150 million parameters. By comparison, mainstream large models routinely reach hundreds of billions or even trillions of parameters. The dataset collects scorecards for 106 small models, including benchmark scores, parameter counts, and model links.

One notable detail: the entire evaluation ran on a single B200 GPU (NVIDIA's latest-generation AI chip) for just 30 minutes, with the dataset released under the Apache 2.0 license (a permissive open-source license). The curator, Enderchef, is an independent developer.

Industry view

Supportive voices see this as filling a gap. We note that large-model evaluations are everywhere — but nobody has seriously benchmarked small models side-by-side. Phones, IoT devices, and offline scenarios precisely need models that fit in small boxes. An independent, low-cost, and open benchmark is foundational infrastructure for developers making selection decisions.

But skepticism is also abundant. First, at the 150-million-parameter scale, even moderately difficult tasks like translation are handled poorly, let alone coding or analysis. A high benchmark score doesn't mean real-world usability — as the industry often puts it, "the benchmark looks pretty, but the moment it hits a real task, it falls apart," and plenty of small models fit this pattern. Second, one person, one GPU, one leaderboard: the sample is thin and updates may not keep pace with model iteration speed. Third, the sustainability of an open-source dataset is also a question — once the maintainer stops updating, the leaderboard quickly goes stale.

Impact on regular people

For enterprise IT: Small and mid-sized businesses that are data-sensitive but can't afford large-model APIs can take a look at this leaderboard as a reference starting point for localized deployment — but don't expect it to replace mainstream large models.

For individual professionals: Cloud-based large models like ChatGPT, Tongyi Qianwen (Qwen), and Wenxin Yiyan (ERNIE Bot) remain the first choice for office work — small models won't enter your workflow anytime soon.

For the consumer market: Over the next year, more offline AI will appear in phones, smart speakers, and in-car infotainment systems, but functionality will still be limited to simple Q&A and command execution. Complex work will still need the cloud.