This week, what caught our eye is Pathway's BDH (post-Transformer architecture) experiment: training from scratch at the 10M to 1B parameter scale, performance is nearly identical to GPT-2 in the same range. The key point — it runs on ordinary GPUs, with no dependence on H100 clusters.
What this is
BDH stands for "Baby Dragon Hatchling." You can think of it as a network structure that mimics human-brain neural circuits rather than pure mathematical attention mechanisms. Pathway wants to prove one thing: Transformer (the mainstream architecture for large models, the foundation of all GPT, Claude, and Llama) is not the only path, and may not be the optimal one. They ran controlled training at three parameter tiers — 10M, 100M, and 1B — using loss (the lower the value, the more accurately the model predicts data) as the metric. BDH and the GPT-2 baseline nearly overlap.
Parameter scale can be understood as the "brain capacity" of a model: the more parameters, the more knowledge the model can theoretically learn; 1B is roughly 1 billion parameters, and the largest version of GPT-2 was 1.5B.
Industry view
Supporting voices cluster around two points: open-source reproducibility — code and weights are fully public, and the community has already started a second round of verification. If this curve holds when extended to 10B or 100B, training costs could drop by an order of magnitude, giving AI's mid-size and smaller players another shot.
But the counterarguments are equally sharp. Most researchers point out that GPT-2 represents 2019's state of the art. To judge whether an architecture is truly a breakthrough today, the question is whether it can beat modern models like Claude or Llama. A Reddit comment put it bluntly: "Matching a toy from six years ago doesn't mean much." Additionally, BDH's inference efficiency (the speed and VRAM usage when the model actually runs and answers questions) has not been disclosed — cheap training doesn't equal cheap deployment, and half the picture for this path is still unfinished.
Impact on regular people
For enterprise IT: still a "good to know." Architecture typically takes 18–36 months to move from experiment to production deployment, and this won't change enterprises' compute procurement decisions this year.
For individual careers: no direct impact. But if you're responsible for selecting AI vendors, going forward you can ask one more question: "what architecture are you running on?" Cost structures are diverging, and the premium on pure GPU-stacking solutions may compress.
For the consumer market: zero short-term perception. In the long run, falling training costs will make small and mid-size companies willing to build vertical models (smaller models specialized for industries like healthcare, law, and education), and the AI products you use may become more specialized and cheaper.