801-30B-A3B-Base
801 Moves Self-Improvement Into Pretraining: The Real Bottleneck Is Now Systems Engineering
GAIR trained 801-30B-A3B-Base on 11.8T tokens in three months, beating Qwen3 and Nemotron peers.
1 article tagged with this topic
GAIR trained 801-30B-A3B-Base on 11.8T tokens in three months, beating Qwen3 and Nemotron peers.