small language models

$807, One H100, 2.8B Params: Why Small, Specialized Beats Big, General

$807, one H100: Unbounded Labs trained Bart, a 2.8B pre-1931 English LLM, plus the first vintage benchmark. Does small-and-specialized have a future?