1 article tagged with this topic
Two 30B MoE models now run on a 128GB AMD Strix Halo mini PC at 30-44 tok/s. Models that once needed multiple H100s now fit a $1,400 desktop.
