What This Is

Exo Labs is an open-source project focused on local large-model deployment—running AI models on users' own hardware rather than in the cloud. This week they disclosed a figure on Reddit: by clustering two Mac Studios equipped with Apple's m5u chip and 96GB of memory via RDMA (a technology that lets two machines cooperate as if sharing a single pool of memory), they hit 4.8TB/s of memory bandwidth.

Their core claim is "linear bandwidth scaling with cluster size"—add one machine, double the bandwidth. If true, this means stacking consumer-grade hardware into clusters could approach the performance of data-center GPUs. The 96GB m5u Mac Studio retails for roughly ¥30,000 (~$4,200 USD).

Industry View

The local AI community is split. Optimists see this as a cure for the "NVIDIA tax"—running 70B+ parameter models without paying for H100/H200. But the pushback is just as concrete:

  • Bandwidth and latency are two different things. Exo's own staff admitted in the comments that the real bottleneck of their setup is latency (the time for data to shuttle between machines), not bandwidth itself;
  • Whether 4.8TB/s is a peak or sustainable figure is still being independently tested by the community;
  • Apple silicon has thin CUDA ecosystem support (NVIDIA's dominant AI compute framework, effectively the lingua franca of AI). Most mainstream models aren't optimized for it, and real-world efficiency suffers.

One experienced user's takeaway in the comments: buy the single 256GB version first, expand later. Safer than betting on a dual-machine cluster.

Impact on Regular People

For enterprise IT: Local AI finally has a "doesn't burn cash" option. If the approach matures, it's especially meaningful for data-sensitive small and mid-size organizations (healthcare, legal, government)—they could own AI capabilities without going to the cloud.

For individual professionals: Now isn't the time for consumer-grade users to jump in. A ¥30,000 Mac Studio running 70B models is still slower than cloud APIs (paid per call). But it's worth watching—this is the start of a trend.

For the consumer market: If Apple keeps pushing Mac Studio clustering, a wave of displaced older workstations will hit the secondary market. Small businesses with real compute needs should watch that window.