AI inference

2 articles tagged with this topic

AI Cold Start Cut from 116s to 1.4s — A Hidden Inflection in Compute Bills

Scaling AI inference, new nodes waited 116s. Nydus + JuiceFS cuts it to 1.4s and unifies model weights—quietly reshaping AI cost and elasticity.

Aug 192 min read

C++20double buffering

C++20 Double Buffering Ends Data Queuing: Underlying Engineering Sets AI Limits

C++20 lock-free double buffering doubles memory to parallelize data generation and processing. As LLMs surge, it eliminates idle cycles caused by data

May 62 min read