Miles v0.1 went open-source on GitHub this week, systematically breaking down for the first time the engineering challenges of "taming" large models (reinforcement learning post-training — making the model repeatedly solve, grade, and improve). "Taming" large models used to be the core black box of OpenAI and Google; we've noticed that Chinese teams are now turning this into an open-source pipeline.
What This Is
Miles comes from the RadixArk team, built on top of the existing slime framework. Traditional training and inference are two separate steps: first train the model, then deploy it. But "taming" a model is different — the model must continuously generate answers, score them, update, and regenerate. The inference engine is actually embedded inside the training loop.
Miles decomposes the job into three components: SGLang lets the model generate answers quickly, Megatron-LM computes gradients (the mathematical signal for "which way to adjust"), and Ray schedules processes and GPUs. It supports two deployment modes: colocated (the same set of GPUs alternates between sampling and training) and disaggregated (sampling and training each occupy a dedicated set of GPUs, enabling true parallelism).
Most noteworthy is "weight sync" — every time the model updates, the copy responsible for generation becomes stale. Miles provides multiple synchronization paths: CUDA IPC (direct GPU-to-GPU transfer within the same machine), NCCL broadcast (multi-machine, multi-card parameter sync), RDMA based on Mooncake (making GPUs across two machines exchange data as if they lived on the same machine), and a disk-delta scheme that transmits only weight deltas.
Industry View
Supporters see this as a continuation of the open-source-ification of LLM infrastructure — inference engines already have vLLM and SGLang, training frameworks already have Megatron and DeepSpeed, and now the most complex piece, the "training-inference loop," is also starting to be broken apart and open-sourced. This lowers the barrier for small and mid-sized companies to do their own "taming."
But skeptics flag three issues. First, the hardware barrier hasn't moved — the article's authors themselves admit "there's no suitable hardware available right now to run full training." Without dozens of H100s (NVIDIA's top-tier AI chip, each priced at several hundred thousand RMB), the framework remains out of reach. Second, the inherent instability of RL training itself — in async mode, data staleness (after several rounds of model updates, sampled data may come from an outdated older model) — which the authors explicitly point out is "not just a system tuning problem, it also affects final training quality." Third, the proliferation of competing open-source frameworks (slime, OpenRLHF, Verl, etc.) with significant design divergences — enterprise selection is itself a cost.
Impact on Regular People
For enterprise IT: Going forward, building industry-specific LLM customizations (legal, medical customer service) won't necessarily require depending on big-tech APIs (Application Programming Interfaces — the "channels" used to invoke someone else's model). Small and mid-sized teams can theoretically use open-source frameworks to do their own "taming" — but they need a compute budget.
For individual careers: Understanding the engineering complexity behind "taming" models means you won't be easily snowed by vendor pitches like "our RL training is very strong" when negotiating with AI suppliers. There's a lot of hard engineering under the hood — not mysticism.
For the consumer market: As open-source RL frameworks mature, we'll see more AI products that truly "understand specific scenarios" — rather than cookie-cutter LLM wrappers everywhere.