The oft-cited claim that 90% of Agent projects get stuck at deployment is not substantiated in this article. But a more reliable assessment is this: when a model repeatedly fails on multilingual tasks, internal protocols, or long contracts, the problem often lies not in reinforcement learning, but in a gap in foundational capabilities.
We are seeing large-model teams shift from “reinforcement learning first, knowledge later” to “strengthen the foundation first, then optimize outputs.” The choice of training method directly determines the cost and final performance of enterprise AI projects.
What this is
Mid-training, also known as continued pre-training, teaches an existing model specific domain materials while retaining next-token prediction as the training method. It is well suited to building stable, large-scale capabilities in language, industry knowledge, and code.
Reinforcement learning is not useless; it is better suited to making tasks the model can already solve occasionally reliable. SFT (supervised fine-tuning) teaches the model how to answer through question-and-answer examples. But if a model cannot answer correctly after repeated attempts, the required knowledge may never have entered the model in the first place, leaving reinforcement learning without reusable examples of correct behavior.
A practical criterion is pass@k: making multiple independent attempts on the same task, with at least one successful attempt. If the model can occasionally succeed with more attempts, the capability exists but is unstable. If it always fails, Mid-training should be the first consideration.
Industry view
We believe this shift has practical implications for both enterprises and model providers. Companies do not need to place all their bets on expensive post-training. Data quality, professional-task evaluations, and continuous-update capabilities are becoming more valuable infrastructure.
The risks are equally clear: continued training can cause “catastrophic forgetting,” meaning that learning a new domain degrades performance in the original language or general capabilities. The article cites Mistral 7B, which gained Korean proficiency after continued training on a roughly 80% Korean and 20% English corpus, but saw a decline in English performance. Full-parameter training also means higher costs, making it more appropriate to validate the approach first with a small model and a small dataset.
Moreover, Mid-training can handle only relatively stable knowledge. For frequently changing facts or content that requires original sources, RAG (retrieval-augmented generation, which lets a model query an external knowledge base in real time) remains more appropriate. Writing dynamic information into model parameters may instead produce outdated answers.
Impact on regular people
For enterprise IT: When procuring model services, enterprises should look beyond leaderboard scores. They should require vendors to explain the sources of domain data, backtests for general capabilities, and version-update mechanisms. Projects involving internal policies need to distinguish stable knowledge from dynamic information.
For individual careers: When nontechnical roles use professional AI tools, repeatedly rewriting prompts may offer limited returns on tasks where the model keeps answering incorrectly. A more effective approach is to provide authoritative materials, restrict the permitted citation sources, and reserve high-risk conclusions for human verification.
For the consumer market: As specialized models continue absorbing industry materials, translation, contract review, and customer-service assistants will become more closely aligned with specific business needs. But a model that “remembers more” is not necessarily more reliable. Consumers still need to watch for outdated information, degraded capabilities, and missing sources.