What this is

The days of running a large model while separately purchasing an embedding API may be numbered, courtesy of an open-source plugin. We've noticed that French AI company NuMind submitted a plugin to vLLM (an open-source LLM deployment framework) this week: once bolted onto a running large language model (LLM, such as Qwen, Llama, or DeepSeek), it makes the model spit out embeddings as a side effect.

Embedding is the underlying technology of AI retrieval—converting text or images into a string of numbers so a computer can judge "how similar two pieces of content are." Previously, this typically required separate calls to OpenAI's text-embedding-3, Cohere, or domestic alternatives like BGE and M3E. NuMind's approach is straightforward: prompt the model to "summarize the above content in one word," then extract the last hidden state (an intermediate vector produced by the model's internal computation) and use it as the output. The code is released under Apache-2.0 and is currently pending merge into vLLM's main branch.

Industry view

The optimistic read: the cost structure of doing RAG (having AI first retrieve from a company's own documents before answering—the current default pattern for enterprise AI deployment) could be flattened. Many companies pay to run an LLM and pay again for an embedding API—one server could potentially do both jobs. NuMind itself is a beneficiary of this approach; their NuExtract3 (a 4B-parameter open-source structured document extraction model) was built using a similar method.

But what warrants our caution: hidden states were never designed as embedding vectors in the first place, and their quality is highly dependent on model scale, prompt, and task type. In production, purpose-trained embedding models are still likely more accurate and faster. This path only works for teams "self-hosting large models"—companies consuming OpenAI or DeepSeek cloud APIs can't access the model's internal state and won't save a cent. Also, vLLM hasn't officially merged this PR yet, so in the short term it remains in a "fun to tinker with, not stable" state.

Impact on regular people

  • For enterprise IT: Teams self-building AI infrastructure (privately deployed LLMs) should recompute their budget—embedding line items may be cut or scaled down.
  • For working professionals: Regular employees don't need to pay attention yet; once wrapped at the product layer, it may take a year to surface in everyday tools.
  • For the consumer market: End users will barely feel it; but over the long term, pricing for enterprise knowledge bases, AI customer service, and similar products may trend down.