A developer embedded an untrained Forth interpreter in the weights of NVIDIA's Nemotron 3.5 Lightning, leaving 3B of the 30B model's parameters active. After LoRA fine-tuning, scores on 600 questions rose from 79.1% to 92.1%, but the cost of executing one word at a time still limits its practical value.
The important development is not that the model “learned” a low-level language, but that computation has been moved into the model generation process itself. This approach reduces explicit tool calls, but it also puts inference efficiency in the spotlight.
What this is
This is a research project: within the same Nemotron model, the developer added 27 network layers whose weights are calculated by a script rather than learned through training. It functions as a Forth interpreter—a stack-based, word-at-a-time programming language. Each time the model produces a word, the interpreter executes one step and feeds the result back as the text for continued generation.
In testing, 765,734 machine Tokens—the model's internally generated computational steps—matched a Python reference implementation exactly. LoRA (low-rank adaptation, a fine-tuning method that adjusts only a small number of parameters) addressed the model's tendency not to “trust the machine”: even after receiving an answer, it would still repeat the calculation manually.
Industry view
We believe this experiment demonstrates that model weights can carry deterministic computational components, providing a concrete example of “execute while generating.” The developer also released the code, tests, training recipe, and failure records.
The risks are equally clear: every machine step requires a full forward computation, making it far slower than a conventional tool call. Numerical precision is also capped at 20 digits, and the model still loses to Python on long-integer tests. Moreover, most of the test set came from a specific arithmetic collection, so the results cannot be used to infer improved performance on general tasks. This remains a research demonstration, not a deployable product.
Impact on regular people
For enterprise IT: It offers a new approach to coordinating internal computational components with models, but at this stage it is better suited to research teams validating the concept; existing toolchains should not be rebuilt around it.
For individual professionals: More reliable local models may gradually take on computation and automation tasks. For now, its direct value lies in technical learning and open-source experimentation, rather than an immediate boost to office productivity.
For the consumer market: Consumers are unlikely to encounter this approach directly in the short term. Only if its efficiency problems are solved could model-driven computational services begin appearing in more desktop and mobile applications.