In testing on an 840M-parameter model, 4-bit training ran about 28% faster than the widely used BF16 training, with final validation losses of 1.688 and 1.691 respectively—virtually identical. These results matter: they show that the compute bottleneck in large-model training may soon be eased through lower numerical precision.

What this is

This is an independent reproduction of the native 4-bit forward training that Thinking Machines Lab has called Inkling 2 Small. Four-bit quantization stores and computes model data using fewer bits, reducing memory and compute overhead. This approach uses NVFP4 for the forward pass and FP8 for the backward pass while retaining high-precision master weights.

The test used a single NVIDIA B200 GPU and PyTorch 2.8, covering models with 219M, 840M, and 3.3B parameters. Training speedups expanded from 18% on the smallest model to 28% for the 840M-parameter model and 38% for the 3.3B-parameter model. The author argues that as models grow, low-precision training may amortize fixed overhead more effectively.

The evidence still has limits. The largest model was trained for only 1,000 steps and had not fully converged. Multi-GPU setups, mixture-of-experts models, and some high-precision scaling schemes were not tested. This is therefore not yet sufficient proof that large-scale training costs have fallen.

Industry view

More efficient training still has practical value. If these results scale reliably, large-model companies could do more training with the same hardware or train larger models within similar budgets. Vizuara AI has released the code and itemized data, lowering the barrier for other teams to verify the approach.

The risks are equally clear. Blackwell has issues involving numerical conversion and attention backward propagation that cannot be fully reproduced, potentially affecting rigorous controlled experiments. Whether low-precision training remains stable during long runs and across models with tens to hundreds of billions of parameters requires more evidence. Community reproduction demonstrates promise, but it cannot replace the vendor’s own engineering validation on multi-GPU, large-scale training.

Impact on regular people

For enterprise IT: In the future, model procurement need not compare only price and benchmark scores; it can also compare compute cost per unit. If low-precision training matures, hardware budgets for private-model training and industry-model fine-tuning may fall, although deployment gains will still depend on software support and service stability.

For individual careers: In the short term, the impact is more likely to emerge first in model development, cloud computing, and the chip ecosystem, rather than directly changing the daily tools used in most jobs. Companies should watch whether vendors translate efficiency gains into lower prices or faster delivery.

For the consumer market: Falling training costs may expand the service supply of smaller model companies, but they will not automatically produce cheaper or more reliable end-user applications. Consumers should watch whether enterprise model-service prices, response speeds, and product coverage improve as a result.