返回首页

对比阅读

对比阅读:4-Bit Training Works on One B200, but Scale Still Determines Speedup 与 单张 B200 跑通 4 位训练,模型提速还要看规模效应

AEN
Thinking MachinesNVIDIA B200Low-Precision Training·

4-Bit Training Works on One B200, but Scale Still Determines Speedup

In testing on an 840M-parameter model, 4-bit training ran about 28% faster than the widely used BF16 training, with final validation losses of 1.688 and 1.691 respectively—virtually identical. These results matter: they show that the compute bottleneck in large-model training may soon be eased through lower numerical precision.

What this is

This is an independent reproduction of the native 4-bit forward training that Thinking Machines Lab has called Inkling 2 Small. Four-bit quantization stores and computes model data using fewer bits, reducing memory and compute overhead. This approach uses NVFP4 for the forward pass and FP8 for the backward pass while retaining high-precision master weights.

The test used a single NVIDIA B200 GPU and PyTorch 2.8, covering models with 219M, 840M, and 3.3B parameters. Training speedups expanded from 18% on the smallest model to 28% for the 840M-parameter model and 38% for the 3.3B-parameter model. The author argues that as models grow, low-precision training may amortize fixed overhead more effectively.

The evidence still has limits. The largest model was trained for only 1,000 steps and had not fully converged. Multi-GPU setups, mixture-of-experts models, and some high-precision scaling schemes were not tested. This is therefore not yet sufficient proof that large-scale training costs have fallen.

Industry view

More efficient training still has practical value. If these results scale reliably, large-model companies could do more training with the same hardware or train larger models within similar budgets. Vizuara AI has released the code and itemized data, lowering the barrier for other teams to verify the approach.

The risks are equally clear. Blackwell has issues involving numerical conversion and attention backward propagation that cannot be fully reproduced, potentially affecting rigorous controlled experiments. Whether low-precision training remains stable during long runs and across models with tens to hundreds of billions of parameters requires more evidence. Community reproduction demonstrates promise, but it cannot replace the vendor’s own engineering validation on multi-GPU, large-scale training.

Impact on regular people

For enterprise IT: In the future, model procurement need not compare only price and benchmark scores; it can also compare compute cost per unit. If low-precision training matures, hardware budgets for private-model training and industry-model fine-tuning may fall, although deployment gains will still depend on software support and service stability.

For individual careers: In the short term, the impact is more likely to emerge first in model development, cloud computing, and the chip ecosystem, rather than directly changing the daily tools used in most jobs. Companies should watch whether vendors translate efficiency gains into lower prices or faster delivery.

For the consumer market: Falling training costs may expand the service supply of smaller model companies, but they will not automatically produce cheaper or more reliable end-user applications. Consumers should watch whether enterprise model-service prices, response speeds, and product coverage improve as a result.

BZH
Thinking MachinesInkling 2 SmallNVIDIA B200·

单张 B200 跑通 4 位训练,模型提速还要看规模效应

在 840M(8.4 亿)参数规模的测试中,4 位训练比常见的 BF16 训练快约 28%,最终验证损失分别为 1.688 和 1.691,几乎没有差异。这组结果值得关注:它显示大模型训练中的算力瓶颈,开始有机会通过降低计算精度来缓解。

这是什么

这是一次对 Thinking Machines Lab 所称 Inkling 2 Small 原生 4 位前向训练的独立复现。4 位量化,是用更少的数字位保存和计算模型数据,以减少显存与计算开销;这次方案在前向传播中使用 NVFP4,在反向传播中使用 FP8,同时保留高精度主权重。

测试基于单张英伟达 B200 GPU 和 PyTorch 2.8,覆盖 2.19 亿、8.4 亿及 33 亿参数模型。训练提速从小型模型的 18%,扩大到 8.4 亿模型的 28%和 33 亿模型的 38%。作者认为,模型越大,低精度训练可能越能摊薄固定开销。

但证据仍有边界:最大模型只训练了 1000 步,尚未充分收敛;多 GPU、混合专家模型和部分高精度缩放方案也未测试。因此,这还不是大规模训练成本已经下降的充分证明。

行业怎么看

训练效率提升仍有现实价值。若结果能稳定扩展,大模型公司可以用同等硬件完成更多训练,或在相近预算下训练更大模型。Vizuara AI 已公开代码和逐项数据,也降低了其他团队验证这条路线的门槛。

风险也很明确。Blackwell 在数值转换和注意力反向传播上存在不可完全复现的问题,可能影响严格对照实验;低精度方案能否在长时间训练、数百亿至千亿参数模型中保持稳定性,仍需更多记录。社区复现说明方案具备潜力,却不能替代厂商在多卡、大规模训练中的工程验证。

对普通人的影响

对企业 IT:未来模型采购不只比较价格和榜单分数,也可以比较单位算力成本。若低精度训练成熟,私有模型训练和行业模型微调的硬件预算可能下降,但部署收益仍取决于软件适配与服务稳定性。

对个人职场:短期内,影响更可能先出现在模型研发、云计算和芯片生态,而不是直接改变多数岗位的日常工具。企业需要关注供应商是否把效率收益转化为更低价格或更快交付。

对消费市场:训练成本下降可能扩大中小模型公司的服务供给,但不会自动带来更便宜、更可靠的终端应用。消费者真正值得观察的,是企业级模型服务价格、响应速度和产品覆盖范围是否随之改善。