In a series of tests on Qwen3 models, Huawei’s MACaQ achieved a KL divergence 23.5% lower than Unsloth at nearly identical file sizes. This indicates that its quantized models produce outputs closer to the originals, further lowering the barriers to deploying large language models locally.
What this is
MACaQ is Huawei’s open-source quantization method for large language models. Quantization compresses model parameters into fewer bits to reduce VRAM and storage requirements. The poster used KL divergence to measure the difference between the compressed and original models; a lower score generally indicates outputs closer to the original model.
The tests covered Qwen3-4B, Qwen3.6-35B-A3B and Qwen3.8-27B. MACaQ outperformed same-sized versions from Unsloth or Bartowski across all three test sets. On Qwen3.8-27B, its KL divergence was approximately 65% lower than Bartowski’s. The project also claims that a 122-billion-parameter model can be quantized in 64 minutes using 32.5GB of VRAM.
Industry view
The key development is not any single benchmark result, but Huawei’s decision to release an internal tool publicly, complete with code, a paper and a llama.cpp branch. This lowers the barrier for developers to test and reproduce its results, while giving the local model ecosystem another quantization approach to compare against alternatives.
However, these figures come primarily from tests by the project team and the community and cannot yet be treated as definitive. Different models, prompts and hardware environments may produce different results. MACaQ also does not currently provide a large selection of ready-to-use model files. Quantized models may still vary in inference speed, multimodal capabilities and accuracy on complex tasks, so their practical value depends on real-world business testing.
Impact on regular people
For enterprise IT: Businesses can run open-source models on less demanding servers, reducing their reliance on costly cloud inference. Before deployment, however, they still need to assess performance, maintenance requirements and security implications.
For individual professionals: As the file sizes and hardware requirements of local models continue to fall, more private AI tools will become available for handling internal documents, code or sensitive data.
For the consumer market: If more models and tools adopt this method, smaller and cheaper local assistants could reach PCs and edge devices. The final user experience, however, will still depend on the compute power and software support of each device.