llama.cpp
llama.cpp Adds MageVL-Cowork, Expanding the Local Multimodal Ecosystem
llama.cpp now supports MageVL-Cowork with ready-to-use GGUF weights, lowering the barrier to running vision-language models locally.
Back to home
GGUF
18 articles tagged with this topic
KlangAI
Danish Open-Source Speech Model Tops Leaderboard, Puts Privacy First
KlangAI's Pianissimo-da leads a Danish speech-recognition benchmark and offers GGUF, ONNX, MLX, and CoreML formats.
Qwen3.8-Flash-Next
Unsloth Improves Qwen3.8 Quantization Accuracy, Lowering Deployment Barriers
Unsloth's Dynamic v3.1 quantization boosts accuracy by 10%–15% and cuts the smallest Qwen3.8-Flash-Next build to 28.8GB.
Hugging Face
Hugging Face Adds GGUF to Transformers — Research, Debug, Deploy on One Stack
Hugging Face adds native GGUF support to Transformers, letting Qwen3.5-4B quantized hit 98% of llama.cpp speed on M2 Max (70.4 vs 71.8 tok/s).
Hugging Face
Hugging Face Audit: 14% of Open-Source AI Model Files Are Mislabeled
Audit of 443 GGUF files across 25 Hugging Face repos found 64 (~14%) with filenames that don't match actual precision. Local AI users should recheck t
Alibaba
Alibaba's Qwen Local Updates Again — Another Option for Running LLMs on Your PC
Alibaba's Qwen local GGUF build updates with lower VRAM needs. The bar for self-hosted LLMs drops a notch, but this is routine open-source maintenance
AI9Stars
39B Open-Source LLM Runs on a PC — Without the Vendor's Help
AI9Stars' G9v3-39A5B was quantized to GGUF by Reddit user linuxid10t via a llama.cpp fork—open-source LLM tooling still runs on community relay.
Qwen
Undergrad's Rented 4090 Beats Six Quantization Teams; Qwen 9B Near-Lossless
An undergrad with a rented 4090 quantized Alibaba's Qwen3.5-9B to near-lossless, beating six established compression teams 31-3-0.
Unsloth
Unsloth Releases 30B Open-Source Model — Local LLMs Step Out of the Geek Bubble
Unsloth ships a GGUF-quantized Muse-Glimmer-30B, letting consumer laptops run a 30-billion-parameter LLM locally. On-prem AI is shifting from hobbyist
Google
Google Gemma 4 Fixes Chat Template — Local LLM Usability Inches Forward
Google fixed Gemma 4's chat template bug; community quantized versions updated. Not major news, but proves local AI usability inches up via detail ref
Mistral
Mistral Local GGUF Bug Fixed — Open Source QA Gaps Are Bigger Than You Think
Mistral Medium 3.5 GGUF files corrupted, community-fixed. Reveals open source QA gap: APIs tested, local formats not—impacts enterprise deployments.
Unsloth
Qwen3.6 GGUF Benchmarks
Un sloth claims top KLD-vs-disk-space performance for Qwen3.6-35B-A3B quants in 21 of 22 pareto frontier comparisons.
Apr 173 min readopcnew Editorialwww.reddit.com
Gemma- 4
Gemma 4 and Qwen 3.5 GGUFs: Detailed Analysis by oobabooga
Oobabooga published 5 benchmark reports covering 70-90 GGUF quants each for Gemma 4 and Qwen 3.5 models using KL Divergence methodology.
Apr 153 min readopcnew Editorialwww.reddit.com
Qwen3.5
Qwen3.5-9B GGUF Quant Rankings: Q8_0 Dominates KLD Scores
KLD benchmarks across community GGUF quants show Q8_0 variants cluster near 0.001 KLD, with quality degrading shar ply below Q5.
Apr 143 min readopcnew Editorialwww.reddit.com
llama.cpp
端侧AI 模型部署实战五(Android大模型加载)
Step-by-step JNI bridge implementation for running quantized LLMs on Android using llama.cpp.
Apr 143 min readopcnew Editorialjuejin.cn
Unsloth
Unsloth Releases Full GGUF Quant Suite for MiniMax M2.7
Unsloth uploads 22 GGUF quantizations of MiniMax M2.7, ranging from 1-bit (60.7 GB) to BF16 (457 GB).
Apr 123 min readopcnew Editorialwww.reddit.com
MiniMax-M2.7
MiniMax-M1 229B MoE Gets First GGUF Quants for Apple Silicon
MiniMax-M2.7 (229B MoE) quantized to Q3_K_L (110GB) and Q8_0 (243GB) GGUF formats, now on HuggingFace.
Apr 123 min readopcnew Editorialwww.reddit.com
Gemma 4
Gemma 4 Local CUDA Setup: Precision Traps and Real Benchmarks
Running Gemma 4 locally on CUDA requires strict dtype matching at KV cache boundaries or output degenerates silently.
Apr 72 min readopcnew Editorialreddit.com