MoE
27 articles tagged with this topic
llama.cpp Has 50 PRs Pending — Local AI No Longer Needs a High-End GPU
Open-source llama.cpp has 50+ performance PRs pending merge, some claiming 3x CPU inference speedup. Local LLM deployment is shedding its dependence o
Ornith Fine-tunes Beat Qwen Original, Near 27B Full Power on 32GB GPU
Reddit dev tested Qwen3.6-35B-A3B fine-tunes on tool calling for 300+ hrs on 32GB V100. Ornith 1.5 and Tiel-Coder beat original, near Qwen3.8-27B.
Meta Puts a 5.3B-Parameter Model Into 3GB of Phone Memory—For Research Only
Meta releases MobileMoE: 5.3B total parameters, under 1B active, and under 3GB at INT4—but its FAIR NC license bars commercial use.
Three Days Stuck on Local Deepseek — 'Local LLMs' Aren't Just Install-and-Go
Hardware enthusiast spent three days on local Deepseek: 48GB+24GB dual GPUs ran slower than one card. Local LLMs aren't install-and-go.
dots3-note Goes Open Source: 280B Params, 512K Context, AI Note-Taking Bet
dots3-note entered llama.cpp this week: 280B params, 512K context, multimodal, focused on long-context memory and note-style learning.
Ornith 1.5 Shipped with an Untrained MTP Head — Open-Source AI's QC Problem
Ornith 1.5 shipped with an untrained MTP head. Not a bug — a symptom of QA gaps in open-source AI that any cost-cutting enterprise should heed.
开源社区冒出个神秘大模型 — 跑分超 Qwen 顶级版，但没人知道谁训的
A MoE model g9v3-39a5b hit benchmarks this week, outscoring Qwen 27B on some tests. llama.cpp port already live. No company, no launch.
Ornith 1.5 Drops Three Sizes at Once — Indie Devs Now Ship 397B MoE
Developer tarruda dropped Ornith 1.5 on r/LocalLLaMA: 9B dense, 35B MoE, and 397B MoE in one release. The 397B scale is now reachable for indie develo
397B Open-Source Model Targets Claude Opus 4.8 — Is Open Source Catching Up?
Ornith-1.5 397B claims parity with Claude Opus 4.8 on coding and agent benchmarks—open source's first credible reach toward the closed-source frontier
Z.ai's GLM-5.3 Proves Post-Training Trumps Parameter Scaling
Z.ai's GLM-5.3: same parameters as before, just one more month of post-training. The bet: methodology beats scale past a threshold.
Qwen Tells Devs: 'Don't Wait for 35B-A3B' — Open-Source Cadence Shifts
Qwen devs told Reddit 'don't wait for 35B-A3B,' sparking open-source debate. We unpack three possible reasons — and the risk for small teams.
Qwen 35B Pre-Launch Hype Validates China's Open-Source LLM Roadmap
Reddit's local AI community awaits Qwen 3.8 35B A3B — likely a MoE model (35B total / 3B active), extending China's "small but strong" open-source pla
Ling 3.0 Tiny Hits 36 token/s on a 4GB Old PC — The Small-Model Trend Behind One Reddit Post
Reddit user: Ling 3.0 Tiny hits 36 tokens/sec on 4GB VRAM, claims near-Qwen 3.5 9B quality. Discount the post, but the small-model trend is real.
Inclusion AI merges Ling 3.0 into llama.cpp — 1B active params, reasoning model
Inclusion AI's Ling 3.0 lands in llama.cpp. Tiny variant: 1B active params, labeled a reasoning model. Multi-step thinking on consumer hardware.
Qwen3.8 27B Falters on 12GB VRAM: Older MoE Wins
Reddit test on a 12GB VRAM laptop: Qwen3.8 27B's 2-bit version misses classic logic puzzles, 3-bit stutters at 7.5 chars/sec. Older Qwen3.6 MoE wins.
Qwen Pulls a Local-Deploy Favorite — Is Alibaba's Open-Source Cadence Shifting?
Qwen dropped a popular MoE model from GitHub, alarming r/LocalLLaMA. China's top open-weight LLM is moving to on-demand over completeness.
Qwen 3.8 Spotted on GitHub — Alibaba's Open-Source Cadence Outpaces Rivals
Traces of Qwen 3.8 35B-A3B surfaced in Alibaba Tongyi's ms-swift framework on GitHub, hinting at the next open-source drop from a leading model family
1500 Yuan GPU Runs Qwen 30B — Local LLM Hardware Bar Drops Again
Reddit user runs Qwen 30B MoE on RTX 3050 6GB, hitting 30 tokens/sec at 90k context. Enterprise IT should reassess local LLM costs.
280B Model, 16B Active: Open-Source AI Shifts From Total to Active Parameters
dots3-note preview: 280B MoE, 16B active, 512K context, multimodal. Open-source AI race now measures compute cost, not parameter count.
DeepSeek V4 Hits 70 tok/s on a 32GB Laptop — Local LLMs Finally Fit on One Machine
A developer got DeepSeek V4's massive MoE running on a 32GB laptop at 70 tok/s input by resharding weights to SSDs and speculative expert prefetch — p
300B MoE on a 32GB Laptop — Local AI Finally Gets Usable
A developer ran DeepSeek V4's 300B MoE model on a 32GB laptop. The real story: hardware bottlenecks have shifted from compute to SSD read speed.
Tinygrad Tests MoE on Blackwell: Local AI Geeks Build Priciest Hardware Lego
Tinygrad MoE test on Blackwell+M3 Ultra RDMA cluster (~2TB VRAM). A geek experiment—localists stress-test open-source frameworks with radical hardware
Qwen3.6-27B Ties Coder-Next: Pick Models by Scenario, Not Benchmarks
20-hour test: Qwen3.6-27B ties MoE Coder-Next overall but differs by task. Disabling "thinking mode" surprisingly boosts stability. Scenario fit beats
MiniMax-M1 229B MoE Gets First GGUF Quants for Apple Silicon
MiniMax-M2.7 (229B MoE) quantized to Q3_K_L (110GB) and Q8_0 (243GB) GGUF formats, now on HuggingFace.
37 LLMs Benchmarked on MacBook Air M5 32GB: Full Speed Results
Community benchmark of 37 local LLMs on M5 Air 32GB using llama-bench reveals MoE models as clear winners for speed-to-quality ratio.
Running Gemma 4 26B-A4B on vLLM: Community Troubleshooting Notes
Developers report mixed results deploying Gemma 4 26B-A4B on vLLM, with INT4 quants too slow on DGX Spark GB10.
APEX Quantization vs K-Quants: Why MoE Coding Models Need Different Compression
APEX quantization targets MoE architecture coherence layers at Q8, outperforming generic K-quants for multi-file coding agents.