Qwen
RTX 3090 Single Card Doubles Qwen 27B Inference — Local LLM Playbook Rewritten
Solo dev L-Forster's CUDA megakernel lifts Qwen3-27B on a single RTX 3090 from 73 to 140 tok/s (~2x), KL divergence 0.0009. Open-sourced as open-jet.
Back to home
CUDA
10 articles tagged with this topic
NVIDIA
NVIDIA's cuPhoton Brings GPU to the Observatory — Tightening the CUDA Moat
NVIDIA launches cuPhoton, moving imaging for telescopes, lasers, X-ray sources to GPU and extending CUDA into research infrastructure.
NVIDIA
NVIDIA's Green Contexts: One GPU, Many AI Tasks, No Bottlenecks
NVIDIA's Green Contexts fine-tunes AI compute allocation on one GPU. As LLM API prices bottom out, hardware scheduling is the next cost frontier.
llama.cpp
Local AI Is Still a Niche Game — One Help Request Exposes the Real Barrier
A Reddit user ran Qwen on a 4060Ti on Linux, stuck days on CUDA conflicts. One help request exposes whether consumer hardware can run local AI.
PyTorch
Local LLM devs are cursing CUDA OOM — MEM v3 wants to end midnight reboots
This week a Reddit dev shipped MEM v3: real-time GPU VRAM monitor that auto-tunes batch size. Not sexy, but it kills the 3 AM reboot pain every local
NVIDIA
H100 Shortage Isn't About Performance — The Real Moat Is a Decade of CUDA Ecosystem
H100 is AI training's de facto standard. 1979 TFLOPS FP8 per card is just surface — the real moat is CUDA's decade-plus ecosystem shaping next-decade
NVIDIA
NVIDIA turns GPU docs into AI-callable modules — LLMs now compete on real work
NVIDIA's CUDA MCP lets AI search GPU docs and write optimized code. The takeaway: LLMs will compete on real-world work, not just intelligence.
NVIDIA
Nvidia's Real Moat Isn't Faster Chips — It's 20 Years of Software Lock-in
Nvidia's trillion-dollar AI dominance came not from faster chips, but a 20-year software stack since 2006 nobody dares replace.
PyTorch
PyTorch Dominates 80% Dev Desktops—Nvidia Sells the Shovels in LLM Rush
PyTorch is the AI standard, but software unification exposes CUDA's hardware monopoly. LLM bottlenecks shifted from framework wars to GPU compute and
Gemma 4
Gemma 4 Local CUDA Setup: Precision Traps and Real Benchmarks
Running Gemma 4 locally on CUDA requires strict dtype matching at KV cache boundaries or output degenerates silently.
Apr 72 min readopcnew Editorialreddit.com