llama.cpp
llama.cpp’s Auto Allocation Isn’t Always Optimal: Local AI Still Needs Tuning
A Swift-1.5-Qwen3.8-27B user found llama.cpp left layers on CPU; forcing all layers onto an RTX 3090 raised speed from 60 to 76 tokens/s.
1 article tagged with this topic
A Swift-1.5-Qwen3.8-27B user found llama.cpp left layers on CPU; forcing all layers onto an RTX 3090 raised speed from 60 to 76 tokens/s.