llama.cpp
3.9GB AI Box Delivers Only 58K-Token Context: Specs Are Not Usable Capacity
A 3.9GB device can run only about 58K tokens of LFM2.5-1.2B's advertised 128K context because KV cache memory remains costly.
1 article tagged with this topic
A 3.9GB device can run only about 58K tokens of LFM2.5-1.2B's advertised 128K context because KV cache memory remains costly.