tokenizer

Qwen Wins at Code, Gemma at Text — Tokenizer Gap Is the Hidden Cause

A Reddit user's test found: the same 330 lines of code became 1,609 tokens in Qwen but 4,258 in Gemma — a 2.6x gap that may explain their very differe