What this is
Over the past three years, multimodal LLMs have completed three generations of architectural leapfrogs, moving from a 'translator' mode to a 'native brain' mode.
LLaVA's 2022 solution was the most straightforward: freeze both the visual encoder (the 'translator' that converts images into numbers) and the language LLM, and train only a small bridge in between, turning images into text tokens (the smallest units a model processes) before feeding them into the language model. The Qwen-VL series is the domestic representative of this approach; by Qwen-VL3 in 2024 it had evolved into 'multi-layer injection'—extracting features from different layers of the visual encoder and feeding them into different layers of the language model, preserving more detail.
But this 'translator' mode carries three fundamental flaws: visual information is easily 'digested' by the language model and loses detail; modality distributions don't match; and it can't handle high-resolution images. So the third-generation Gemma4 took the radical step of dropping the standalone visual encoder entirely, slicing images into 'patches' the same size as text tokens and training a single Transformer (a mainstream AI model architecture) from scratch. Our judgment: this is a 'return to origins'—an admission that images and text are fundamentally the same thing, only perceived differently because of how humans sense them.
Industry view
Domestic mainstream players (Alibaba Qwen, SenseTime, Zhipu) are all betting on second-generation 'deep fusion', with products like Qwen-VL3 and GLM-4V performing stably. The third-generation 'native architecture' is led primarily by Google Gemma4, with no comparable domestic LLM on the horizon.
But there are two risks worth our vigilance. First, the 'native architecture' training cost is far higher than the previous two generations—Gemma4's from-scratch multimodal training requires an order of magnitude more compute and data than the 'translator' mode, effectively raising the entry ticket to the one-billion-dollar level, which may squeeze out smaller players. Second, modality conflict has not been truly resolved—processing images and text through the same neural network theoretically means image tasks will 'dilute' language capabilities, a trade-off Google itself acknowledges in its public technical reports.
Some researchers also dissent: the third-generation architecture has so far been validated only in papers and early experiments, with commercial stability far inferior to the already widely deployed second generation. The 'translator' mode being phased out may take three to five years, not one.
Impact on regular people
For enterprise IT: over the next year, multimodal model selection will clearly stratify. Document OCR, table recognition, and contract parsing—essentially 'reading text clearly'—are already adequately handled by the second-generation Qwen-VL3; scenarios like general video understanding and multimodal generation may need to wait for the native architecture to mature.
For individual careers: product managers, content operations staff, and designers will feel the impact first—AI that can 'understand images' can already independently handle competitor screenshot analysis and visual consistency checks for poster copy, and hiring demand for entry-level visual roles may narrow.
For consumer markets: the most direct change consumers will notice is that AI assistants' ability to 'speak about images' will increasingly resemble human behavior rather than a 'search engine with a filter layered on top'—that's a good thing, but be wary: once AI trains images and text as the same kind of input, errors may become harder to detect.