This week, the canonical-basis project on GitHub surfaced a striking number: using its method to flip large models inside-out, a 500-million-parameter model may have just 11 truly working independent patterns.
What this is
The canonical-basis project offers a set of mathematical tools that rotate the model's internal coordinate system into a standard frame, letting us see inside without losing information. It uses orthogonal matrices (a rotation transformation that preserves original relationships) and singular vectors (tools describing the main directions of data variation) to reorganize weights — output unchanged, structure exposed.
Researchers found three interesting things: "bipolar oscillators" exist inside models — certain dimensions appear in pairs and suppress each other; layers "breathe" across the network, alternately absorbing and filtering information; and there are even "self-defense mechanisms" that automatically erase local perturbations within two or three layers.
The most surprising finding: in a 500-million-parameter model, the effective rank of the correlation matrix (a measure of the number of independent information dimensions) may be as low as 11 — meaning only around 11 independent modes actually drive the model.
Industry view
Supporters see this as progress for interpretability research. Anthropic and DeepMind have been betting on mechanistic interpretability (the approach of breaking models down into understandable functional units) over the past two years, and this toolkit fills a missing piece. It provides a standardized coordinate system that enables apples-to-apples comparison across models like Qwen (Alibaba's open-source model) and Pythia (EleutherAI's research model).
But the criticisms are worth hearing. One pushback: "seeing" doesn't equal "understanding." These geometric structures are products of mathematical transformation, not necessarily "thought modules" in any human-cognitive sense. Another concern: this remains research-stage work. Enterprises care about "will the model err, and why" — not whether there are "oscillators" inside. Other researchers point out this rotation method hasn't been validated at the hundred-billion-parameter scale.
Impact on regular people
For enterprise IT: if these tools mature, the cost of debugging AI could drop. Eventually teams will pinpoint specific internal mechanisms — especially valuable for high-compliance sectors like finance and healthcare.
For individual careers: not useful in the short term, but in the medium-to-long term it may enter office workflows — for example, AI could not only give answers but also tell you "which rules this judgment is based on."
For consumer markets: the direct impact is limited, but it indirectly boosts the credibility of AI products — when vendors can clearly explain "why it answered that way," users become more willing to use AI in high-stakes scenarios.