The same Zhipu AI engine shifted from answering based on training memory to calling web search in 14 days. This change reminds us that how brands appear in AI answers is being rewritten, and visibility findings from older models may already be obsolete.
What this is
The article’s author connected each AI engine to its official search tool, allowing the model to query live web pages while generating answers, then used reproducible probes to determine whether it was accessing the web or relying on information memorized during training.
On October 31, Zhipu’s glm-4-flash recommended the iRobot Roomba s9+, launched in 2022, and explicitly stated that its information reflected the training cutoff date. On November 2, the upgraded glm-5.3-flash returned currently available models along with search citations. With just 14 days between the two tests, the author labeled the former the “memory layer” and the latter the “search layer.”
The measurement method is even more worth replicating: every behavioral conclusion should record the model name, verification date, and request content simultaneously. When errors occur, results should also be cross-verified using different implementations. Otherwise, an API error may be mistaken for a model that does not support search, while a code defect may be wrongly attributed to the model.
Industry view
We recognize that this is more than a routine model upgrade. As new-generation models increasingly connect to real-time search, the brand perceptions enterprises built up through training corpora are being influenced by more immediate web content. If AI visibility teams fail to record versions and dates, they may end up measuring a previous-generation model that has already been replaced.
However, the claim that “new models necessarily use the web” cannot yet be treated as an industry-wide conclusion. The current tests cover only two specific Zhipu versions; behavior may vary across vendors, APIs, tool permissions, and regional settings. The article also acknowledges that web-access behavior is not the same as brand recommendation quality: retrieving a current product does not mean the model will recommend it accurately or present it fairly.
Attribution remains another risk. An agent, a program that can autonomously call tools to complete tasks, may encounter an API returning a 400 error but cannot determine which test the error affected. The author ultimately used cross-verification across Python and Java implementations to confirm that the change in model generation, rather than a framework failure, caused the shift. Automated measurement therefore still requires human review of business meaning.
Impact on regular people
For enterprise IT: Model versions, web-access settings, and request content should become traceable records. Otherwise, the same AI visibility report may lose comparability. When procuring a platform or marketing service, buyers should also confirm the test date and the exact model tested.
For individual professionals: When using AI to research products, prices, or policies, users should check the cited sources and publication dates. Answers without live web evidence should not be treated as the latest facts. Day-to-day product evaluation should also retain the habit of verifying information through official channels.
For the consumer market: Brand exposure may be shifting from “what the model learned long ago” toward “what the web says right now.” This will increase the importance of real-time content, credibility, and third-party information in AI recommendations. It is not yet sufficient, however, to prove that search rankings will translate directly into sales.