What this is
We noticed one number: 6.82%. That's how much the Octopus framework—jointly published by vivo and Shanghai Jiao Tong University (SJTU)—leads the prior best method on continual learning tasks for multimodal large models (those handling both images and text).
Large models have an old ailment: they learn new things but lose old skills—the jargon is "catastrophic forgetting." Existing fixes all fall short—storing old data for repeated training racks up storage and privacy costs; adding a separate module per task bloats the model; imposing constraints during training has limited effect.
Octopus takes the third route, but reframes it: instead of constraining "where parameters should go," it constrains "where gradients (the direction of parameter updates) should go." It uses a method called HiFGO—no need to access old data—relying only on "gradients of old parameters on new data" as a proxy to identify which update directions would "step on" prior knowledge, then steering clear of them. On the UCIT benchmark, it outperforms the prior best by 2.14% on average and 6.82% in final performance.
Industry view
Supporters argue this paper hits a real pain point. In on-device AI deployment, privacy regulations are tightening and old data is increasingly inaccessible; cloud storage costs for old data are also rising. If Octopus's path holds up, phones running AI that continually learns user habits without uploading data would have a compliance-ready solution. The vivo team's involvement also signals productization intent—phone makers have a real demand for "local continual learning."
But we note several caveats. First, UCIT is an academic benchmark—still distant from real business scenarios—and the paper shows no results on production-scale data. Second, the claim that "learning new can actually boost old capabilities" (called "positive backward transfer" in the paper) has been contested in continual learning for years and needs independent reproduction. Third, a 2-6% improvement doesn't always translate to user-perceivable differences in production; the added compute overhead could instead strain on-device chips.
Worth noting: this is foundational research accepted at a top venue—at least 2-3 years from productization. Enterprises shouldn't bet on it in the near term. What's worth tracking is whether the "gradient orthogonality" path gets validated and simplified by more teams.
Impact on regular people
For enterprise IT: If future AI on phones and IoT (Internet of Things) devices can continually learn without collecting user history, it's good news for data compliance. But don't rush to plan today—wait for open-source code and independent reproduction.
For individual careers: Continual learning is the consensus direction in AI, but in the near term, products like ChatGPT and ERNIE Bot won't upgrade because of this. The paper's real impact may take about 2 years to trickle down into the tools you use.
For the consumer market: vivo's involvement suggests Android OEMs may market "local learning without uploading data" as a differentiator. If you care deeply about privacy, watch for progress on these features.