Open ETF Agent v0.5 did not use “passing tests” to claim that full long-term memory had been implemented. Instead, it confined its conclusion to three explicit operations: saving, revising, and forgetting. This restraint deserves more attention from technical teams than another GitHub link.
What this is
This article examines an “evidence contract”: every technical claim in an article must map to a specific version, a runnable entry point, and a verification result. Open ETF Agent is an agent project for analyzing exchange-traded funds (ETFs)—index or asset-pooled funds traded on an exchange. Its Memory demo checks memory writing, recall, and forgetting separately.
The key practice is retaining both a pinned Commit—an immutable snapshot of the code—and the current main branch, which is updated continuously. The former answers which code the article relies on; the latter tells readers where to begin now. Tests can establish only limited behavior. For example, they can show that old information no longer enters subsequent inputs after it is forgotten, but cannot also prove that a model accurately recognizes every preference worth saving.
Industry view
We believe this approach addresses a central gap in how AI engineering is communicated: a successful demo, an existing repository, and a working product capability are three different things. For enterprise procurement, “what exactly does the run prove?” will also become an important criterion for evaluating vendors.
The risks are equally clear. An evidence chain adds maintenance overhead: when the relevant code, dependencies, or inputs change, older verification results may become invalid. Reproducing a pinned version may also sacrifice access to the latest features. More fundamentally, automated checks work well for verifying explicit state changes but struggle with open-ended judgments such as “Does the model understand correctly?” or “Is the summary reliable?” Narrowing the scope is honest, but it may also make a project appear less capable, while strict version management can slow iteration.
Impact on regular people
For enterprise IT: When evaluating AI tools, do not rely on demos and claims that “tests passed.” Ask which version was tested, under which input conditions, and what artifacts were produced. Procurement and acceptance testing should also record the runtime environment so that old conclusions are not reused after an update.
For professionals: When reading tutorials and evaluations, a pinned code snapshot is usually more reliable than copying from main directly. When encountering claims about “automatic learning” or “long-term memory,” verify whether the current version actually implements them rather than inferring support from adjacent features.
For the consumer market: When an app says it remembers user preferences, the crucial question is whether it actually stops invoking old data after deletion. For regular users, verifiable commitments of this kind bring privacy and trust closer to real improvement than slogans about intelligence.