Back to home

Compare

Comparing: Multi-Agent Runs Do Not Prove the Main Model Used the Experts & 多专家 Agent 跑完了,却不等于主模型真用了

AEN
Multi-Agent SystemsAI AgentsContribution Auditing·

Multi-Agent Runs Do Not Prove the Main Model Used the Experts

Across 48 valid runs, 70 of 75 final submissions contained text later found in the main Agent’s input. But this seemingly impressive figure does not prove that the experts contributed to the answer. What the industry actually needs to measure is not “how many experts ran,” but how much judgment value the additional calls and costs bought.

What This Is

The hard part of multi-expert Agent systems—where a main Agent delegates tasks to specialized child Agents and then synthesizes their results—is that participation and contribution are not the same thing. Five questions require separate evidence: whether an expert was authorized to execute, whether it submitted a report, whether its text entered the main Agent’s input, whether the final answer adopted the expert’s judgment, and whether the answer became better as a result.

This article draws on a set of research evaluations from 2026. Across the first 59 runs, 53 included a Reviewer, the child Agent responsible for validation, while 12 included a regular expert. The 48 runs generated 75 readable submissions, and at least one passage from 70 of them was later matched in the main Agent’s input. This only shows that some material entered the context. It cannot be described directly as a “93% expert adoption rate,” much less prove that answer quality improved.

A more reliable method is to trace the chain behind a key conclusion: original submission → actual main Agent input → final claim → original source. Text entering the context does not mean the model changed its judgment based on it. Conversely, a failed text match does not rule out a handoff through a citation, selected passage, or paraphrase.

Industry View

We believe multi-Agent systems are shifting from architecture demonstrations toward auditable operation. Teams used to report the number of experts launched, requests made, and submissions produced. They now need to establish separate denominators for runs, executions, submissions, and model requests, so that one piece of work is not counted as multiple contributions. A single status flag such as “used=true” should likewise be replaced with three types of records: input coverage, claim correspondence, and incremental effect.

The risks are equally clear: finer-grained evidence will expose duplicate delegation, missing materials, omitted constraints, and false contributions such as “the main model had already seen the same source.” Selecting only successful multi-expert cases and comparing them with all ordinary tasks would also misidentify sample-selection bias as an architectural benefit. Only by fixing the questions, sources, timing, model configuration, and total budget—and conducting blind evaluations of a single main model against the multi-expert system—can teams approximate a valid comparison of cost and quality.

Impact on Regular People

For enterprise IT: When procuring multi-Agent services, enterprises should require vendors to present separate evidence of calls, handoffs, adoption, and impact, rather than accepting demonstrations and participation counts alone. Audit records should also reference existing logs instead of creating a separate “contribution ledger” that might alter or rewrite business facts.

For individual professionals: People who use Agent tools need to learn the difference between “the system ran” and “the conclusion was adopted.” When reviewing reports, checking sources, applicable conditions, and key limitations is more valuable than prioritizing generation speed or length.

For the consumer market: Consumer-facing multi-assistant products may increase call costs and waiting times, but they may also deliver more trustworthy answers by preserving constraints. Whether they gain an advantage will ultimately depend on whether the errors users care about decline—not on how many experts were launched behind the scenes.

Source: juejin.cn
BZH
Main AgentChild AgentReviewer·

多专家 Agent 跑完了,却不等于主模型真用了

在 48 次有效运行中,75 份最终提交有 70 份能在主 Agent 后续输入中找到文字片段,但这个看似亮眼的数字不能证明专家贡献了答案。行业真正要核算的,不只是“跑了多少专家”,而是多出的调用与成本换来了多少判断价值。

这是什么

多专家 Agent(让一个主 Agent 分派任务给多个专业子 Agent,再汇总结果的工作方式)的难点,在于“参与”与“贡献”不是一回事。专家获准执行、提交报告、文字进入主 Agent 的输入、最终答案采用专家判断,以及答案因此变得更好,是五个需要分别取证的问题。

文章基于 2026 年一组研究测评:前 59 次运行中,53 次有 Reviewer(负责复核的子 Agent)准入,12 次有普通专家准入;48 次运行产生 75 份可读提交,其中 70 份在后续主 Agent 输入中至少匹配到一个片段。这只能说明部分材料进入了上下文,不能直接称为“93% 专家采纳率”,更不能证明答案质量提高。

更可靠的检查方式,是沿一条关键结论回查链路:原提交 → 主 Agent 实际输入 → 最终主张 → 原始来源。文字进入上下文,不等于模型据此改变判断;片段未匹配,也不排除通过引用、选区或改写完成交接。

行业怎么看

我们认为,多 Agent 正在从架构展示转向可审计性建设。团队过去习惯报告专家启动数、请求数和提交数,现在需要把运行、执行体、提交和模型请求分别设分母,避免把一次工作拆成多份贡献。类似“used=true”的单一状态,也应拆成输入覆盖、主张对应和增量效果三类记录。

风险同样明确:更细的取证会暴露协作中的重复委派、材料丢失、限制条件被省略,以及“主模型其实早已见过相同来源”这类伪贡献。若只挑成功完成的多专家案例,再与全部普通任务比较,还会把样本选择偏差误当成架构收益。固定问题、来源、时点、模型配置和总预算,并进行单主模型与多专家的盲评,才接近成本与质量的有效对照。

对普通人的影响

对企业 IT:采购多 Agent 服务时,企业应要求供应商分别说明调用、交接、采用和效果证据,而不只接受演示案例与参与次数。审计记录也应引用既有日志,避免另建一套可能改写业务事实的“贡献账本”。

对个人职场:会使用 Agent 工具的人,需要学会区分“系统跑过”与“结论被采用”。审阅报告时检查来源、适用条件和关键限制,比看重生成速度和篇幅更有价值。

对消费市场:面向消费者的多助手产品可能增加调用成本与等待时间,也可能因保留限制条件而提升答案可信度。能否形成优势,最终取决于用户关心的错误是否减少,而不是背后启动了几个专家。

Source: juejin.cn