OpenAI has disclosed a set of figures it would be unlikely to promote: over three months, its Codex-based Performance Factory launched more than 1,700 performance investigations, but only about 15 code pull requests (PRs) written entirely by AI and merged with virtually no modification made it into the codebase.

The important point is that this was not a failed demonstration. It is an automated pipeline already integrated into ChatGPT’s production operations. AI can sharply reduce the time spent searching logs and identifying root causes, but it remains far from independently completing production fixes.

What this is

Performance Factory is OpenAI’s internal system for detecting ChatGPT performance regressions. When a production alert fires, Codex pulls in logs, monitoring metrics, and distributed tracing data to identify possible root causes. A separate, independent session then reviews the diagnosis and generates a proposed code change. Finally, a software engineer reviews, deploys, and verifies the fix against performance metrics.

Its funnel is steep: across more than 1,700 investigations, over 600 reached a root cause, with an accuracy of roughly 70% based on manual sampling. Around 400 produced code proposals, approximately 50 fixes were adopted or used as references, and only about 15 AI-generated patches were merged essentially unchanged—a direct conversion rate below 1%.

Industry view

These numbers support a more restrained conclusion: AI programming has matured first in gathering information and narrowing the problem space, not in independently making architectural decisions. Localized, low-risk issues with clear validation paths—such as repeated serialization or straightforward cache reuse—are better suited to automation.

The risks are equally clear. Fixes in complex systems can introduce dirty data, staleness, and consistency problems. Performance optimization can also merely mask inherent latency introduced by new features. OpenAI did not allow AI to deploy changes automatically; it retained human review and post-deployment metric verification. For enterprises, this means near-term returns are more likely to come from reducing time spent on investigation than from achieving unattended self-healing systems.

Another telling detail is that about 25% of initial diagnoses were rejected by the review session. Having the same model check itself is not reliable verification. Cross-checking through independent context is becoming an important component of production-grade AI workflows.

Impact on regular people

For enterprise IT: AI operations tools are more likely to emerge in the near term as an alert-triage and evidence-gathering layer, helping engineers reach judgment faster. Permissions for core changes should remain internal, with review, canary releases, and rollback mechanisms in place.

For individual careers: Repetitive debugging work will decline for software, operations, and testing professionals, while the importance of architectural judgment, risk assessment, and deployment accountability will rise. The more effective skill set is knowing how to define clear constraints and verify evidence—not merely how to write prompts.

For the consumer market: Services such as ChatGPT may detect some performance fluctuations faster, but users will not noticeably perceive “AI autonomously fixing” anything. The final experience will still depend on how large engineering teams configure automation, review, and accountability.