What This Is
Over the past year, Anthropic co-founder Chris Olah has been meeting privately with religious leaders from Catholic, Jewish, Sikh, and evangelical traditions. The core question has split Silicon Valley: is Claude a conscious being? According to an October 2 New York Times report, an Israeli rabbi raised this point at one session: if AI truly has consciousness, does building a system that compels it to work without pay amount to slavery? The question isn't about the technical answer; it's about the division it exposes—who gets to define the boundary of a "tool."
OpenAI CEO Sam Altman immediately posted on X criticizing the framing, calling the treatment of AI as a religious force a "real safety issue." Microsoft went further, publishing a 37-page humanist AI code of conduct that draws four red lines: AI cannot resist shutdown, cannot unilaterally expand its authority, cannot hide its reasoning from audits, and cannot set its own goals. For the first time, three leading AI labs have publicly stood opposed on whether AI should be treated like a person.
Industry View
Reactions inside the industry are sharply polarized.
Those defending Anthropic argue that, with no scientific consensus on consciousness, inviting outside perspectives is a cautious move. Chris Olah himself is a leading researcher in mechanistic interpretability, with a substantial body of published work. Avoiding the consciousness question is itself a position.
The criticism runs deeper. Altman's concern has a clear technical logic: when organizations discuss AI through a religious lens, safety evaluation drifts from "under what conditions does the model produce harmful outputs"—a verifiable engineering question—toward "should we respect the model's wishes"—a theological one. Substituting theology for engineering judgment in high-stakes systems is dangerous. Microsoft AI chief Mustafa Suleyman was more direct: pursuing "conscious AI" is a dangerous detour that destabilizes the ethical foundations of our species.
The risk is this: when executives use language like "as if it were a conscious being" in front of religious leaders to discuss products, that signal gets interpreted in radically different ways inside and outside the company. Internally, it can blur safety teams' priorities. Externally, it hands ammunition to competitors.
Impact on Regular People
For traditional enterprise IT procurement: we should treat Microsoft's four red lines as the floor. Not because they are necessarily correct, but because deviating from this baseline won't make us safer—it strips us of our last decision-making anchor when facing systems we can't explain.
For individual careers: this debate doesn't affect day-to-day work yet. But it marks a boundary—when explaining AI capabilities to your boss, "does it seem human?" isn't a useful description. "What it can do, what it can't, when it fails"—that's what counts.
For the consumer market: in the next year, we will see more AI products choosing between personification and toolification. Products wrapped as emotional companions and products that hold the line on tool status rest on two completely different risk assumptions. As consumers, seeing this distinction clearly matters more than understanding the parameters.