What Leaders Get Wrong About AI Consciousness

Anil Seth, a neuroscientist at the University of Sussex and author of Being You, has written a long essay arguing that consciousness probably requires biology rather than just computation. We’ve translated his argument into implications for leadership development and organizational design. Here’s what executives need to know, and what they shouldn’t overclaim from this work.

First, some honest framing. This isn’t empirical research with sample sizes and p-values. It’s a philosophical argument that synthesizes neuroscience, computer science, and the philosophy of mind. Seth builds his case through reasoning about the nature of the brain, computation, and consciousness rather than through controlled experiments. That doesn’t make it less valuable for our purposes. It means we should treat it as a well-argued perspective from a credible source, not as a settled scientific consensus. Seth himself acknowledges that biological naturalism—the view that consciousness requires life—remains a minority position among consciousness researchers. His credentials matter for assessing the argument’s weight: he runs a major consciousness research lab, has published extensively in peer-reviewed journals, and engages seriously with counterarguments. He’s not selling a product or defending an investment thesis.

The argument

Seth makes four interconnected claims. First, brains aren’t computers. The brain-as-computer metaphor has been productive for AI research, but it obscures an important point: there’s no clean separation between “software” and “hardware” in biological systems. A neuron isn’t just processing signals; it’s maintaining its own existence, metabolizing, and repairing itself. You can’t swap it out for a silicon equivalent the way you’d swap a graphics card. What brains do can’t be separated from what brains are.

Second, other computational approaches exist beyond Turing-style algorithms. Biological systems involve continuous dynamics, randomness, and processes embedded in physical time. An algorithm doesn’t care if there’s a microsecond or a million years between state transitions. Biological systems can’t escape time that way.

Third, life probably matters. Every system we confidently identify as conscious is also alive. Seth connects this to his research on predictive processing: the brain constantly generates predictions about what’s happening, calibrated by sensory feedback. For predictions about the body’s internal state, this process reaches down into metabolism itself. Conscious experience of emotion, mood, and the basic feeling of being alive may be inseparable from being a self-maintaining biological system.

Fourth, simulation isn’t instantiation. A simulation of the digestive system doesn’t actually digest anything. A simulation of the weather doesn’t produce rain. Unless consciousness is purely computational, simulating brain activity won’t produce consciousness any more than simulating fire produces heat.

Thanks for reading Lead from the Front! This post is public; please feel free to share the knowledge.

What this validates

This argument reinforces several positions we’ve been developing. If consciousness itself can’t be captured by computation, then certainly the relational dynamics that depend on consciousness can’t be either. The push toward “AI relationship management” misses something fundamental about what relationships are. Two people in genuine dialogue aren’t exchanging data packets. They’re two living systems co-regulating and predicting each other, updating their predictions continuously.

The implementation gap we talk about—the difference between knowing what to do and doing it under pressure—turns out to be biological. Seth’s framework explains why the brain, under stress, reverts to embedded patterns: not because of bad algorithms, but because stress impairs the prefrontal structures that override automatic responses. This isn’t a software bug to be patched. It’s how biological systems work. Our 18-24-month program structure reflects the time required for behavioral automaticity, and Seth’s emphasis on the brain as a continuously self-maintaining system, embedded in physical time, provides theoretical grounding. Habits form through biological processes that can’t be accelerated past certain thresholds, any more than you can make a broken bone heal in a week through better project management.

The complication

One honest complication: we use AI tools, and so do the leaders we work with. Seth’s argument doesn’t say AI is useless. It says AI isn’t conscious and probably can’t become conscious through current architectures. That’s compatible with AI being useful for many tasks.

But his analysis of “conscious-seeming” AI creates a practical problem. He argues we might be unable to resist the feeling that AI is conscious, even when we know it isn’t, the same way visual illusions persist even when we understand them intellectually. We can’t just tell people to “remember it’s not conscious.” The pull toward anthropomorphism operates below rational override. If you’ve ever caught yourself saying “thank you” to a chatbot, you’ve experienced this. And treating conscious-seeming things as non-conscious may be psychologically unhealthy—Seth cites Kant on this point. There’s something corrosive about habitually interacting with something that seems to have feelings while treating it as if it doesn’t. The psychological cost lands on us, not the machine.

This challenges how we think about AI-augmented leadership development. Should coaching involve AI practice partners? If leaders spend hours talking with systems that seem empathetic, does that build capacity for human empathy or erode it? We don’t have confident answers yet.

What to do differently

What changes in practice? For individual leaders: treat AI tools as tools. Use them for research, drafting, analysis, and pattern-matching. Don’t use them as substitutes for human relationship practice. The micro-habits we recommend—three-second pause before responding, daily authentic check-ins—require practicing with actual humans who have actual stakes. Simulated conversations may be worse than no practice at all if they train leaders to interact with entities whose responses don’t carry real consequences.

For organizations: establish clear norms around AI anthropomorphism. When employees start attributing feelings to internal AI systems, that’s a warning sign, not a feature. Audit the language your organization uses about AI capabilities. “The AI thinks” or “the AI wants” embeds assumptions that deserve scrutiny.

For program design: preserve human-to-human interaction as the irreducible core. AI can augment research, personalize content delivery, and track metrics. It cannot replace the coach, the peer cohort, or the real-stakes conversation where someone’s reputation is actually on the line. Progressive stress exposure requires real stress, not simulated approximations.

What not to conclude

A few overclaims to avoid. Seth isn’t arguing that AI is useless—AI systems are genuinely intelligent in many functional senses, they just aren’t conscious, and consciousness matters for some things more than others. Biological naturalism is a minority view, and other serious researchers disagree, so we haven’t “solved” the AI consciousness question. The argument that machines don’t suffer doesn’t resolve questions about what interacting with suffering-seeming machines does to us, so we can’t simply ignore AI ethics. And the claim is narrower than “human relationships will always beat AI assistance”—some aspects of relationship, development, and consciousness may require biology, but plenty of tasks don’t.

The real takeaway

Seth’s argument matters less for what it tells us about AI and more for what it reminds us about ourselves. We are not algorithms running on biological hardware. We are living systems, maintaining ourselves through continuous metabolic activity, embedded in time, predicting and regulating in ways that reach down to our cellular foundations.

Leadership development that treats behavior change as software installation will keep failing. The 70% failure rate of change initiatives isn’t a project management problem. It’s a category error about what humans are and how we change. Our work has always emphasized the biological reality of behavior change: the 66 days for habit formation, the stress-induced reversion to automatic patterns, and the necessity of practice under progressively realistic conditions. Seth’s framework doesn’t prove we’re right. It suggests we’re asking the right kind of questions.

The competitive advantage goes to organizations that understand what requires human judgment, relationship, and presence—and what doesn’t. That line may be harder to draw than the AI boosters suggest.

Source: Anil Seth, “The Mythology of Conscious AI,” Noema Magazine, 2025. Seth directs the Sussex Centre for Consciousness Science and is the author of Being You: A New Science of Consciousness (2021).