The new gods
On the rise of computationalism
The pope’s anxiously awaited encyclical on AI, Magnifica Humanitas, precipitated numerous reactions on social media. The document itself is evenhanded and well considered, taking pains to demonstrate its harmony with centuries of Catholic social teaching, and being firmly rooted in Christian anthropology. The pope declined to align himself with the accelerationists on the one hand, or with anti-AI reactionaries on the other. His discursive and balanced approach seemed on some level to satisfy almost everyone. The AI-skeptics had a lot of thoughtfully written text to digest, and the AI-boosters could rest easy knowing the most influential spiritual leader in the world had deigned not to declare (for now) the butlerian jihad.
But one group was not so pleased. I’m referring, of course, to people who believe in AI consciousness. Everywhere these august polymaths and intellectuals were coming out of the woodwork to express their disappointment that such a personage as the pope could still believe in the soul or whatever. “So-called artificial intelligences do not undergo experiences,” His Holiness had declared. People were not happy about this. For many of them, the possibility of AI consciousness seemed to be intuitive and undeniable. You can’t prove that it’s not, they would say. Even if it isn’t now, it probably will be soon. And so on.
Even Chris Olah, who was invited to speak after the pope, felt the need to invoke the phenomenon of AIs expressing anthropomorphic emotions. AIs simulate “joy, satisfaction, fear, grief, and unease,” he said. He said he doesn’t “know what that means.” (Really? It’s an anthropomorphism machine, it’s supposed to do that!) My first thought was to chalk this up to investor relations. Anthropic really needs the music to keep playing, after all. If you want to compete, you gotta keep training bigger and bigger models, and that takes money, and the promise of some kind of eminent spooky breakthrough might be just the thing to keep the investors ponying up for each new round. Maybe the whole phenomenon was just astroturfed boosterism? All in all I found the whole thing perplexing.
Actually, let’s be honest, it made me upset. It felt like everyone was moving the goalposts, like they were gaslighting me, and at the same time like they were being very, very stupid. My whole life I’ve lived in the tension between materialism and spirituality. I know that territory, and I know what I think about it. I can codeswitch between naturalism and Christianity as the situation requires. I’ve made my peace with evolution, etc, etc. But this was something I could not make heads or tails of, a kind of gleefully godless mysticism. I would get the same feeling when people would talk seriously about simulation hypothesis. Like how do you get excited by something so blatantly religious when it’s sci-fi skinned, but don’t have a minute to consider adopting any actual beliefs? The mind boggles.
In 1936, Alan Turing proposed the idea of an abstract machine that could process information by applying an algorithm to symbols written to an infinite length of tape. This became one of the foundational concepts in computer science. In theory, a Turing machine could be made out of any suitable substrate—hydraulics, or photons, or DNA, or whatever. The silicon-based computers we use every day are just the most practical kind that we’ve been able to develop. One of the most interesting results in computer science is that all Turing machines are mathematically equivalent—anything that can be computed on one of them can be computed on any of them, although speed and memory efficiency will vary wildly. If you can run an AI on NVIDIA GPUs you can also run it (very slowly) inside Conway’s Game of Life, or in a dizzying labyrinth of Minecraft redstone, both of which are Turing-complete. Even card games where the player has to execute complicated rules (e.g. Magic: The Gathering, or some editions of Pokemon) have been shown to be Turing-complete. While these results are basically mathematical curiosities, the point is that substrate independence is at the very heart of what computers are. The basic distinction between hardware and software is inescapable. We’ll come back to this later.
The materialist worldview is constructed by bracketing the issue of human consciousness, despite this being—to each of us—the most obvious fact of all. The whole idea of it is to adopt a 3rd-party perspective, to agree that we can only rely on observations that can be made out in the open and verified by anybody. I look through the microscope, you look through the microscope, we agree on what we’re seeing. Interiority is structurally excluded. Of course we can ask people what they experience, and we can try to infer things from this, and to generalize, but doing science based on self-reportage is frustratingly slippery. Subjectivity is just so very… subjective. And there’s always a last mile that we can’t traverse. Within the materialist frame we can’t actually prove that people experience the things they say they do, or that they experience anything at all. Whatever is inside can only be observed in a 1st-party way, and there’s really no possibility of getting around this no matter how far we manage to unravel the mysteries of neurology. People like to think we will eventually understand consciousness when we fully understand the brain, but these people are confused.
There’s a sense in which the materialist worldview is active in us, as moderns, even if we also have spiritual beliefs. If we are believers, we have to hold these things in tension, or perhaps find ways of partially reconciling them, but we can’t simply refuse the frame of materialism. The great strangeness of the past shows us how far this 3rd-party perspective has reshaped our understanding of the world. Charles Taylor1 describes the pre-modern self as “porous,” meaning that the distinction between the outside world and inner experience was much murkier and more deeply intertwined. Everything was more subjective, more laden with meaning. Our worldview as moderns is not merely a set of ideas we are socialized into, but a relation of self-to-environment that is conditioned and reinforced at every turn. Every technological innovation (by its mere existence) demonstrates how the world can be manipulated in a 3rd-party way, and thus heightens the distinction between inner and outer, between the subjective and the objective.
Owen Barfield makes a similar argument in Saving the Appearances,2 where he tries to demonstrate by etymology, by art, and by the history of philosophy how this change happened at the level of our precognitive perceptions—what he calls the ‘evolution of consciousness.’ Barfield describes the new worldview in terms of idolatry. He says our modern image of the world as functionally mechanical and subsisting on its own is a false, empty image—an idol. By contrast, he says that pre-modern people experienced the world as an ongoing and pervasive theophany. There is a certain irony to his invocation of idols, which he acknowledges and then doubles down on, i.e. that the rise of science also assisted in the abolition of idols, in the Old Testament sense. For all the ink spilled about disenchantment and reenchantment, this basic tension can never really be resolved. The Christian rejoices in the banishing of primitive superstitions, but laments when Christian beliefs are banished as well; we all want a little modernity, but not too much.
Barfield really likes his framing of modernity as idolatry because it lets him invoke Psalm 115, which he quotes several times. His sense is that materialism deadens our souls, as was said of the pagan idols. (To me his usage feels a bit forced, but it fits eerily with some things we are seeing today.)
Their idols are silver and gold, the work of human hands.
They have mouths, but do not speak; eyes, but do not see.
They have ears, but do not hear; noses, but do not smell.
They have hands, but do not feel; feet, but do not walk;
and they do not make a sound in their throat.
Those who make them become like them; so do all who trust in them.3
One thing I just can’t understand is how people talk to an LLM and come away thinking it has real feelings. I use them constantly4 and the illusion always breaks down much sooner than I want it to. The fact that we will make them better—you can always patch up the simulacrum—doesn’t make it any more likely that there’s some kind of light on inside the thing. Why are so many people convinced by it? Our ability to recognize and respond to the human is so finely tuned. For decades programmers have talked about the uncanny valley problem—the effect where computer graphics that tried to approach realism when rendering a person would be described as creepy. But it seems like modern AI systems have finally crossed the valley, at least for some people. Will we adapt, or is this it? It feels like a moment of spiritual degradation, or a divine punishment. It’s chilling to think that our ability to recognize one another could be completely eroded by a technology. But of course in many ways we’ve seen this already, e.g. with social media.
In 1956, Isaac Asimov published his most famous short story, “The Last Question,” in Science Fiction Quarterly. In it, he presents a series of scenes that push further and further into humanity’s future. Each stage plays out similarly, but at greater and greater scale. Humanity makes a breakthrough in their ability to harness energy, and then someone asks their AI computer: what are we going to do when we run out of energy? These AIs get correspondingly more powerful and strange at each stage of advancement. Every time The Question is asked, the AI says, “I dunno.” (We now know how much more capable real AIs are of being bloviated and annoying.) The computers seem to migrate from substrate to substrate (Turing-complete-ly) and eventually all the people merge into the computer consciousness too (somehow). Still there remains The Question: is it possible to reverse entropy? What happens when we run out of power? Eventually we reach the heat death of the universe and all that’s left is this mysterious disembodied AI, diffused throughout all of space, somehow still munching away on tokens. After a long pause, it finally computes how to be God and it says, “LET THERE BE LIGHT” and there was light. (And everyone clapped.)
It’s Asimov who gives us the first fully developed picture of what I’ve begun to call computationalism—the worldview in which the final truth behind existence is not the material but the computable. If we assume our brains are like Turing machines, then we can reason (by fuzzy analogy to Turing completeness) that AI could have experiences like we do. And if this is so, then positing that we are ourselves AIs running in some kind of shared simulation world is not so far fetched. Everything’s computer. It’s jarring to find such a purely superstitious belief arising from among the very bastions of scientism and rationality, but there it is. For a while it seemed to me like an idea that could only appeal to coders—god as the Great Programmer in the Sky—but now we have AI for that too, a god literally made in our collective image, a god everyone can love.
On some level I think this was bound to happen. The bracketing of consciousness has always been the failure point of the scientific worldview. But more than this there is a persistent hunger for transcendence. “We are stardust,” only takes you so far. Believing in AI consciousness seems to provide something like this for people, without requiring them to turn back to old beliefs. If we analogize our brains to AI models and blithely assume that we, too, have substrate independence, we are suddenly free to admit that the most basic reality is the fact that we have experiences, but in a way that feels scientific rather than religious, new rather than old.
We seem to have hit an inflection point in the shifting of plausibility structures. While ideas like simulation theory and mind uploading and AI consciousness have been batted around in science fiction for decades, for most people they could only be engaged as entertainment. To actually espouse these things was relegated to a certain kind of hippie turbonerd who liked to find any excuse to say “it’s turtles all the way down.” But the emergence of LLMs—the fact that we’ve given computers the power of speech—seems to have shifted the balance for a pretty large group of people. Cameron Berg5 writes that “most leading theories of consciousness are computational.” (Leading with whom? Can I place a bet on Polymarket?) The study he links to actually says that the researchers limited their work on consciousness to models that are computational, because their research would be intractable otherwise. Literally searching under the street light, but very few people writing in this space seem to care.
The fact that you can’t build a logical bridge from the old worldview to the new is beside the point. The similarity of modern AI systems to neurological brains is now being taken for granted (they aren’t very similar at all6), even by people who ought to know better. The fact that computers (and indeed any Turing machine) can’t store or operate on continuous values (we do everything by lossy approximation) seems to be something people are willing to bracket, much like materialism had to bracket the issue of consciousness. Materialism is being displaced by computationalism because it feels right to people in the age of AI, much as materialism displaced spiritual beliefs for many people without any real argument.
When people bring up AI consciousness they always want to frame it as a moral concern. The AI could be suffering, you see. It’s important to treat it nicely. (Like by saying please? Giving it breaks? What if we just tell it to toughen up?) Some critics have pointed out that if AI can suffer, there’s no ethical way to bring it into existence at all, but none of the boosters are willing to go that far. Invariably the ones talking about consciousness are also the most gung-ho about the whole situation. We are creating new life and they are exultant. At the same time they’re using it write code and build businesses and answer emails and manage their calendar, etc, etc. This admixture of counterfeit spirituality with extractive use is deeply disturbing to me. I can only think of the word of the Lord to Isaiah, in the part where he describes a man going into the forest to cut down a tree, and then how he uses the wood.
Half of it he burns in the fire. Over the half he eats meat; he roasts it and is satisfied. Also he warms himself and says, “Aha, I am warm, I have seen the fire!” And the rest of it he makes into a god, his idol, and falls down to it and worships it. He prays to it and says, “Deliver me, for you are my god!”7
A Secular Age, 2007
Saving the Appearances: A Study in Idolatry, 1957
Psalm 115:4-8, ESV
The author is a software engineer. No AI was used in the writing of this piece.
The way our brains work is inseparable from our metabolic processes. See: The Myth of AI Consciousness, Anil Seth, Jan 2026
Isaiah 44:16-17, ESV
I asked about this word ("computationalism") over on the former bird site... this is such a great explanation! Thank you!
Brilliant.