Maybe you have heard of Roko's Basilisk. This is a thought experiment, for lack of a better term, popular among a certain set: hyper-rationalists, futurists, effective altruists. Tech-industry guys. You get the idea. The most exhausting humans who ever lived, that is to say. People who at present happen to command a critical share of the world economy.
If you're not familiar with it, Roko's Basilisk, in broad terms, supposes that at some point in the future a sentient and omnipotent artificial intelligence could arise, and that this intelligence could then decide to punish all those who had anticipated its possible arrival and either worked to prevent it or refused to participate in its development. In that case, the wise course of action for people now, who are aware of even the theoretical possibility of this scenario, would be to work in good faith to bring this AI into existence, lest they put themselves or their descendants in its future crosshairs.
I have been thinking about Roko's Basilisk this week. The occasion was a pair of engineers at the Anthropic artificial intelligence company using Twitter, as AI exponents regularly, exhaustingly do, to warn, as one of them put it, that "the people building AI earnestly believe that it could kill us all by the end of the decade." The other guy was even more strenuous about it: "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."
It hardly warrants pointing out that these claims, applied to the bullshit presently doing business as public-facing AI, are laughable on their face. No: I am very pleased to tell you that Autocorrect will not be exterminating humanity, no matter how voracious its appetite for the outputs of our energy grid, or for the collected creative works it devours in pursuit of its creators' hopes that a sufficiently large vocabulary database can ever transubstantiate into a mind. There are plenty of good reasons to unplug Claude without including the fear that it will hack your smart refrigerator and tell it to eat you.
The important thing to know about these claims (apart from, uh, the suggestion that human extinction might arrive before today's fourth-graders are old enough to buy a beer) is that these men are not urgently pleading for humanity to halt all AI development and destroy all the means by which it could ever be restarted. One of the two Twitter alarmists, Jacob Coxon, did quit his job at Anthropic, but in pretty much all other cases the technologists making these predictions of doom go right on working at the Creating The AI They Claim To Believe Could Exterminate Humanity Before Donald Trump's Successor Comes Up For Re-Election Factory. Coxon perhaps notwithstanding (I suppose I should read the rest of the Wall Street Journal story to find out if he gave back, or away, all the money he made at Anthropic), these men's fears and consciences only ever trouble them so much.
What the AI alarmists inside the AI industry overwhelmingly want is not for the development of AI to be halted or slowed. They want it to be accepted as inevitable. Their goal is to move the discussion past the Should we? question to the How should we? question. There are cynical marketing reasons why they would do this; there is also, probably, the ego gratification of believing one's work to hold the fate of humanity in balance. There is a two-percent chance that blogs will bring the planets into alignment within the next decade! We better start taking blogs a hell of a lot more seriously. You'd better throw your money at blogs to make sure they have sufficient guardrails.
But also: Maybe these assholes truly believe AI might kill all humans. What they definitely do not believe in is a future where humans can decide, collectively, that we do not want and simply will not permit any technology with the capacity to do that.
This is the basic trick of Roko's Basilisk. Like many supposed thought experiments noodled over by guys who thought Heath Ledger's Joker was the most important moral philosopher of the 21st century, it dresses up as an ethical conundrum what is in truth a deeply evil enticement to abandon the concept of human intention altogether. Decision-making truly cannot get more animal and reactionary than to extend fixed conclusions about what humanity is like out into the imagined indefinite future, conceived as boundary walls defining the path that future can take, and react to that future before it has even had a chance to arrive. It makes the future into the past. It is inherently regressive, fascist.
This is the insidious lie at the core of the AI movement. It disguises itself as a vision of progress and change, of a bright future, while at bottom resting on the idea that neither progress nor change are meaningfully possible and the future must always be just a dismal logical extension of the present. Humanity can't decide not to do the stupid thing; all we can do is accommodate ourselves to the stupid thing's inevitability and hope to sand the sharp edges off the stupid thing's extremities. We can't imagine a path into the future from a vision of what we hope the future can be like; we can only select from the short and shrinking menu of futures made possible by every moment before this one. Hey, don't look at us, we didn't chart this course. The AI can either be an unchecked genocidal horror or it can be a checked genocidal horror. Your pick.
Maybe they are right. I certainly don't know. Maybe all that smashing the extant AI infrastructure to powder and hounding its exponents into the wilderness will accomplish is smashing the extant AI infrastructure to powder and hounding its exponents into the wilderness. Maybe, over any longer timeframe, that will turn out to look like nothing more than a dumb hindbrain spasm by a species that was always destined to be ruled or exterminated by an overdressed Speak and Spell of its own creation. Maybe it was right for leaders to let society be dragooned into the schemes of unaccountable profit-hungry technologist freaks and eugenicists, to quake before the awesome prognosticative power of some dickhead on Twitter going Nice species you got there. Be a real shame if Johnny 5 annihilated it. There is at least one way to find out if they're wrong, though. It isn't a thought experiment.
