You're a streetcar operator. If you need more specifics, let's say it's fin de siècle Vienna and you've got a jaunty cap.
Oh no! Your streetcar is barreling down on a group of people. But you observe that they lie just beyond a junction in the track whose switch you control. If you pull the lever, your streetcar will move to the other track—and safety!
... or will it??? There's actually a single hapless civilian over there. Perhaps it's a toddler in a baby carriage whose wheel is stuck in the track; maybe it's an elderly person whose walker has likewise been immobilized. They couldn't have known that a streetcar would be on it. If you pull the switch, you'll obliterate them. But if you don't pull the switch, you'll kill the hapless civilians directly in front of you. What do you do? Your jaunty cap has become a thinking cap.
The trolley problem, as it's known, is a classic philosophical puzzle. It is intended to reveal that action and inaction are both morally consequential acts, but to me it also serves as a kind of incompleteness theorem for utilitarianism. No matter what you think you know about the situation, the decision is fraught. Does it matter if the person on the other track is young or old? Ill or well? What about the group directly in front of you? What if it's one person? Two? Ten? What if they're Nazis? What if they're artists? What if they're currently Nazis who have a change of heart after they've been spared? Or militant leftist activists who abruptly become Nazis when they realize they've been subjected to a cruel trolley problem experiment?
Among the many valid takeaways (and please go on and read other exegeses of the trolley problem, there are many), what I'm trying to highlight here is the uncertainty: You'll never actually be able to know every counterfactual from every branch of every decision you didn't make, so there won't ever be a concrete right or wrong answer in a utilitarian framework. Instead, we deal with the trolley problem by a whole host of intellectual and moral supports that live outside of utility and beyond the moment of decision. This is nothing new. Social contract theory dates to prehistory.
Artificial intelligence breaks the social contract in a way that is invisible to a utilitarian. Think about the trolley problem again, but with a self-driving trolley. It feels very different: The first time, you put yourself in the trolley driver's shoes. You thought about how it would feel to have to make the decision. You understood the imperfection of the setup and rued the unjustness of the scenario the driver was in. You may have thought, as I do: Well, in the heat of the moment, you're going to have to make the decision. Sometimes it'll go one way, and sometimes another; overall you just hope it balances out.
Something big changes when there's a preprogrammed, artificial intelligence behind the wheel. All I can imagine is that scene in the movie adaptation of Leave The World Behind, in which a family is almost killed by a bunch of Teslas crashing into each other. Even if we don't love it, we tolerate humans rolling through stop signs, but we can't do the same for electric vehicles. The tendencies of a self-driving car—or its biases or errors or flaws or bugs or maybe just the incomplete imaginations of its programmers—will be relentlessly manifested, and thereby amplified. There will be no balancing of scales, no helpful case law, no endless and therefore human ruminations on the real meaning of the dilemma. Unless we can stop them before they get going, they'll just keep crashing into the same damn person over and over. The important question to ask is not whether we should stop them, but by what means.
Utilitarianism is a simplistic way to talk about decision-making and morals, because utilitarianism is a simplistic way to think about decision-making and morals. But I think that's what we're actually doing when we talk about artificial intelligence in terms of capability. When we emphasize how much it sucks now (even though that's clearly true), we have accepted a framing that makes it harder to reject once it doesn't suck.
Instead, I think the time has come to reject an ability-based framing altogether. There are so many reasons for this; I'll mention two. First, the premise of an objective quantity that we might call general ability is flawed and incomplete and not the basis of how we order society now. We understand and accept that intelligence is widely distributed and oriented. Nevertheless, on the one end of the spectrum, people with developmental delays or intellectual disabilities are still recognized as humans with rights and dignity. And on the other end of the spectrum, we value all kinds of expertise and intelligence. We're not all aimed at the same goal, nor should we be.
This is one way to expose the facile void at the heart of "effective altruism." To an effective altruist, the world is very simple and the path to the better one is easy to find: Simply perform better measurements of everyone, and optimize all of their circumstances for whatever function is en vogue on the message board that morning using whatever technology gets them there fastest. But we do not establish civil institutions for reasons as trivial as optimizing for single-valued functions, because there is no single "better/worse gauge" when it comes to participating in society. Aptitude-based arguments are traps when we think about intelligence generally and AI specifically.
Another reason to reject an aptitude-based test for AI is that any metric will be at best incomplete and at worst intentionally biased or limited (in other words, cooked). There is no sense in which we can fully test the entire suite of circumstances an AI agent will encounter without actually deploying it without limitation. So we would wind up with systems designed to pass tests and not actually be safe in practical, demonstrable, trustworthy ways. This is a very real problem, as AI hallucinations demonstrate: There is no sense in which we could ever control for all the butterfly-effect conditions that could lead to the AI car "deciding" (if we even have access to an interpretable window into its thinking) "Ah, fuck it, I wanna hear this engine sing, I'm going to mash the accelerator now" when it sees a stop sign.
Yet this potentially obscures some meta-trolley problems. For example, human motorists kill lots of people. Do we ignore this cost for the sake of maintaining a status quo that doesn't really seem all that ideal? Self-driving cars don't get drunk, or high, or fall asleep behind the wheel. What is the cost of our inaction and discomfort? Plus, as it stands, preprogrammed intelligence is entering the car already, as demonstrated by lane assist and all the other beeps and boops new cars make. How much should we limit this, and why? That's not a rhetorical question, and I don't think it matters what I think the answer is or where I draw the line. But I do think there are right and wrong ways to talk about it, because someday soon someone (not me) will confront the following question: What do I actually code into the (very real, very deterministic) algorithm for situations in which a self-driving car detects likely collisions ensuing from its assessment of its current state and all of its possible actions?
The core idea I want to propose is that liability is a better way to discuss (and, eventually, make decisions about) AI than ability or utility.
Accepting the premise that humans are imperfect drivers, but also that we don't want unpredictable psychobots unified under the command of an unaccountable tech bro, what do we really want out of a solution to the self-driving car problem? Should we reify inaction in all scenarios, so that as long as the car stays the course when something's going wrong, it isn't at fault? Or do we try to make an assessment of the costs of the different collision outcomes? What happens when it "chooses" "wrong"? Who gets to say what's right or wrong? Who is responsible if a decision is determined to be wrong? The engineer? Which engineer? What are culpability and liability in this scenario?
These questions are only a starting point, and will eventually need to be asked in sectors where AI's presence is not yet as obvious or immediately controversial as it is behind the wheel. It’s tempting to dismiss the slop and the crud that we see all around us from AI, because a lot of it's facially dumb, and that is legitimately infuriating. And it's easy to get mad at, because its externalities harm us all. But the era of its usefulness, however slight, is here. And as the implications of that ripple outward, we will need a more stable and coherent way to discuss limiting its uses, if we want those limits to exist at all.
I don't think I'm being a wild-eyed accelerationist to say that parallels to the self-driving car debate will need to be had concerning schools and healthcare, to name two areas with lots of active investment, and from there to wider segments of society. Even if you believe, as I do, that much of the current activity is nothing more legible than the frenzy of an economic bubble, the core developments that we're seeing now are substantive enough to persist, in much the same way as the internet stuck around after the dot-com crash. To say it another way, AI is not VR.
Precisely because we will be living with increasing amounts of AI in our lives, I think we have to insist that liability exists somewhere in these settings, and figure out where to place it, because without a concept of liability, the social contract falls apart. And the questions will only get more complex from here, not less. When a chatbot encourages a user's destructive behavior, who should bear responsibility? If an AI MRI technician misses a patient's cancer diagnosis, where does the patient go for restitution?
The answers to these questions cannot be found within debates about what AI can and cannot do, but in conversations about what it should and should not do. Pointing out all of AI's flaws and missteps has not yet proven to be a reliable method for slowing its development and proliferation, and as a result we have passed the threshold for ignoring the problem away. As we shift to thinking about the ramifications of that, we might have better luck by placing real, visible consequences at the end of any roads an AI lab wishes to travel down. The quicker we can shift our focus away from AI's ability (or lack thereof) and onto questions of liability, legislation, and regulation, the more prepared we will be for an uncertain future.
