Smoke Signals: the Dangerous Fatalism of AGI in Silicon Valley | Invesloan.com

Last week, Palantir CTO Shyam Sankar mentioned something bizarre in an interview on CBS News. “So many AI researchers have started smoking as a way of signaling to other researchers how much they believe in AI,” he said. Their thinking, he said, goes like this: “I believe so much in AI that it’s going to cure cancer. So I’m showing you my chips are all in.”

I laughed as I read the quote. It read like a headline in The Onion, mocking the blind optimism of the AI-pilled. But then I felt sad and worried for them. Are they smoking because they believe AI is saving the world, I wondered, or because they believe AI is ending it?

It’s been an especially alarming few months here in San Francisco. OpenAI’s bots went rogue and hacked another company, and then an Anthropic researcher quit, warning that AI could wipe out the human race. It seemed more believable to me that these researchers were smoking because they didn’t think they’d be around for much longer — or at least freaked out about that possibility. I thought of the HBO show “The Leftovers,” which features a doomsday cult that chain-smokes. Because they believe the world has ended, they don’t mind obliterating their lungs. They’re living as though they’re already dead.

I asked a handful of people who work in AI and none of them knew anyone who had suddenly taken up cigarettes because of AI. So I don’t think what Sankar observed is widespread. But in a story last month about tech founders who smoke, my colleague Henry Chandonnet quoted a guy who wasn’t worried about his habit because he believed AI will soon cure lung cancer. Among the people I spoke to, a few recognized that underlying sentiment: AI’s going to fix (or destroy) everything, so YOLO. Last year, my former colleague Rob Price reported on these attitudes. Some were spending down their savings. Others were taking dangerous drugs and having orgies. Last year, my former colleague Rob Price reported on these attitudes. Some were spending down their savings. Others were taking dangerous drugs and having orgies.

While AI boomers and doomers expect completely opposite outcomes, I’ve found it interesting that their very different beliefs often end up manifesting as the same behaviors. Smoke or don’t smoke, it doesn’t matter: The future is inevitable. In either case, they’re relinquishing their fate to a humanity-ending or humanity-saving AI — and ignoring one of the most evidence-backed things they can do to live a healthy future. It’s a kind of fatalism, believing that what you do won’t change what happens next.

I see traces of that in so many corners of society. What’s the point of studying, students say, if they’re never going to be as good as ChatGPT at coding or math? What’s the point of agonizing over a breakup text, people say, if a bot can write it for them? What’s the point of working hard, employees say, if their companies are going to replace them with an agent? In a recent survey by Verasight, 78% felt that they had little or no say over how AI is being developed and regulated.

Even the people who have the most say don’t seem to believe they do. Anthropic CEO Dario Amodei has issued increasingly grave warnings about the technology that he himself is creating. He has to keep going, he says, because if he doesn’t, a less responsible steward of the industry will. Amodei and his peers sound as if they’re utterly powerless in the face of an unstoppable god. If it’s really that bad, couldn’t they just pull the plug? “Saturday Night Live” captured the strangeness of it all in a recent sketch impersonating Amodei. “If we can pressure lawmakers to create guardrails,” Jane Wickline’s Amodei deadpanned, “we will be able to stop me.”

We need people thinking seriously about the risks of these technologies, and we also need people dreaming big about the immense good they can bring. But neither of these outcomes is inevitable. As we entrust more and more of our work and our lives to AI agents, we shouldn’t cede our own agency to them too. I’m not taking up smoking anytime soon.