I

Here’s a belief that may be more widespread than you’d think:

“In a world without an intelligence explosion, I am going to die no matter what. Therefore, I should support accelerating the development of superintelligence / a technological singularity.”

This argument cleanly maps onto a traditional 2x2 Pascal’s wager:

	Alignment works Takeoff would go well	Alignment fails Takeoff would go badly
Accelerate Takeoff while alive	Immortality +∞ utility	You die, sooner ≈ baseline
Hold back Takeoff after you die	You die of old age Baseline	You die of old age Baseline

Now, the obvious tradeoff here is that a “bad” intelligence explosion is going to kill you and everyone you care about (or worse). In fact, depending on your AI timelines, this death may happen a lot sooner than it would in a world without an AI takeoff.

For you, my hypothetical friend and strawman, this doesn’t matter: dying in 5 years is worse than dying in 50 years, but in either world you are dead for the next 20 billion years¹.

That’s bad! You’d much rather spend that time, I don’t know, playing cards with your friends, making art, learning mathematics, or exploring the universe. If none of these appeal to you, synthetic heroin and the infinite pleasure Goon-o-tron 3000 are on the table.

For at least part of my life I found this whole argument pretty compelling.

I don’t think I ever felt the call of the infinite pleasure machine, but death is scary.

I don’t want to die! Considering the absolute state of academia, the slow pace of biological and medical research, the existence of nuclear weapons and the mess that is geopolitics, the odds of humanity reaching longevity escape velocity within my lifetime seemed pretty low.

It’s not like I really had to think about it, or faced any hard moral decisions: it was all so far away, and I was in no position to have any impact on anything². Knowing who Nick Bostrom is didn’t change that.

Little did I know that the future would catch up faster than any of us would like. We’re now at the point where modern AI systems are capable of writing novel mathematical proofs and developing software with minimal human supervision, while venture capital is chomping at the bit to let AI loose on biomedicine.

II

Why isn’t the Takeoff Pascal’s Wager a compelling argument?

Well, it is a compelling argument. That’s the problem. It’s compelling for people in a narrow range of takeoff timelines.

If you were, say, 90+ years old, and believed an intelligence explosion would happen in exactly 30 years, then this argument would probably not move you. You’ve lived a long and hopefully fulfilling life, but the singularity is most likely going to happen without you.

On the flip side, if you were a mere 12 years old, the same timeline of 30 years would have a very different effect: you would be crazy to push for a fast AI takeoff, because AGI is going to happen within your lifetime regardless of what you do. What matters to you is a safe takeoff, without societal collapse or deadly bioengineered viruses.

The Takeoff Pascal’s Wager is compelling exactly as long as you fall within the band of people who are aware of the technological risks³, and neither too old (you’re going to miss it) nor too young (it will happen regardless), according to your own timelines.

This splits the world into three distinct groups: Safety-ism, Wager, and Skepticism.

Thesis: As the impact of AI on the world becomes more visible, everyone’s timelines are going to shorten. This means that young people will slide off the hill, towards safety-ism⁴. On the other side, older people may slide onto the hill, towards accelerationism.

As for myself, not too long ago I was wistfully staring at the night sky, hoping to explore the stars in a hypothetical future that I knew could never truly be, and yet here we are a few years later, with hacking scandals and AI progress so rapid that it feels like we’re rushing head-first into disaster.

Of course, that pushed me to the left, off the hill into the safety, alignment and oh-god-how-will-society-deal-with-this camps.

III

Does any of this matter? Maybe. It matters insofar as you believe that there are people out there who right now have the power and influence to meaningfully impact the future of AI, and who believe that there’s a real risk they will die of old age (or a terminal illness, or revolting underlings) unless AGI happens quickly enough and they are on “the winning team” (whatever that means).

Unfortunately, we just so happen to live in a society. Power exists, and power skews old.

As timelines shorten across society, these powerful people might find themselves with not-quite-enough time to think through the implications of superintelligence, nor enough time (or willingness) to read the entirety of Yudkowsky’s works.

Basically, there’s a chance they’ll end up on the hill of accelerationism.

For many of them, we wouldn’t even know: we don’t really talk about death, so unless you’re Bryan Johnson and ready to make it your whole personality, it’s better to keep your mouth shut.

So, if you find this argument remotely compelling, you should worry about what’s going to happen as powerful older people across the world become more aware of technological acceleration, and what it means for their own mortality⁵.

Alignment is hard enough on its own, you do not want to be caught up in shadow battles against moguls, CEOs, and politicians who deep-down yearn to cement their legacy.

But I’m sure it’s going to be fine. It’s not like anyone was ever driven mad by the hunt for immortality, right?

Oh, you’re saying that did in fact happen? Multiple times, even?

Well, that was like 1000 years ago. I’m sure the great leaders of the modern world are fully in touch with the transcendent, and willing to freely hand over the power that they’ve fiercely clung to for decades.

Oh.

Wait, Putin and Xi were discussing that people may live to 150 this century using organ transplants and biotech??

Ah well, nevertheless.