How to Stop an AI Apocalypse: Aligning Intelligence, Starting With the Self

Johnson opens with a compressed history. The universe has existed for 13.8 billion years and Earth for roughly 4.5 billion, and for most of that time life existed as single cells. Then came Homo sapiens, a species that thinks, and thinks about thinking. It acquired fire, language, agriculture, cities, writing, industry, electricity, nuclear and solar power, the internet, and finally AI. In Johnson's framing, it took almost 13.8 billion years for intelligence to make more intelligence.

Johnson says we are "baby steps away" from superintelligence, which they call possibly "the most extraordinary event in the history of the universe." They expect it to "rewrite reality" and to pop the bubbles we live in, revealing dimensions and realities beyond our current imagination and comprehension.

Johnson acknowledges a wide range of opinions. Some people believe humans are inevitably doomed, while others believe AI will save us from ourselves. What we do know, they argue, is that AI is improving faster than we can comprehend and in ways we cannot predict. They also say there is no way to stop or even slow its development. Nation-states, corporations, ideological groups, and individuals will all "unabashedly" use AI for their own self-interested goals.

The core danger, in Johnson's view, is human psychology. People tend to be reactive rather than proactive. They call this a failing of the human mind, and one that may be "unforgiving to our survival" in this situation. Building AI systems aligned with continued human existence is, they say, a discipline in its own right that deserves the greatest minds of our generation.

For people not working directly on AI alignment, Johnson offers a different role. They can contribute by recognizing that aligning all intelligence is the central opportunity of our time, and that this can begin with oneself.

Johnson describes their daily routine as a trial version of human AI alignment. Blueprint treats them as a collection of 35 trillion cells and aligns all of those cells toward one objective: continued existence. They say they did this to demonstrate "a beta version of a future human," one intelligent being trying to align itself.

The motivating questions are:

• What if a person could improve themselves at the speed that technology improves?
• What if everyone could?
• What if decay and decline were not inevitable?

Johnson says these ideas would have sounded ridiculous in the 1990s but no longer do.

To make the idea concrete, Johnson poses a thought experiment. Imagine you had access to an algorithm that could produce the best physical and mental health of your life. The catch is that the algorithm, not you, would decide what you eat, when you eat, and other health-related choices. Would you opt in or decline?

Johnson admits the scenario is oversimplified. To answer it sincerely, each person would need to ask hundreds more questions. They then report their own experience. Two years after saying yes, they say they have never been happier, healthier, more emotionally stable, or more intellectually alive. They no longer spend time thinking about their next meal or when to go to bed. They also no longer struggle over whether to do "this naughty thing" that would speed up their aging.

Johnson recognizes that the idea may trigger a kind of existential crisis for listeners. They compare its scale to the realization that Earth is not the center of the universe.

Johnson is candid about Blueprint's current limits. The first version took years and millions of dollars to build. Most of its processes are still manual and clunky, and it requires a large team. They say it is impractical to imagine the current version being scaled across society.

They do not see this as a problem, because, as they put it, this is how innovation works. The task is to look past the awkward parts and find "the gem of inevitability" that the prototype has demonstrated.

Johnson says their mind is constantly searching for the most important information available and the highest-value use of their time. The task they have set themselves is to write a letter that prepares the audience for the future.

Their central claim is that the future will not resemble the past. Every earlier generation could learn from history and model their lives on it, including how they thought about individuals, countries, currencies, and relationships. Johnson argues that, for the first time, we cannot do this. They call it the "zeroth nature" of computational intelligence: AI removes our ability to use the past to predict the future.

Their advice is to be aware of, and cautious about, the assumptions we make about what comes next. If people make this a habit in how they process information, Johnson says, they will be better able to sift through arguments, trends, and norms and keep their thinking in step with technological change.

Johnson offers three questions to ask when making life decisions:

1. What must remain true for this to continue to be true?
2. What new thing would make this untrue?
3. What wildly unexpected surprise would change the question?

Johnson ends by reducing humanity's priorities to three items: don't die, don't kill each other, and align AI with "don't die."