Sebastian Mallaby - Are we building a new God with AI?, Do founders need to have a Messiah complex, Is regulation of AI going to result in regulatory capture, What should Europe focus on

Why the AI safety fight runs through Washington and Beijing

Demis Hassabis builds AI because understanding the construction of reality brings him closer to the being that created it, and Sebastian Mallaby reads that as proof this is not a normal technology. The people leading the field distrust it in general and trust themselves in particular, which is why Mallaby thinks a messiah complex comes with the job. He dismisses the regulatory capture charge and argues the fight that matters is between Washington and Beijing, while Europe's answer is its own compute.

Takeaways

• Hassabis builds AI to get closer to God. Mallaby spent more than 30 hours with Hassabis, who told him AI is not a normal technology but a new form of cognition, and that understanding how reality is constructed brings you closer to its creator.
• Safety was baked into DeepMind's founding. Hassabis met Shane Legg at a 2009 lecture where Legg described a Halloween scenario of artificial general intelligence arriving around 2030, and their bond became the seed of DeepMind, so safety was in the vision from the start.
• Founders need a messiah complex to go first. Mallaby says Musk, Altman, Amodei and Zuckerberg distrust the technology in general but trust themselves, and that this supercharged version of a common human trait pushes each of them to get there first.
• Mallaby calls the regulatory capture charge nonsense. He calls the argument made by Jensen Huang and David Sacks wrong, since Amodei wrote about safety risk as a Google researcher before OpenAI existed, but he faults frontier labs for attaching a 10 percent annihilation number to an unknown.
• Chernobyl-scale accident is the likelier first shock. Mallaby expects a mid-level disaster such as Chernobyl or Three Mile Island, followed by political reaction and safety precautions, rather than the annihilation scenario, and says no data supports a probability either way.
• US and China hold the coordination problem. Europe will follow American rules because its companies need American customers and capital, as ASML has accepted chip export controls, while China is the only country that cannot be coerced. He finds the Bessent AI-safety channel encouraging, though a GPU deal is far off.
• Europe's answer is its own data centres. Once Europe has its own data centres it can run open-weight models on European soil, an approach Joe Tsai endorsed for European security, and Mallaby says the remaining obstacle is the continent's high energy costs.

How the book came together

• Mallaby finished The Power Law in 2022 and went looking for a new subject.
• Hassabis agreed to more than 30 hours of interviews, and ChatGPT launched just as he said yes.