Why Sam Altman is against a single superintelligence

Altman prefers many slightly superhuman AIs to one powerful system

Sam Altman argues that AI is better built and released step by step, with many capable systems rather than one dominant superintelligence. Models are already finding serious software flaws and solving hard research problems, so capability and risk arrive together. The cost of that capability keeps falling, which moves the constraints to memory, chips and data centers.

Takeaways

• Altman prefers many AIs to one. Altman argues a world of many slightly superhuman AIs is preferable to one very powerful system, and links this to the resilience of pluralistic society. He also dislikes the term agent swarms and prefers agent teams, which he thinks have interesting alignment properties.
• AI will uncover many more serious vulnerabilities. Models keep improving at finding bugs, and people have only begun scanning the large amount of code online, so Altman expects more serious vulnerabilities than anticipated. Collison says architectures may need to change too, which is harder but more durable.
• Altman defends iterative deployment despite cyber risk. Altman says the field has not explained why the downside is worth it, yet OpenAI keeps deploying iteratively. Sandboxing works to a point and monitoring further, but ultimately models must be trusted as aligned. Collison says insiders doubt sandboxing is possible.
• Frontier research costs should fall rapidly. Altman estimates the Navier-Stokes proof cost around $10m and bets it could be redone at one hundredth of that a year later. He is confident of 10x yearly gains in compute allocation and says OpenAI does not reward heavy compute use.
• Memory shortages show the market failing for now. Altman says DRAM makers historically invested $20bn of capex for $19bn of revenue, and may now each earn about $100bn of free cash flow in a year. He wants cost per gigawatt to fall from a $70bn five-year benchmark toward $20bn.
• The must-have personal agent has not arrived. Altman says no one has yet built the personal agent people cannot live without, but expects it to resemble current products that work reliably. Collison says businesses resisting AI apps misread defensibility, which comes from brand and product rather than web forms.
• Tokens are a fake unit for pricing. Altman says OpenAI nearly charged by consistent units, but customers wanted to know what they pay for. Collison says tokens are about 1% of Stripe's operating expenses and could reach tens of percentage points of a company's costs at steady state.

Compute and chip supply views

• Altman says self-replicating data centers are within grasp, with AI controlling robots that run fabs and build turbines, solar panels and data centers. OpenAI already has a robotics program aimed at building data centers.