00:03:19 · John Coogan
[00:03:19] S1: Yeah. Totally. This one's for you. The fun the the the the most interesting benchmark to me has always been Arc AGI v three. We've interviewed the team over there many times and had a lot of fun understanding what goes into that that benchmark. And 5.6 SOL scored a massive 7.78%, which is tiny considering that the whole point of Arc AGI is that a human should be able to get 100% on it and basically any human. So it is a true test of AGI in the sense of, you know, can you give this test to just actually anyone, not, you know, the crazy math projects, the crazy hard programming projects, the hacking. All of that stuff is very economically valuable, of course, but there's a more interesting question where, you know, when there's less of a spiky frontier and there's just this question of what is something that anybody can do that AI can't? Because we've been searching for those and the Arc AGI team has done a fantastic job building out these puzzles that AI has historically struggled with. Arc AGI, one, model sort of climbed two, became a little bit more complicated and now, three, we're starting to see glimpses of progress, although 7.76% isn't 99%. We're nowhere near saturation, but it's still a huge jump. Opus 4.8 had 1.5%. So, GPT 5.6 SOL is showing more generalization, more spatial reasoning, more puzzle solving abilities. So fun, fun stuff. I am trying to refresh my timeline. But the blog post is also very, very fun because it includes games. I'm a big fan of the GPT 5.6 launch games. I got immediately sucked into the to the the sailing mini game, which is, like, very high fidelity, but also delightful to actually play.
00:05:18 · Jordi Hays, John Coogan
[00:05:18] S0: Should we play it? [00:05:19] S1: Yes. We should definitely play it. Yeah. Saltwind. You you you guys play it. I want production team to see what they can get. I think my time was twenty five seconds. [00:05:25] S0: And is this hosted on a on a site? [00:05:28] S1: I think this is I mean, this is hosted on the open end blog, but I think the the idea is that you could vibe code this in the latest GPT 5.6 in the app, in ChatGPT, and then deploy it and have someone are you trimming the sails appropriately? Because it looks like you're losing speed. You're losing wind. It's not working. I'm gonna smoke you. I got twenty five seconds. Wow. Amateur hour over here. Look at this. Boost? Yeah. Yeah. How do Well, well, the whole the whole game, which you probably missed, is that there is a little bar there where you have to trim the sails to be in the sweet spot of the wind while you're turning. So as you turn see the bar? There's a recommendation for where you put the sails. You gotta keep that line in see? It's moving over. You gotta you gotta press the [00:06:14] S0: down. See. [00:06:15] S1: See. See. Exactly. Keep trimming those sails while you steer the ship. This stuff is very, very fun. I am trying to open. [00:06:25] S0: One interesting data point from the livestream, which was just an hour ago. They said, already, Soul has been transforming our research program. As one example, GPT 5.6 Soul autonomously post trained 5.6 Luna. [00:06:39] S1: Yeah. That's fair.
00:06:39 · John Coogan, Jordi Hays
[00:06:39] S1: Yeah. That's fair. [00:06:39] S0: A lot of people are having fun with that. Dylan Fields says, a lot of people wanna compare Fable versus 5.6 Soul. This is a mistake. They're apples and oranges. Despite all the research achievements, we are still very, very early in exploring the tech tree for model training. [00:06:58] S1: Do want [00:06:58] S0: to take? [00:06:59] S1: Cool. Sorry. I'm just getting set up again. What else is in here? Oh, yes. I I I do think that didn't Dylan Eberscato write something about this? What what was the the essay he wrote about interactive memes and this idea of generative AI enabling these vibe coded mini games. Like, we've been seeing a bunch of them with the Copybearer simulator, the Coconut simulator, where it's something that's just a joke that's funny for a few people, But and normally, you would instantiate that in a in a tweet. Or maybe if you were getting really crazy, you'd do a Photoshop edit of a meme. But now you can go and create a full mini game, something that runs in the browser. And soon, something that runs in Unreal Engine and can actually be distributed on the Steam store. We're already seeing that with, like, the data center simulators and all these funny simulator games that are going on Steam. All this all the all the advances in the coding model certainly speeds up the ability to actually deliver polished software. I'm I'm I'm particularly excited for, [00:08:05] S0: like Yeah. Dylan's title was the future of entertainment is interactive. [00:08:09] S1: Yes. Yes.
00:08:09 · John Coogan, Jordi Hays, Eric Seufert
[00:08:09] S1: Yes. Yes. [00:08:10] S0: But, but, yeah, that that's part of what I honestly love about AI is there's a lot of things you can make now that never would have made sense to make because they would have taken you four days and it was good for, a small laugh. And now you can do it in four minutes. And and, it's just fun. [00:08:26] S1: Yeah. I I I think there's gonna be there's if you have some sort of, like, small custom some sort of custom functionality in your business, it feels like there's Is this the David Senra simulator? Why is this David Senra? [00:08:42] S0: Late nights in a Miami abandoned apartment Backrooms. Complex in 2015, just recording podcasts and reading. [00:08:50] S1: This is very creepy, like, horror backrooms, liminal space game. [00:08:55] S0: Stanley Tang, cofounder and CPO over at DoorDash says, I have an insane magic trick that so far none of the models can figure out including mythos. It's a bulletproof trick that I've shown to a 100 plus people including magicians that couldn't figure it out. It's not anywhere on the internet. Only way to know it is through first principles reasoning. [00:09:13] S2: Mhmm. [00:09:14] S0: Told everyone I'll believe in AGI when it can crack this trick. Well, GPT 5.6 just did How? I want him to I want him to actually open like [00:09:24] S1: Well, now Okay. [00:09:24] S0: Like, give us now that now that a model cracked [00:09:26] S1: it Explain it. Lot of magic tricks are like sleight of hand. So is he uploading a video or something? Like
00:09:26 · John Coogan, Jordi Hays
[00:09:26] S1: it Explain it. Lot of magic tricks are like sleight of hand. So is he uploading a video or something? Like [00:09:32] S0: Well, yeah. So John Palmer says I have a hilarious joke that so far none of the models think is funny. It's a bulletproof joke that I've told to a 100 people, including comedians, and no one laughed. It's not anywhere on the Internet. Only way to know it's funny is a first principle sense of humor. Told everyone I'll believe in AGI when it tells me a joke. The joke is funny. Well, 5.6 just did.
00:09:53 · John Coogan, Jordi Hays
[00:09:53] S1: Huge. Huge news. Huge news. Yeah. People are going back and forth. GBT 5.6 is a Porsche. Fable's like warp drive. I had a different experience. A fable is an f one car. 5.6 Solit Ultra is a Tesla Model x Plaid, does it find things that Fable misses during plannings and coding? Yes. Most of the time. But for the hardest problems, does Fable routinely find things that Fable that five point six doesn't? Also, yes, some of the time. Is five point six way faster and affordable? Yes. With an unlimited token budget, what am I currently using? 95 per 95 plus percent of the time, GPT 5.6 from Sikhi Chen. So interesting take that the Pareto frontier is alive and well, and everyone's duking it out for their slice of the AI opportunity. Very interesting seeing how the market share is shifting while during a time of acceleration, you have multiple companies that are growing revenues, even accelerating revenues, while market share is declining because the overall market's growing so fast that you that if you're only growing at 300% and someone else is growing 400%, you're losing market share where you have, like, one of the greatest businesses by modern metrics. Very, very interesting dynamics in AI. [00:11:04] S0: It's also funny because yesterday with Ben Thompson, you were like, it's some slow summer. [00:11:08] S1: And then in in [00:11:09] S0: the in the span of twenty four hours, you get Mach four five Muse 1.1. [00:11:16] S1: Yeah. I mean, this I don't know. This isn't this isn't as dramatic as the AI talent wars. It's not as dramatic as