The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:48:41 · Harry Stebbings, Thomas Sohmers
[00:48:41] S1: about does scaling laws look like at three times what it is now? If AGI has been declared now by Janssen, forgive me, but what is three times this? [00:48:51] S0: That's a good question. I mean, GPT six Astra, my first, you know, twenty four hours with it were basically as magical as my first experience with GPT 3.5 in November '22. I I was at the, ChatGPT launch at NeurIPS in 2022, and it was so funny because, you know, Sam and and Ilya were there, and and they... You know, it was a party in in New Orleans for the NeurIPS conference. And, basically, at the end, they just said, hey. We launched this little fun, you know, experiments called ChatGPT. Go check it out. And, like, zero fanfare, you know, it was it was really just a side mention. And I don't think anyone really gave it a thought at the event. But when I went back to the hotel, I loaded it up, and I got back at, you know, ten, 11PM, or whatever. And I was up for four or five hours straight just giving random props and and this that this was the most magical experience that I've ever had with a computer. I would say I got very close when Sora two came out that I had similar experience, short amount of time, but just mind blown by the quality of the the videos and especially the weekend that Sora two launched when there being no restrictions on what you could generate. But, yeah, GPT six Astra, I do think is is AGI. And to to your question of, like, what does that mean going forward? I think my guess is this is as good as basically anyone's. But
00:48:51 · Thomas Sohmers, Harry Stebbings
[00:48:51] S0: That's a good question. I mean, GPT six Astra, my first, you know, twenty four hours with it were basically as magical as my first experience with GPT 3.5 in November '22. I I was at the, ChatGPT launch at NeurIPS in 2022, and it was so funny because, you know, Sam and and Ilya were there, and and they... You know, it was a party in in New Orleans for the NeurIPS conference. And, basically, at the end, they just said, hey. We launched this little fun, you know, experiments called ChatGPT. Go check it out. And, like, zero fanfare, you know, it was it was really just a side mention. And I don't think anyone really gave it a thought at the event. But when I went back to the hotel, I loaded it up, and I got back at, you know, ten, 11PM, or whatever. And I was up for four or five hours straight just giving random props and and this that this was the most magical experience that I've ever had with a computer. I would say I got very close when Sora two came out that I had similar experience, short amount of time, but just mind blown by the quality of the the videos and especially the weekend that Sora two launched when there being no restrictions on what you could generate. But, yeah, GPT six Astra, I do think is is AGI. And to to your question of, like, what does that mean going forward? I think my guess is this is as good as basically anyone's. But [00:50:12] S1: Why was GPT Astra so good for you? Why was it comparably such a breakthrough? Because I I haven't... It's great, but honestly, kinda the same as before.
00:50:20 · Thomas Sohmers
[00:50:20] S0: Oh, in terms of the things that I've found LLMs to fail the most at in the past. So I'll give a case where it is more linear improvement. So just in terms of general coding capabilities, performance, and analyzing problems, etcetera, it is a step function improvement, but not mind bogglingly. So there are a bunch of things that other models have not been able to fix or kind of went in circles and found inelegant solutions and it's still like a human software architect that was able to to come up with a better solution. With Astra, they're just initially giving in a couple of really hard problems that I've not been able to solve with other LLMs. Was able to do it one shot, having it go through a code base and find both performance improvements, bugs, etcetera. Basically, discovering new spaces that I didn't know existed in our our bunch portions of our code base. So that's one element. Step function, but not mind boggling. The second case that was mind boggling just from a, like, wow, was the the computer usability is with set of generic tools. So, like, being able to do Blender animations. Like, you know, it's become... There... There's a bunch of memes online of of it recreating different videos, etcetera. But just the fidelity of that and where that was basically impossible with GPT 5.6 Soul was massive increase capability. And, like, I had it design... You know, do interior design of my house just based on a couple pictures and just like, wow. Like, I did not think that at... What's fundamentally a text model could could do that. And then finally, like, the the biggest thing for Positron was I've been trying with every single new model release to have these models be able to, like, actually take a relatively simple logic design problem, you know, implementing a a encryption block in this case, and being able to take that through the full RTL to GDS flow. So from basically the specification of do this encryption function, implements the Verilog, so the hardware description language for that. So write that code and then be able to take that code and go through all the way until you have got a chip design that theoretically you could go tape out. LMs could do different portions of that and could, like, write the scripts and, you know, fail a lot of different midpoints on the way.
00:52:31 · Thomas Sohmers
But a big problem with the electronic design automation tools, the EDA tools for doing chip design, is that they were designed in the nineties, early two thousands. They're really unintuitive. None of the documentation exists out, like, in the public web, and so that these models don't have, like, a real good innate view of them. But GPT six, with both combination of computer use and just an ungodly amazing scripting ability has been able to take this this tech hack block and implement it with the TSMC n three PDKs and take that all the way to GDS and do that in, like, a little over, like, 50 and meet timing at, you know, over a gigahertz, etcetera. And that as a task, if I was giving to someone similarly new to a thing, like, getting the flow mostly working, I would say would take on the order of a week And getting it optimized to the point that Astra is at with with that design would maybe be one or two additional weeks depending on the person. So compressing that two to three weeks down to two days and change, it's still mind boggling. It... Like, it shouldn't be this good at this as I would naively think about its training sets. But, obviously, with OpenAI's own chip developments, you know, in house, they've... I'm I'm glad that those capabilities are getting added to the the models they're releasing to the public and not just being kept in... Inside.
00:53:48 · Harry Stebbings
[00:53:48] S1: We see Jalapeno, terrible name, I think, personally, but, you know, their own chip development, Anthropica developing their own chips, DeepSeaker supposedly developing their own chips. We see the commoditization of the chip player with everyone building their own ships. How should we think about that?
00:54:02 · Thomas Sohmers
[00:54:02] S0: As a consumer of all these things, if I take my Positron hat shirt off, I would say that that's a great thing for the industry having, fundamentally, that's gonna bring costs down and capabilities up and and bring it to to more people. I think it's such an interesting world where when I got started in the semiconductor space, you know, thirteen years ago, you know, silicon was a dirty word in Silicon Valley. And now you have all the biggest companies in the world being, you know, somehow connected to the semiconductor industry and the most interesting, exciting applications and the the companies building them, you know, vertically integrating down to the silicon layer. The interesting thing with all the the ones that you mentioned and and, you know, the broader set is companies have the same macro goals. The implementation details are all unique though, and that's... Just as an engineer and and technologist, it's exciting to me that there are a lot of different ways to skin a cat, and people can have their own architectural view and and go about implementing it and and get different results. And, you know, hot chips, the biggest conference for for this design space and was where OpenAI unveiled Jalapeno last month. I'm happy that the industry is still pretty open and willing to share not as much details as people would have shared, you know, five, six years ago, but still a good amount of in the open discussion of of things. And so I think how that applies to Positron is, you know, we have our particular architectural views and way that we've decided to do things and that will evolve in in the future as will everyone else's. But, you know, there's still plenty of space to make bets and go in different directions. And the great thing about the market is that the market gets to to decide what is is valuable, and those that create value will receive a reward for that.