00:14:26 · Eric Landau
[00:14:26] S1: Well, one thing we we didn't really expect, the ChatGPT moment, it just kind of came, and I remember very very precisely, like, where I was when I first saw it. It was, like, the day of... I was in Boston for... I was seeing a customer, and I was actually talking to, Victor from Synthesia the next day for my podcast. And, I remember what I did was I took... I was preparing a bunch of questions. I thought, oh, what would be fun is I create half the questions, and this new ChatGPT thing creates the other half of questions. And then I asked Victor, which one's human, which one was, AI? And he was shocked they were so close together. I mean, that was where where we were Wow. That's, like, real. Three, yeah, three years ago. So that's something that we were, like, very surprised that Yeah. This vector of AI in particular would be the one that would take off. And even people at OpenAI were... Like, we've talked to some of the people that were, heading the ChatGPT project, and they were all all... Also surprised. So that's something that we weren't expecting. Yeah. I think what we got right in general was just the importance of data, because, again, just from first principles, there's only three ingredients that go into an AI system. It's models, compute, and data. And, if you don't get one of the ingredients right, then the output is not going to work. Mhmm. So you need to make sure that you're getting high quality data for whatever AI system that you're training.
00:15:41 · Sebastian Johnson
[00:15:41] S0: What do you think about the LLMs at the moment in general? We're seeing a plateauing of the improvements across all models, we're seeing increased level of competitiveness, seeing like Gemini is now often talked about as the better model compared to OpenAI, it's managed to catch up. What's your view? Do you do you see a world in which the models become commoditized?
00:15:59 · Eric Landau, Sebastian Johnson
[00:15:59] S1: Yeah. So I I think I have a different frame on it. One is, like, where does the scaling law actually, like, manifest? Mhmm. And one, kind of metric that I I track and a bunch of other people track is, from the METR Institute in terms of the average length of task that an AI system can, successfully complete. And if you look ten years ago, the average length of task would be something like recognizing a cup, which takes a human, like, a a millisecond to do. So an AI could do something that takes a human a millisecond to do. And then a couple years later, it might be to, write an email, without actually, having the the context lost in the email, and that might take, like, five minutes. Now we're at the place where an AI system can successfully successfully do something like... That takes a human two hours, and this number doubles every seven months. So now these AI systems are doing things that humans take two hours. In seven months, it'll be four hours. Once it hits to the level that it's actually two weeks of time, that's the typical kind of cadence cycle of what we do in our company, like, two week sprints. So that's where you'll have an AI, like, as a as a coworker, as a colleague, as an employee, and that will look like a very different world. And that... This, exponential, this doubling every two months, that hasn't slowed down at all. So while it's easy... Like, you know, in certain metrics, there's saturation. It's hard to see, like, the the deltas of improvement. [00:17:23] S0: Yeah.
00:17:23 · Sebastian Johnson, Eric Landau
[00:17:23] S0: Yeah. [00:17:24] S1: There are places where the AIs are just kind of continuously improving at the rate or faster than the rate than than we expected. So while, yeah, there's gonna be a lot of competition, the progress of the models is not slowing down at all. And I think that's, like, a misconception that some some people have. [00:17:38] S0: Do you have a peg for who might come out on top, or are you able to look... Looking at those metrics? Are you able to see differences in the providers?
00:17:45 · Eric Landau, Sebastian Johnson
[00:17:45] S1: I think there can be multiple winners because there's there's not just one end metric that matters. Mhmm. And, I think, like, one also misconception that people have is that the intelligence of the model is the only thing that matters. And the the analogy that I give is that, you actually don't need very high intelligence for a lot of use cases. You don't want Einstein to be bagging your groceries. Like, I don't I don't need, like, a PhD level physicist to be the one that just kind of recognizes the carrot from the milk and puts them in the in the right place in the bag. Right? I need someone that... I need, like, a a model that's, like, sufficiently good. Yeah. So you can have model providers that actually do some set of tasks, like, really, really well. And and now Anthropic is doing really well in, like, focusing on enterprise and coding, and they're really winning in that space. And OpenAI is focusing on consumer use cases and the regular chatbot and, like, being more of a personal assistant. There is a world where... Like, there's more than one winner. Mhmm. At the end of the day, if I was to be a, like, purely betting man, I would say that probably the company that's attached to the biggest cash cash machine will win, which is, Google. I mean, they're, you know, they have a huge ATM in their pocket that's constantly being fed, to to data centers. They can continuously create models, but I think it won't just come down to a singular, player at the end of the day. [00:19:03] S0: Okay. Interesting. Multimodels but focused in different niches.
00:19:03 · Sebastian Johnson, Eric Landau
[00:19:03] S0: Okay. Interesting. Multimodels but focused in different niches. [00:19:07] S1: Different niches. Yeah. Models are... They have personality. Right? They're like people. Right? And they... They're gonna be good at things, and we're gonna have different, preferential, kind of treatment and, interactions with with different types of models. So there's no reason why... Yeah. You you know, you'll want a model that's, like, super friendly and, like, this is your friend model, and this is your coworker model Yeah. And this is, like, your, you know, how... Your servant model or whatever. Those can all exist, like, in in tandem with each other. They don't have to be one, one, end model. [00:19:41] S0: How how do you think about leveraging AI in your business internally? We've seen things like, OpenClaw. You know, these are major regenerative tools be... Know, that felt like another ChatGPT moment when had this new technology come out. We're seeing a wide range of tools across marketing, go to market. How are you getting up to speed with the tools and implementing them across your own business?