The MAD Podcast: How AI Gets Built — with Matt Turck
00:51:59 · Nathan Benaich, Matt Turck
12 interviews over like fourteen hours across like US, India, Europe, Germany, France, is like, it would be physically impossible for me to have spent like fourteen hours across 12 people. Like, there were several people who spoke to this model like for an hour and twenty minutes. [00:52:14] S1: Which just in itself is fascinating. Right? Like the fact that humans feel so comfortable so quickly speaking to an AI. [00:52:22] S0: Yeah. Yeah. It's probably a bit recursive in the sense that, like, we're all in front of a computer all the time and, like, these are generally, like, tech questions. And so I think people prefer to... Unfortunately, prefer to speak to their phone than speak to a human. [00:52:36] S1: That's a fascinating thought. That actually speaking to an AI is what people prefer. Yeah. Which is actually a theme that we heard over the years is like, you know, we we maybe not the years, but the months that people are starting to prefer to talk to an AI customer service agent over a person. [00:52:55] S0: Yeah, because I think in that case it's like a human is pretending to be a robot and doing a really bad job at it because you're a human, you're like, can you please just not gonna talk like a robot? And then now you get a robot that's actually like really good and then the [00:53:07] S1: human can Like a human.
00:53:07 · Matt Turck, Nathan Benaich
[00:53:07] S1: human can Like a human. [00:53:09] S0: Yeah. Okay. Amazing. Yeah. So that I think that that kind of idea of like automated customer research has been very powerful for me and I can just see how, you know, even if you had a portfolio, you would run like this kind of agent interviews on your portfolio companies and how customers are using it. So then at least you get some first party data when you go into the boardroom, you know. Or if you're an LP and like your investors are saying like, oh, this company is really good. Like, is it really good? [00:53:35] S1: A big thing that happened this year in the last few months in particular is to this point, like the rise of personal agents, the instincts and the muse and the rockbots. What do you make of that? [00:53:49] S0: I don't give them my data. I don't really trust trust them at the moment, but I think a lot of people do. [00:53:58] S1: A lot of people do. [00:53:59] S0: Yeah. [00:54:01] S1: So you... What you're saying is that you're you don't give your credit card information to a company that did not exist two months ago and doesn't require you to authenticate?
00:54:01 · Matt Turck, Nathan Benaich
[00:54:01] S1: So you... What you're saying is that you're you don't give your credit card information to a company that did not exist two months ago and doesn't require you to authenticate? [00:54:10] S0: Yeah, pretty much. Well also because I feel like I can do roughly the same thing with ChatGPT, like I don't actually see like what I get on top. Codex and Claude already have like low level access to my computer, has access to my browser, to like various messaging apps and things like this. So I I trust that already, so I don't really wanna increase like the risk on top of that. But I do think it brings up an interesting point which is like which vendor do you trust the most for these kinds of tasks? And not that long ago I heard like Nikolai at Revolut is now like thinking, okay, well I have like, you know, I don't know how many customers, but like millions or tens of millions or something. And they clearly trust me with my money and my credit card data. And, you know, now they have like an AI assistant in there. And so I could sort of see myself trusting Revolut more to handle purchases on my behalf than I would like some third party startup that's grown in twelve months and probably doesn't have correct InfoSec and other things. [00:55:12] S1: So you think it becomes a feature of broader platforms versus the opposite?
00:55:12 · Matt Turck, Nathan Benaich
[00:55:12] S1: So you think it becomes a feature of broader platforms versus the opposite? [00:55:16] S0: I think that's that's one possible route. You know, whether the platform ends up buying one of these teams is, you know, that that that can always happen. But I heard, you know, another point would be somebody prefers to use like Meta's muse because the likelihood that one of these startups survives for ten years and if they do something bad that you can sue them and get your money or get some money is very low. Whereas like suing Meta, like they have a history of like paying you when there's some issues. So I've heard that's, like, a reason why some people prefer to use Meta. Like, at least I know who to pick up the phone to. [00:55:48] S1: But but those those new kind of personal agents seem to be playing a big role in the diffusion of AI, though, which is one of the themes that you highlight in the report as well. You talk about the increasing rise of Codex in particular beyond coders.
00:56:06 · Nathan Benaich
[00:56:06] S0: Yeah. Yeah. Yeah. Exactly. And I think the biggest message there would be, you know, at the same time you see this like recursive self improvement. You see people doing all sorts of crazy stuff with their AIs and that's like a 0.1% or something of the AI usage base. And the vast majority of other people are barely paying for any AI and so getting like a very subpar experience. And part of this is just like bad product, bad marketing. Also just that, you know, a year ago or two years ago, there was all this like narrative around you need to become a prompt engineer. Like, you really need to understand how to write these prompts because if you write crappy prompts, the model does a bad job. So I can only imagine how many people who don't work in technology saw that and were like, I have no idea how to learn this. Like, this is way too complicated. You know? It's kinda like writing code, but in a human human language, like, screw this. Like, I'm done. Like, AI sucks. And and now it's like, do you really need to be a prompt engineer? Like, I think you more just need to be good at just like talking to your phone, just like brain dump exactly what you... Well, like everything you're thinking of and the system can organize itself. So I think this like prompt engineer thing is kind of gone. Like the model that can... Has enough affordances to not need that. But so now it's much more easy for a normal person to use, the but normal person got sick of it during the prompt engineering days and hasn't come back. So you're right that like creating these on ramps with like personal agents would be a great way to get people back into the system and you know maybe that's what they understand AI to be no longer like chat.
00:57:33 · Matt Turck, Nathan Benaich
[00:57:33] S1: And to unpack a couple of things that you that you just said were alluded to. So there's this idea that there is an increasing gap, between people using AI and non AI, and you have a slide that talks about how Codex adoption remains far higher inside of OpenAI than amongst external users. [00:57:53] S0: This is wild. Yeah. Yeah. I mean, the broad stat is like something like a 100% of people in OpenAI use Codex and then of their organizations that use it, it's something like [00:58:04] S1: two thirds maybe, something quite high. Yeah, 69%. Yeah. Okay. The separate analysis of S and P 500 companies AI disclosure found that 69% reported live deployments, but only 2% disclosed metrics tracked over time. And that's in contrast to 97.9% of active OpenAI workers use Codex in the preceding twenty eight days. [00:58:25] S0: Yeah. Yeah. Yeah. And individuals is like less than 1% use Codex. And then of the people who do use Codex, the vast majority of them is like 85% do tasks on it that would roughly take like thirty minutes. And only like 25% I think do it... Do tasks that would take like eight hours or more. [00:58:40] S1: So the conclusion of this is that the the people that get it are just accelerated further and further, whereas the people that don't, comparatively slow down? [00:58:51] S0: Yeah. It's basically like you're you're kind of going through like a theme park or something and you just you discovered like level one and there's like this entire like other world that you just haven't even opened the door to.