The MAD Podcast: How AI Gets Built — with Matt Turck
00:49:59 · Nathan Benaich
[00:49:59] S0: The coolest use case I had recently is on like expert research where, you know, I think you can, well, if you're sort of into cheap man's version, it's like you ask your friends. But if you wanna spend a bit of budget and ask beyond your network, then you're gonna ask like GLG or AlphaSense, etcetera. And then they will go find humans for you. They will introduce you. You take the call. It's a ping ponging, like really annoying and limited by language, time zone, etcetera. But there's like a new product that I started using called Lato, which is basically like replacing the fact that I have to do the work with a model doing the work. And so I'm already like heavily using, in this case Codex, I kind of switched depending on what model is better at the time to sort of like just process all my ideas and the context I have on other companies that I looked at over history, how they performed, what questions I asked, what responses I got to just like figure out what is the highest value question I can ask of a team to inform like whether I should spend time or make the investment or not. And so I can take all that information and then with Lapto like construct an expert interview and then decide on cohorts I want and then just hit launch study. And then the system goes out and finds people, consents them to see if they're the right fit for the study. And then they go speak to an AI. And I was like shocked because, you know, I hit launch study, I went to sleep. I had like three days to basically give the company an answer. It was like in competitive markets, so I think the nuances of customer research is important. You'll wake up the first day, have three interviews from like pretty senior people. I have the raw transcript. The transcript can come back into my model, answer the open questions we had. We, me and my model. And then and then figure out, okay, we have enough data points for this, but like it's kinda opened up this question stream that we should ask. Maybe we should actually retune some of the interview questions to really coax the person to answer those questions. So it like updates the study questions and relaunches the survey. Next day, five five people. Again, this repeats. And over like two and a half days, had 12 in...
00:51:59 · Nathan Benaich, Matt Turck
12 interviews over like fourteen hours across like US, India, Europe, Germany, France, is like, it would be physically impossible for me to have spent like fourteen hours across 12 people. Like, there were several people who spoke to this model like for an hour and twenty minutes. [00:52:14] S1: Which just in itself is fascinating. Right? Like the fact that humans feel so comfortable so quickly speaking to an AI. [00:52:22] S0: Yeah. Yeah. It's probably a bit recursive in the sense that, like, we're all in front of a computer all the time and, like, these are generally, like, tech questions. And so I think people prefer to... Unfortunately, prefer to speak to their phone than speak to a human. [00:52:36] S1: That's a fascinating thought. That actually speaking to an AI is what people prefer. Yeah. Which is actually a theme that we heard over the years is like, you know, we we maybe not the years, but the months that people are starting to prefer to talk to an AI customer service agent over a person. [00:52:55] S0: Yeah, because I think in that case it's like a human is pretending to be a robot and doing a really bad job at it because you're a human, you're like, can you please just not gonna talk like a robot? And then now you get a robot that's actually like really good and then the [00:53:07] S1: human can Like a human.
00:53:07 · Matt Turck, Nathan Benaich
[00:53:07] S1: human can Like a human. [00:53:09] S0: Yeah. Okay. Amazing. Yeah. So that I think that that kind of idea of like automated customer research has been very powerful for me and I can just see how, you know, even if you had a portfolio, you would run like this kind of agent interviews on your portfolio companies and how customers are using it. So then at least you get some first party data when you go into the boardroom, you know. Or if you're an LP and like your investors are saying like, oh, this company is really good. Like, is it really good? [00:53:35] S1: A big thing that happened this year in the last few months in particular is to this point, like the rise of personal agents, the instincts and the muse and the rockbots. What do you make of that? [00:53:49] S0: I don't give them my data. I don't really trust trust them at the moment, but I think a lot of people do. [00:53:58] S1: A lot of people do. [00:53:59] S0: Yeah. [00:54:01] S1: So you... What you're saying is that you're you don't give your credit card information to a company that did not exist two months ago and doesn't require you to authenticate?
00:54:01 · Matt Turck, Nathan Benaich
[00:54:01] S1: So you... What you're saying is that you're you don't give your credit card information to a company that did not exist two months ago and doesn't require you to authenticate? [00:54:10] S0: Yeah, pretty much. Well also because I feel like I can do roughly the same thing with ChatGPT, like I don't actually see like what I get on top. Codex and Claude already have like low level access to my computer, has access to my browser, to like various messaging apps and things like this. So I I trust that already, so I don't really wanna increase like the risk on top of that. But I do think it brings up an interesting point which is like which vendor do you trust the most for these kinds of tasks? And not that long ago I heard like Nikolai at Revolut is now like thinking, okay, well I have like, you know, I don't know how many customers, but like millions or tens of millions or something. And they clearly trust me with my money and my credit card data. And, you know, now they have like an AI assistant in there. And so I could sort of see myself trusting Revolut more to handle purchases on my behalf than I would like some third party startup that's grown in twelve months and probably doesn't have correct InfoSec and other things. [00:55:12] S1: So you think it becomes a feature of broader platforms versus the opposite?
00:55:12 · Matt Turck, Nathan Benaich
[00:55:12] S1: So you think it becomes a feature of broader platforms versus the opposite? [00:55:16] S0: I think that's that's one possible route. You know, whether the platform ends up buying one of these teams is, you know, that that that can always happen. But I heard, you know, another point would be somebody prefers to use like Meta's muse because the likelihood that one of these startups survives for ten years and if they do something bad that you can sue them and get your money or get some money is very low. Whereas like suing Meta, like they have a history of like paying you when there's some issues. So I've heard that's, like, a reason why some people prefer to use Meta. Like, at least I know who to pick up the phone to. [00:55:48] S1: But but those those new kind of personal agents seem to be playing a big role in the diffusion of AI, though, which is one of the themes that you highlight in the report as well. You talk about the increasing rise of Codex in particular beyond coders.
00:56:06 · Nathan Benaich
[00:56:06] S0: Yeah. Yeah. Yeah. Exactly. And I think the biggest message there would be, you know, at the same time you see this like recursive self improvement. You see people doing all sorts of crazy stuff with their AIs and that's like a 0.1% or something of the AI usage base. And the vast majority of other people are barely paying for any AI and so getting like a very subpar experience. And part of this is just like bad product, bad marketing. Also just that, you know, a year ago or two years ago, there was all this like narrative around you need to become a prompt engineer. Like, you really need to understand how to write these prompts because if you write crappy prompts, the model does a bad job. So I can only imagine how many people who don't work in technology saw that and were like, I have no idea how to learn this. Like, this is way too complicated. You know? It's kinda like writing code, but in a human human language, like, screw this. Like, I'm done. Like, AI sucks. And and now it's like, do you really need to be a prompt engineer? Like, I think you more just need to be good at just like talking to your phone, just like brain dump exactly what you... Well, like everything you're thinking of and the system can organize itself. So I think this like prompt engineer thing is kind of gone. Like the model that can... Has enough affordances to not need that. But so now it's much more easy for a normal person to use, the but normal person got sick of it during the prompt engineering days and hasn't come back. So you're right that like creating these on ramps with like personal agents would be a great way to get people back into the system and you know maybe that's what they understand AI to be no longer like chat.