The a16z Show
00:16:58 · Josh Elman, David Pawlan
[00:16:58] S0: K. And and I have [00:16:59] S1: one more question on the flowers. [00:17:00] S0: Yeah. Because... So you you were [00:17:01] S1: just ordering flowers for your... [00:17:02] S0: Yourself, I guess, just to, like, test their flower. My girlfriend. [00:17:04] S1: She got a got a nice delivery of of flowers. [00:17:07] S0: Okay. So this is actually really important because if you're delivering to somebody else [00:17:11] S1: Mhmm. [00:17:11] S0: You were talking about saving money, saving money, saving money. But, like, you don't want your girlfriend to think you just got the cheapest flower possible. I hope... Well, maybe, you know, she likes your, you know, penny saving nature. But but otherwise, like, I would think you also wanna, like, impress her and also be able to get her the ones if she's really into orchids to get her orchids. So did it do anything to try to make sure also that it wasn't just this pure price saving, but also something that's gonna actually make you, you know, earn the excitement and stand out a little bit for [00:17:38] S1: So for this experiment, no. And and the purpose of this was I wanna test two different dimensions. Yeah. And if I'm going to run this test, I might as well have a beautiful consequence of my girlfriend getting flowers Fair enough. As the outcome. But it... It's a great point. Right? Like, in in the real world, if you're gonna ask that question, there's gonna be added dimensions that are, you know, second layer dimensions to cost efficiency or to delivery time. And make [00:18:01] S0: sure the price doesn't show
00:18:01 · Josh Elman, David Pawlan, Anish Acharya
[00:18:01] S0: sure the price doesn't show [00:18:02] S1: up when she actually gets the flowers. Or like a note as an example. Some of them did ask, do you want me to write... Leave a note with with the flowers when they arrive? [00:18:09] S2: Yeah. It's interesting because there's this sort of thing that has to happen when you talk to the agent about these things, which is your implicit preferences have to be made explicit. [00:18:17] S1: Mhmm. [00:18:17] S2: You know, and there are people who like to think like, oh, I'm not cheap. I just like to save money. Right. But you're not gonna have to kinda maybe like get over that and tell the agent, I'm actually pretty cheap. Right. And I just need you to actually run with that. Whereas when you're actually taking these actions on your own behalf, you can build those preference in without having to, like, sort of look in the preference mirror. Yep. So there's also something very fascinating about how does the agent interact with you in a way that sort of maintains your sense of identity and safety and doesn't cross any of these boundaries. Totally. [00:18:46] S1: And and I think something that is really interesting with that specifically is Dot because it has so much of this context that you've already built with ChatGPT. Interesting. When we filmed the first podcast, it hadn't come out yet. [00:18:57] S2: And then
00:18:57 · Anish Acharya, David Pawlan, Josh Elman
[00:18:57] S2: And then [00:18:58] S1: I think a few days later, it it came out. That's right. By and large, most people I talk to still are not using it as their preference of the 300... I did a survey of, you know, AI power users, 354 people. Only 3% were actually using it as as their primary agent. However, it does have this kind of added value of the the main thing that people pointed to as to why they really preferred it and why they liked it and why they wanted to try it was because it has all this building context. Yeah. A view just from your chat threads. People are using ChatGPT as a therapist, and so it has so much context on you. [00:19:29] S2: Interesting. [00:19:30] S1: Where it's gonna know that when you say, well, I'm not cheap. It knows it's like [00:19:33] S2: You're a little cheap. A [00:19:35] S1: little bit. A little bit. [00:19:35] S0: I'm curious why people think that ChatGPT has this great set of preferences from... On them. Did you actually get any sense if they said they really like dots, like, what they had been giving to ChatGPT over the years? And do you think they think that's replicable? Or do you think it's like, hey. Once it's already been a few years, we're now in this point where, like, my memory's there. Why would I switch?
00:19:35 · Josh Elman, David Pawlan
[00:19:35] S0: I'm curious why people think that ChatGPT has this great set of preferences from... On them. Did you actually get any sense if they said they really like dots, like, what they had been giving to ChatGPT over the years? And do you think they think that's replicable? Or do you think it's like, hey. Once it's already been a few years, we're now in this point where, like, my memory's there. Why would I switch? [00:19:56] S1: It's totally replicable. It... It's it's just the matter of fact that they've been using it the longest, and so it has the most context on them. But then that brings up another good point of the the friction to switch. Yeah. And I I ran the test across the top 15 agents on the benchmark of which ones can you actually pull context from and say, like, hey. I wanna switch to another one. Give me an empty of all of my memory that you have so I can switch easily. All but one we're at... We're actually willing to do it. And so you could just get a memory file, everything you need, submit it to the next one. [00:20:24] S0: And was it good, or was it uncanny? [00:20:26] S1: It's it's pretty solid. I mean, it's... Obviously, I'm sure they're they're caging some some of some of the fruits of of your labor. Yeah. But by and large, it it did really break everything down.
00:20:26 · David Pawlan, Anish Acharya, Josh Elman
[00:20:26] S1: It's it's pretty solid. I mean, it's... Obviously, I'm sure they're they're caging some some of some of the fruits of of your labor. Yeah. But by and large, it it did really break everything down. [00:20:35] S2: I'm not sure if you've ever looked at any of those markdown files, but it's a little... Terrifying is the wrong word, but it's like, wow, you really got your finger on me. Yeah. Kinda like chafe against me. Like, may... I don't want you to know me as well as you seem to think you know me. [00:20:49] S1: Yeah. After the first few, I just... I decided to stop reading them. [00:20:51] S2: Yes. Yes. Yes. Yes. It's actually a really interesting and now common onboarding pattern where it will say, hey. Go and paste this prompt into ChatGPT. Download the file. Upload it to me. Yep. We'll see how long that lasts. [00:21:03] S1: Totally. Another cool onboarding thing I've seen is you jump on... Like, it asks you to jump on a call. Yeah. And it's like, hey. Let's just just talk to me for a few minutes. Yeah. Yeah. Ask some questions. And it's it's really interesting to see what context it can it can pull from those questions. [00:21:18] S0: And they're very disarming. Like, they just get you Like, it's kinda crazy. [00:21:21] S1: Totally. But I think I think it brings up another variable of context, which is photos. Right? Like, this this new paradigm of using photos to pull an incredible amount of context on you in ways that you wouldn't even think to describe. Yeah. I mean, I
00:21:21 · David Pawlan, Josh Elman, Anish Acharya
[00:21:21] S1: Totally. But I think I think it brings up another variable of context, which is photos. Right? Like, this this new paradigm of using photos to pull an incredible amount of context on you in ways that you wouldn't even think to describe. Yeah. I mean, I [00:21:36] S0: think have you you tried Dazzle? Has Dazzle gotten into your benchmark set yet? That's from Marissa Mayer's new company. [00:21:41] S2: Yeah. [00:21:41] S0: And I think the idea of, like, looking over your entire life that you've captured, that have been moments that are important enough to you to actually save or screenshots or other things [00:21:49] S2: Yeah. [00:21:50] S0: Tell so much about somebody. [00:21:51] S1: An incredible amount. It it hasn't been added to the benchmark yet. It is quickly on the the top of the list to test through. I'm still manually testing all these, so I actually get the personal experience and and understand what the feeling is with each one. But what I really like about how she's approached it is they put a lot of emphasis on this trust layer, and because I think it's a huge variable is I'm giving you all my photos. [00:22:13] S0: That's Yeah.