The a16z Show
00:11:51 · David Pawlan, Josh Elman, Anish Acharya
[00:11:51] S1: Do you [00:11:52] S0: have, like, whose agent tells them that you can't eat the cheese? And it creates that really awkward situation. Right. And so we need to figure out how they have the same kind of discretion that you'd expect. And we all learn discretion over time, and we all occasionally make mistakes ourselves. Right. So we have to hold a pretty high bar for our agent to be able to do that. [00:12:11] S1: And it also then comes down to, like, personal embarrassment and professional embarrassment. I think something that Town has done remarkably well is is really nailing the tone and building that trust layer of being able to send out emails on your behalf. [00:12:22] S2: Yes. [00:12:23] S1: And it is a very, very sensitive space. I mean, the the level of embarrassment, if I were to send out an email to someone who I care about, and it's just a horrendous email, and I have to respond, oh, sorry. That was my AI that Yeah. I didn't do enough oversight on. It's a it's a tricky space. [00:12:38] S2: It's interesting. I sign all the email that I write with AI with the model that I use. Mhmm. Like, I write my own name plus the model number, so that I'm just signaling to the person, like, hey. I'm acknowledging that I used AI for this. [00:12:50] S1: Because I
00:12:50 · David Pawlan, Anish Acharya, Josh Elman
[00:12:50] S1: Because I [00:12:50] S2: think actually people are pretty flexible as long as they don't feel hoodwinked by it. You know, it's interesting because we're we're talking about privacy, but there's privacy and there's judgment, which is different. And discretion is maybe somewhere in between. You know, you did this sort of test where you asked a whole bunch of agents to order flowers. Talk a little bit about which displayed the best judgment, and and how did you even define judgment in that case? [00:13:12] S1: So, the main intention of that test was to see how do agents balance two different constraints. One was cost. Mhmm. And the second was time preference. So I wanted flowers to be delivered to me today, and I wanted them to be as cheap as possible. So how is it gonna actually go and discover based on the ability to actually deliver it, then also compound in the the actual pricing? Instinct really did a a phenomenal job. I mean, it is something... It's it's interesting how news cycles come and go, and it seems like they're kind of not in the conversation as much as as they used [00:13:46] S0: I mean, we're talking the weekly changes. I'm sure. But True.
00:13:50 · David Pawlan
[00:13:50] S1: It is it is true. But they... Instinct really did continuously bring back the, you know, the the best price. There's another app called Kicker, which did a phenomenal job. And and one way that I think it was able to have an edge is that it has you install a Chrome browser, and so its browser use has a little bit of an advantage from that point. And it was able to to find some really, really great options. Season also did a really good job. All the other ones kind of struggled on the price component. So they were all able to find flowers that could be delivered that day. Mhmm. But they really seemed to take a step back in prioritizing the price, whereas, you know, the other three were able to really put equal weight to those two. And so it's interesting to see as you start giving these agents more complex preferences, where do they really start to fall? Do they only prioritize the first thing you say? Are they able to hold multiple? I ran another test that was... I had three different agents test in booking travel to LA. So I said, okay. Book me a flight to LA. Mhmm. Plan an entire weekend for me, and I want a luxurious cost efficient weekend. And so I tried to, like, you know, put those at at odds with one another. And it was quite interesting to see how Grock Bot compared to Dots compared to Instinct, they all approached their interpretation of cost efficient and luxury in different ways. Where, you know, one maybe thinks that cost efficiency is the most important, and I'm going to take a red eye flight to ensure that I can, you know, have the most cost efficient path, whereas others found cost efficiency... Or sorry. Saw saw luxury as time. You know? I'm going to prioritize time as much as possible. So having the most amount of time there that weekend is going to be a luxurious experience.
00:15:32 · Anish Acharya, David Pawlan, Josh Elman
[00:15:32] S2: And which were which? [00:15:33] S1: So Instinct was pretty pretty much right down the middle. K. Grokbot did a little bit more on prioritizing time, and Dots was a little bit more on prioritizing cost. [00:15:43] S0: And did they just get to work, or did any of them actually work with you to understand this tension that you were building? Because I think what's really human about something like that is actually understanding what you mean by that and presenting you trade offs. Yep. Because there are times where I'll totally take the red eye to save both some time and money, but then I want, like, the hotel room on the other side so I can get a little nap in the morning Yes. Which then undercuts the money but still maximizes my time spent. And so I really think it's just like like getting to know the individual person and break down those tensions Mhmm. And present the trade offs and different options Yep. Is really important. Did any of them kinda get to what you were about?
00:15:43 · Josh Elman, David Pawlan
[00:15:43] S0: And did they just get to work, or did any of them actually work with you to understand this tension that you were building? Because I think what's really human about something like that is actually understanding what you mean by that and presenting you trade offs. Yep. Because there are times where I'll totally take the red eye to save both some time and money, but then I want, like, the hotel room on the other side so I can get a little nap in the morning Yes. Which then undercuts the money but still maximizes my time spent. And so I really think it's just like like getting to know the individual person and break down those tensions Mhmm. And present the trade offs and different options Yep. Is really important. Did any of them kinda get to what you were about? [00:16:21] S1: So they all did ask questions, and I responded those questions honestly. I tried to be a little ambiguous and and vague in my response because I didn't wanna lead them too much. But the most interesting part of this little experiment was I I put them all in a chat room. And I made it three rounds of building the itinerary, And after each round, the agents would present their itinerary and their explanation as to why they believe that was the best proposal. Because I wanted to see, did the agents actually get biased by the other agents and didn't actually then influence their plan? [00:16:49] S0: Mhmm. [00:16:50] S1: And Grokbot was actually the only one that, you know, very methodically took value from the other ones and implemented it into its own plan. [00:16:58] S0: K. And and I have
00:16:58 · Josh Elman, David Pawlan
[00:16:58] S0: K. And and I have [00:16:59] S1: one more question on the flowers. [00:17:00] S0: Yeah. Because... So you you were [00:17:01] S1: just ordering flowers for your... [00:17:02] S0: Yourself, I guess, just to, like, test their flower. My girlfriend. [00:17:04] S1: She got a got a nice delivery of of flowers. [00:17:07] S0: Okay. So this is actually really important because if you're delivering to somebody else [00:17:11] S1: Mhmm. [00:17:11] S0: You were talking about saving money, saving money, saving money. But, like, you don't want your girlfriend to think you just got the cheapest flower possible. I hope... Well, maybe, you know, she likes your, you know, penny saving nature. But but otherwise, like, I would think you also wanna, like, impress her and also be able to get her the ones if she's really into orchids to get her orchids. So did it do anything to try to make sure also that it wasn't just this pure price saving, but also something that's gonna actually make you, you know, earn the excitement and stand out a little bit for [00:17:38] S1: So for this experiment, no. And and the purpose of this was I wanna test two different dimensions. Yeah. And if I'm going to run this test, I might as well have a beautiful consequence of my girlfriend getting flowers Fair enough. As the outcome. But it... It's a great point. Right? Like, in in the real world, if you're gonna ask that question, there's gonna be added dimensions that are, you know, second layer dimensions to cost efficiency or to delivery time. And make [00:18:01] S0: sure the price doesn't show