02:51:08 路 John Coogan, Bret Taylor, S5
[02:51:08] S1: Talk about pilot programs and conversion to real customer. Is that timeline a KPI for you? Is that something that you monitor very closely? I have to imagine that it's speeding up. But whenever you shake the hand of a CEO and they say, Let's do this, or someone in the federal government perhaps, an agency lead and they say, I'm ready. I'm I'm bought in on the vision. The demo looked good. But then there's obviously an implementation time. Does that look What like did it look like a year ago? Where is it going in the future? [02:51:43] S2: We track it and we measure it in days not weeks. [02:51:47] S5: And [02:51:47] S2: so, you know, some of our favorite like Nordstrom, know, from, you know, concept to a 100% of their phone calls was thirty five days. [02:51:56] S1: That's good. Said three thousand days is good, then we're back to, oh, you're game in the stats.
02:52:02 路 Bret Taylor
[02:52:02] S2: What's the thing about that too is we started out at 1% of their phone calls in I think four weeks and went to a 100% in just a week and that's because of the Nordstrom team. Yeah. You know, actually in Fiserv, which is an amazing, you know, financial technology company for banks, just broke the record. So I don't know if I'm at liberty to show the number of days Yeah. But That's they have an amazing president named Divya. She was just top down, like, we're gonna do this and broke the record. So we have a leaderboard internally about how quickly when a customer talks to us can we go live and start delivering value. For what it's worth, I think it's really relevant in this world. You can Financial Times, Wall Street Journal, it's all, you know, token maxing. Yeah. What value are we getting from the tokens? Like, is kind of in the headlines right now. We don't really have that issue with our clients because, like, our thing is, like, let's start answering the phone as quickly as possible, start driving those operating expense savings as quickly as possible, but as importantly, that you can actually improve sales, improve CSAT at the same time. Our you know, as you know, we've been really pioneering this idea of outcomes based pricing with the idea being you only pay Sierra when we successfully resolve a call or successfully make a sale. It's it sounds like a small business model thing, but it's big because we don't get paid until it's live. So Yeah. It really aligns like all the incentives. Like, we don't care about the sales process. We care about the go live process.
02:53:23 路 John Coogan, Bret Taylor, Jordi Hays
[02:53:23] S1: Sure. And I [02:53:24] S2: think it really changes, I would say, almost the social dynamics between vendor and customer where we really become a partner because we have the same incentives as our clients. And I think it's a really big shift and you know how much I'm an advocate for it, but I actually think a lot of the, you know, token maxing issues would go away if more companies were just focused on outcomes. [02:53:43] S0: We're How are you tracking the popularity of the message talk to a human? Or the statement. [02:53:52] S1: No. No. When I pick up the phone I say talk to an agent, agent, and then they put me on the phone with an AI agent. It's great.
02:53:59 路 Bret Taylor
[02:53:59] S2: Yeah. No. Thank you for that. You're on my side. Yeah. So we actually have a name for it. We call it greeting acceptance, which is basically what percentage of the time do people get past the greeting and actually give the AI agent a shot. It's going up. People in general I think the problem was we had ten, fifteen years of bots that were really bad. And so we're just you're just, like, paying down this debt because if you were talking to an AI more than three years ago, it sucked. Like, let's just be blunt about it. Now you can talk to an AI and it's conversational, it's multilingual, it has access to systems. I mean, the AI agents of today are just, like, a completely dead it's like horse and carriage versus flying car. Right? They're just very different technologies. The problem is in the first five seconds of a call, you're wondering, is this one of the crappy ones or is this like one of the good ones? And so we have a lot of techniques on our platform to help people with that. I think the real key is making sure the first greeting is personalized, is engaging, is empathetic. And I think the trend is in the right direction. I think over time I mean, don't know if you have this experience. If I have to call a restaurant to make a reservation, I'm like, I'm I'm going to a different place. Like, if you don't have OpenTable or Resi or whatever, I'm out. I think we're gonna get to the point where people demand an AI because they're not gonna wanna wait on hold, you know? And what's also interesting is these AI agents, once they're properly configured, are faster for me as a consumer. They have access to my information. They can get the things done quickly. We're not a 100% there and as I said, we're just paying down the debt of bad tech that we've had for the past ten years. But I'm hopeful Yes. In three or four years it will be the norm.
02:55:30 路 Jordi Hays, John Coogan, Bret Taylor
[02:55:30] S0: When did it become popular to to get the, like, there's a thirty minute wait, like press 1 if you want a callback. Because I felt like that wasn't a thing when I was a kid, but now that feels like a pretty meaningful innovation at some point in the last Yeah. Ten years where it actually is really convenient to not be like, okay, like, I'm on I'm just gonna Some [02:55:51] S1: guy has a patent on that and [02:55:52] S0: has I'm I'm I'm sure, but like that was like the last great innovation Yeah. In customer service. Yeah. [02:55:58] S1: It was. [02:55:59] S2: Well, I think it's going in a really good direction. I mean, I think, you know, not having to wait on hold. We should never have to wait on hold again. Like, that is gonna be a thing of the past. [02:56:07] S1: But the GPUs are on fire. It's like, we'll call you back when the GPUs call like, cool off. We're load balancing right now. [02:56:15] S2: Please wait on hold. Yeah. [02:56:16] S1: Maybe. Imagine that world. That is a bizarre outcome. I mean on that on that note, are are are there any are there any like how how close are we to thinking about like a six for voice models or we were talking to Matthew Prince from Cloudflare about edge computing for voice models. That feels more relevant than, yes, if you're going to cook for an hour on some deep research project, go do it in the on the NBL 72. But how are you actually at the point where you're starting to think about that optimization or is that more a few years away?
02:56:49 路 Bret Taylor
[02:56:49] S2: I think it's probably not a few years away, but it's definitely in the future. But I'm excited for it. I actually agree with your intuition that, you know, we're entering a world where first, the frontier models are actually separated in my opinion. GPT five five, the recent releases from Anthropic, I mean, you can just see it like Yeah. The gap between the best models like GPT five five and the the others, the open source models is growing not shrinking, which is probably not something we would have predicted a year ago or at least, like, certainly not what I saw in X and and the press. But we've reached sufficient intelligence for a lot of different domains. Like, you don't need, you know, a multi billion dollar multi billion parameter account model to do a simple classification or, you know, just do basic, you know, transcription. So as a consequence, I'm hopeful that over the next few years we'll end up with a constellation of models with really purpose built ones. And as you said, just given the power of the hardware in our pockets right now, I think edge will be a part of it. There's just no doubt. And it and it just stands to reason there might be creative ways to, you know, maybe convert what you're doing into tokens, you know, so that you have, like, lower bandwidth connection if you're doing voice. Could there be opportunities there? There also could be some privacy benefits to that. I I, know, I have a strong intuition, though no knowledge, but just intuition. That's gotta be something Apple's thinking about just given their their posture on privacy. So I'm very bullish on the frontier models like, you know, OpenAI and Anthropic have, you know, really shown the the strength of the research groups, but not at the expense of all the specialized models. I think we're in a in a world where, you know, just as we become more sophisticated in deploying AI, we're going to have a constellation of models with just different capabilities. So I'm excited for it, but it's not now. It's it's in the the short term future.