Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy, Jesse Zhang
[00:00:00] S0: How much higher? Give us a sense of how far we are from these being really good. [00:00:00] S1: I think probably 8x higher or something that. [00:00:00] S0: Wow. [00:00:00] S1: It is quite a bit higher. Of course you want to leverage that technology. So now maybe there's creative ways to make a hybrid of the two. Maybe you can have a text model generate the content, but you take into account the audio from before as well, and that makes something very realistic. But at the same time, latency is still the hard problem because at the enterprise what's happening is you are doing a lot. Before you can start responding, you have to figure out what are they asking about? What materials do I need to collect, do I need to hit any APIs and get that data back? And so you have to do it in a way that feels very natural. And sometimes if you think about how a human does it, you might have to say something such as, "Give me a sec to look that up," because it actually genuinely takes 10 seconds for the API to come back. These are all interesting problems to think through. [00:00:00] S0: So give us a sense today: if you add up all the interactions, some idea of how long they are, what type they are, voice versus text versus some other modality. What does the entire corpus of interactions between a Decagon agent and a customer look like today?
· Jesse Zhang
[00:00:00] S1: I would say pretty balanced at this point between chat and voice on a raw customer basis. There are more people in chat, at least for us. But if you just think about the large enterprises—the Fortune 100—they've just been around for so long and everyone just calls them. So voice is just disproportionately higher there. A lot of them are 90, 95% voice and 5% chat. So that's what we're seeing. And then in terms of the types of conversations, it's generally ones that are fairly... you start with the easier one, of course. And so these are things where they're question-answer based. So that's the tier one: hey, you answer their question based on what you statically know. So that could be questions about how your loyalty system works, or questions about if I bought something, would I still be able to refund it? Then the next level is it's still question-and-answer based, but you're leveraging a lot of real-time data. So that could be: I got 2x points on this transaction, but I should have gotten 5—why is that? And then it would actually go and look into your account and reason through things: okay, I see the account's listed this type, let me go find all the documentation on this type. And okay, actually it's because you booked through a travel agency. If you had booked directly with the airline, you would have gotten 5x, but this thing doesn't apply to a travel agency or whatever. And then the third tier is you're actually taking action. So I lost my credit card, I need a new one. And it's actually walking through a pretty large flow. And that's where AI agents have been really excellent because you actually wouldn't expect it to be able to do that. And so it's able to go in—it can be a pretty complicated system where it's: okay, well first I need to figure out what your address is and confirm if the address is correct. And then I need to look and see, hey, do you want me to lock the old card? Okay, great, I'll do that. I might need to check for fraud to make sure this person is not just constantly asking for new cards. And it's just all these things stitched together—that's really what makes it agentic. And that's why there's been such a step-function improvement with LLMs.
· Patrick O'Shaughnessy
[00:00:00] S0: In the spirit of that question of what could go right, what could go right for the company based on the data they're gathering from these interactions that they probably have been doing nothing with historically? What new things can they do for their customer—because on a one-to-one basis, they're just learning more about a person, and on the aggregate basis they understand the behavior patterns of their customer base or something?
· Jesse Zhang
[00:00:00] S1: Oh yeah, that is a huge topic. I think that's a huge part of our product. We have a viewpoint that this data of course is super valuable because it's literally what your customers are saying. But it's very underutilized because historically this is very unstructured data. So what people typically would do is: okay, well every month we have a million conversations. We'll have a full-time team of 20 people and they're just sampling these conversations and trying to check on a rubric and try to compile topics and things. And that won't get you so far. But now what you can do is you can literally have a language model that reads every conversation and extract whatever info you want from it. And so that allows you to do things: okay, well, over time there are these topics that people probably didn't even know about because these organizations are big. So the people in leadership positions, they can only have such granular insight into what's happening. But it'll literally just flag: hey, there's this 2% of conversations where things are not really going that well. And it's because we don't have context on this topic. And so let's flag that. Let's draft a suggestion for what could go better here based on how the human agents are handling or based on the other procedures that we have. And here's a suggestion for how you should adjust the agent. And that allows the agent to improve automatically over time. And that's really critical. So when you think about moats in the agentic world, a lot of it is around: if you've been working with a client for a year, has your agent just continuously gotten better by learning from the data? And that's a different concept than just training on the data. But has it continuously gotten better to the point where it's just very difficult for another agent to come in and perform at the same level?
· Patrick O'Shaughnessy
[00:00:00] S0: There's this funny situation today where I think especially CEOs really want AI in their businesses. They want it now, but they don't know what they want. They don't have a good framework for thinking about, okay, I understand my business—I don't know where to go. It seems I would be missing a major boat if I don't deploy this in my business. I don't know what to do. I don't know where to go first. I don't know who to call. Have you developed any framework for those company leaders that desperately want to use this technology in their business, but they simply don't know, apart from automating customer service or something? I don't mean specific use cases. I mean a framework for thinking about what kinds of problems might be addressable by agents or by LLMs.
· Jesse Zhang, Patrick O'Shaughnessy
[00:00:00] S1: Oh, interesting. I mean, one framework is similar to the sort of framework you talked about before of two ends of the spectrum. And I would say most leaders we talk to are focused on the more bottoms-up end of the spectrum, which is: where are the areas that we should just not have humans doing because it's so mundane and repeatable and there are tons of cost efficiencies there? So I would say that's where folks are typically thinking of. So when we talk to leaders, I think there are a couple observations. One, pretty much all AI initiatives are very top-down at this point because it is such a board-level mandate. So C-suite's very, very invested in: okay, where do we deploy AI? That almost means that if you want to get something going at a larger organization, you have to have buy-in from the top level because it's going to get up there anyways and they have to make the decision at the end of the day. So that's one. Two, the way they think about the use cases to your point is back to ROI. It's: where can we either save much money or make a lot of new revenue? And if you cannot in basically half a sentence explain that, then it's just not going to work right now. They're under a lot of pressure, right? They need to show quick wins. If this is not going to be a quick win that can point to "I saved $10 million," then this is not going to be something that's prioritized. [00:00:00] S0: Do you think coding answers that well? Do you think the ROI is clear in coding?