Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy
[00:00:00] S0: In the spirit of that question of what could go right, what could go right for the company based on the data they're gathering from these interactions that they probably have been doing nothing with historically? What new things can they do for their customer—because on a one-to-one basis, they're just learning more about a person, and on the aggregate basis they understand the behavior patterns of their customer base or something?
· Jesse Zhang
[00:00:00] S1: Oh yeah, that is a huge topic. I think that's a huge part of our product. We have a viewpoint that this data of course is super valuable because it's literally what your customers are saying. But it's very underutilized because historically this is very unstructured data. So what people typically would do is: okay, well every month we have a million conversations. We'll have a full-time team of 20 people and they're just sampling these conversations and trying to check on a rubric and try to compile topics and things. And that won't get you so far. But now what you can do is you can literally have a language model that reads every conversation and extract whatever info you want from it. And so that allows you to do things: okay, well, over time there are these topics that people probably didn't even know about because these organizations are big. So the people in leadership positions, they can only have such granular insight into what's happening. But it'll literally just flag: hey, there's this 2% of conversations where things are not really going that well. And it's because we don't have context on this topic. And so let's flag that. Let's draft a suggestion for what could go better here based on how the human agents are handling or based on the other procedures that we have. And here's a suggestion for how you should adjust the agent. And that allows the agent to improve automatically over time. And that's really critical. So when you think about moats in the agentic world, a lot of it is around: if you've been working with a client for a year, has your agent just continuously gotten better by learning from the data? And that's a different concept than just training on the data. But has it continuously gotten better to the point where it's just very difficult for another agent to come in and perform at the same level?
· Patrick O'Shaughnessy
[00:00:00] S0: There's this funny situation today where I think especially CEOs really want AI in their businesses. They want it now, but they don't know what they want. They don't have a good framework for thinking about, okay, I understand my business—I don't know where to go. It seems I would be missing a major boat if I don't deploy this in my business. I don't know what to do. I don't know where to go first. I don't know who to call. Have you developed any framework for those company leaders that desperately want to use this technology in their business, but they simply don't know, apart from automating customer service or something? I don't mean specific use cases. I mean a framework for thinking about what kinds of problems might be addressable by agents or by LLMs.
· Jesse Zhang, Patrick O'Shaughnessy
[00:00:00] S1: Oh, interesting. I mean, one framework is similar to the sort of framework you talked about before of two ends of the spectrum. And I would say most leaders we talk to are focused on the more bottoms-up end of the spectrum, which is: where are the areas that we should just not have humans doing because it's so mundane and repeatable and there are tons of cost efficiencies there? So I would say that's where folks are typically thinking of. So when we talk to leaders, I think there are a couple observations. One, pretty much all AI initiatives are very top-down at this point because it is such a board-level mandate. So C-suite's very, very invested in: okay, where do we deploy AI? That almost means that if you want to get something going at a larger organization, you have to have buy-in from the top level because it's going to get up there anyways and they have to make the decision at the end of the day. So that's one. Two, the way they think about the use cases to your point is back to ROI. It's: where can we either save much money or make a lot of new revenue? And if you cannot in basically half a sentence explain that, then it's just not going to work right now. They're under a lot of pressure, right? They need to show quick wins. If this is not going to be a quick win that can point to "I saved $10 million," then this is not going to be something that's prioritized. [00:00:00] S0: Do you think coding answers that well? Do you think the ROI is clear in coding?
· Patrick O'Shaughnessy, Jesse Zhang
[00:00:00] S0: Do you think coding answers that well? Do you think the ROI is clear in coding? [00:00:00] S1: It is. And I know the coding agents quite well and the way they do it generally is one: It's very easy to test, to play out to the engineers. And then you just do a poll on the engineers: hey, how much more productive do you think you are? And engineers are often the most valuable resource in these organizations and so those answers are treated with high importance. Yeah, engineer tells you that they're 50% more productive. It's like okay, great. [00:00:00] S0: Are you confident that that's true? That they can self-report and be accurate? There was that meteor study or whatever that came out that actually I don't know if the study's good or not, but productivity was down or flat or something. The self-reported productivity was at odds with actual measured productivity or something. [00:00:00] S1: Oh yeah. Don't know nearly enough about that. But I'm just saying that it doesn't really matter if their entire engineering team is saying, hey, we love this. This is making us 50% more productive. It's super easy— [00:00:00] S0: Yeah—building your thing. [00:00:00] S1: Exactly. And then you think about it from the CEO and they're reporting this to the board or something. It's hey, my entire engineering org said that they're 50% more productive. This is a worthy investment because these people are all paid this much now we can accelerate the product. We can accelerate everything. Yeah.
· Jesse Zhang, Patrick O'Shaughnessy
[00:00:00] S1: Exactly. And then you think about it from the CEO and they're reporting this to the board or something. It's hey, my entire engineering org said that they're 50% more productive. This is a worthy investment because these people are all paid this much now we can accelerate the product. We can accelerate everything. Yeah. [00:00:00] S0: Do you think that the future is—I want to talk about brands and how a given company might want its agent to feel and sound that an agent might be a way to express brand culture, style, tone, whatever. I want to talk about that. But do you think the end state here is that each company sort of has almost a named personified representative that you just come to expect to interact with, and it's not just customer service issues, but it's sales issues? You ask it for advice on what shoe to buy or whatever it might be, and that that's all integrated or that you'll have different agents for different parts of the company? What I'm trying to ask is: what does the future of a company's agent or agents look like in the natural end state five years from now or something?