Invest Like the Best with Patrick O'Shaughnessy
· Jesse Zhang, Patrick O'Shaughnessy
And you can choose, okay, for this use case, we really need it to be over here. That's a lot more robust. But for other use cases where if you're just asking basic questions on their account, we don't want it to be that. We want it to be super free form and that's how we get the customer satisfaction up. [00:00:00] S0: I think most people are still focused on things that could go wrong because it's non-deterministic. What about the total other end of the spectrum? What have been the things that have gone way more right than you expected? Where has the potential of agents outperformed your expectation in terms of what they can handle or what they can do?
· Patrick O'Shaughnessy, Jesse Zhang
[00:00:00] S0: I think most people are still focused on things that could go wrong because it's non-deterministic. What about the total other end of the spectrum? What have been the things that have gone way more right than you expected? Where has the potential of agents outperformed your expectation in terms of what they can handle or what they can do? [00:00:00] S1: It's really just elevating the experience. One sort of metric, which is usually a secondary metric that folks think about later but is quite important to us, is just how often do people come in and just say, "Agent, agent, agent, give me to a representative, I don't want to talk to you." I've done that. You probably have as well where you're just calling into some sort of customer service and you're just pressing zero the whole time. And that's because people are used to bad experiences so they've already lost the trust of these systems. I think what surprised us was that if you just make it really clear off the bat that this is a different experience, people are willing to give it a chance and then the outcomes are just way different. One of our customers, Oura Ring, the wearable ring, we did a case study with them where before having any sort of GenAI system, one in three customers that came in would just not bother saying anything and just keep jamming agent until they got to one. Now it's one in 20 because we just spent a lot of time making the beginning of the process just feel very different and folks are willing to give it a chance. So I think that's been exciting.
· Jesse Zhang, Patrick O'Shaughnessy
[00:00:00] S1: It's really just elevating the experience. One sort of metric, which is usually a secondary metric that folks think about later but is quite important to us, is just how often do people come in and just say, "Agent, agent, agent, give me to a representative, I don't want to talk to you." I've done that. You probably have as well where you're just calling into some sort of customer service and you're just pressing zero the whole time. And that's because people are used to bad experiences so they've already lost the trust of these systems. I think what surprised us was that if you just make it really clear off the bat that this is a different experience, people are willing to give it a chance and then the outcomes are just way different. One of our customers, Oura Ring, the wearable ring, we did a case study with them where before having any sort of GenAI system, one in three customers that came in would just not bother saying anything and just keep jamming agent until they got to one. Now it's one in 20 because we just spent a lot of time making the beginning of the process just feel very different and folks are willing to give it a chance. So I think that's been exciting. [00:00:00] S0: Where do you think that can go? How good can the experience get in ways that it's not yet that good with subsequent evolution of your product but also of the underlying capabilities of the model?
· Jesse Zhang
[00:00:00] S1: The biggest frontier right now is voice. Voice models and so now there's a lot of interesting startups as well working on voice. It's exciting because it's still, I would say definitely not solved. There's a lot to be done there. The bar is very high. So if you just think about how humans communicate for literally the entirety of humanity, I don't know, 150,000 years or something, the UI for every human is language, spoken language. Listen, you speak. And that's how our brains have evolved. That's the most natural way for us to communicate. Only in the last, what, 60 years of that entire time, did we have keyboards and phones and communicating through typing? I would say fundamentally, in any sort of agent that communicates with humans, voice has to be a critical factor because that's just how we communicate, because our brains are so evolved for this. It's very easy to tell when something feels not quite right. And so the bar is very high. Uncanny Valley is quite large. That's why there's a lot of effort going into making the voice experiences good. I would say ChatGPT voice, for example, or the Seamless or these voice-to-voice models, they're starting to feel very impressive. But if you talk to it long enough, you can actually—you can definitely tell it’s not a human, so there's that element of it. But then, even for enterprise use cases us, there's still a ton of hurdles to cross because those models, even though they're good, the hallucination rate's really high, so you can't really use them necessarily as is in the current systems. And so a lot of people, what they do now is they go from voice into text and then back to voice, and then you can run a lot more checks there to make sure that things are accurate. And so a lot of cool ideas there to explore on how do you make it both human-like, but accurate, and how do you tie everything together. So that's where most of the work is going to these days.
· Patrick O'Shaughnessy, Jesse Zhang
[00:00:00] S0: At the risk of getting too technical, why is voice-to-voice interesting and worth pursuing versus just always going back to text and being able to manage that way? [00:00:00] S1: So the fundamental difference is if you're just going to text, then no matter what, the final audio is just a narration of the text. The voice-to-voice is powerful because it takes into the entire audio of what you said, so it knows cadence and maybe how upset you are and the tone and everything. Latency is a lot less as well because you're going straight from voice to voice and latency matters so much when we're talking. When we're talking right now, our brains are constantly going, okay, when's he done talking? When should I start talking? If someone interrupts someone else, it's in a polite way, people adjust very naturally. So that is the biggest proponent of voice-to-voice. And ultimately, I think the prevailing view is that for whatever the final experience is, if you really want to make it indistinguishable from a human, you have to do voice-to-voice, or you have to at least take into account the voice. The issue with voice-to-voice though, is that also fundamentally because voice has a lot more dimensions to it, the amount of tokens you generate per sentence is just a lot higher than when you generate text. The more tokens you have, the easier it is for something to go wrong. And so the hallucination rate has so far just been a lot higher. [00:00:00] S0: How much higher? Give us a sense of how far we are from these being really good.
· Patrick O'Shaughnessy, Jesse Zhang
[00:00:00] S0: How much higher? Give us a sense of how far we are from these being really good. [00:00:00] S1: I think probably 8x higher or something that. [00:00:00] S0: Wow. [00:00:00] S1: It is quite a bit higher. Of course you want to leverage that technology. So now maybe there's creative ways to make a hybrid of the two. Maybe you can have a text model generate the content, but you take into account the audio from before as well, and that makes something very realistic. But at the same time, latency is still the hard problem because at the enterprise what's happening is you are doing a lot. Before you can start responding, you have to figure out what are they asking about? What materials do I need to collect, do I need to hit any APIs and get that data back? And so you have to do it in a way that feels very natural. And sometimes if you think about how a human does it, you might have to say something such as, "Give me a sec to look that up," because it actually genuinely takes 10 seconds for the API to come back. These are all interesting problems to think through. [00:00:00] S0: So give us a sense today: if you add up all the interactions, some idea of how long they are, what type they are, voice versus text versus some other modality. What does the entire corpus of interactions between a Decagon agent and a customer look like today?