00:13:28 · Mati Staniszewski, Joubin Mirzadegan
[00:13:28] S0: that Lam quick back... Question back, is it across all use cases you can tell, or you can... Like, know, like, it would try... Writes a copy for your email or writes a copy for your post? Or is that... What what... Across all settings or just specific settings? [00:13:41] S1: I think unless they... Somebody has to really I think the only way that I generally can't tell is if somebody writes down what they first thought, then puts it into the LLM to to tweak what they thought, and then goes back and then, like, unbundles the things that sound like an LLM. [00:14:00] S0: Right. [00:14:01] S1: But I think that's across every use case. I I have seen I have seen that. Yeah. Do you agree, by the way? [00:14:08] S0: No. I don't think... I I I think I... Yes. In many settings, it's obvious where somebody is an LM and some of the things you mentioned make it obvious, but, of of course, like, there's something about a style. Like, if somebody goes the lazy effort of, hey. Write this for me. Immediate tell of, like, it's easy. But but I see more common the kind of the last thing of what you mentioned. So somebody will write the thoughts, get the LLM, rewrite the thoughts so they are, like, more human. And and I don't know if we are classifying that in the same category, but in those cases, I think it's very not... Or it can be very good. Like, I'm sorry. [00:14:42] S1: I classify it in that same category. I'm just saying if you tell an LLM to write something for you, just like if you tell an LLM to speak something for you [00:14:51] S0: Got it.
00:14:51 · Mati Staniszewski, Joubin Mirzadegan
[00:14:51] S0: Got it. [00:14:51] S1: Is it always gonna sound like an LLM? [00:14:53] S0: Yeah. The... So I think if you just use, like, a default, likely, yes. I do think now you can kind of, like, preprompt it where it becomes becomes a little bit less of a tell. And in some settings, it's it's it's it's it's as good or better than the the human would. And let... Let's let's take a customer support setting where we work a lot. If you're working in customer support, the baseline you are comparing it to is usually a IVR flow or, like, a decision tree of if this, then that, or a potentially human that has maybe not had as much of the training and and will try to help you but might not be able to help you. In that setting, I find ElevenLabs can provide as good, if not bad... Just on the text level, if better than human human experience where you get the support and, you know, you are comparing it to to something that's not there. [00:15:47] S1: Totally agree [00:15:47] S0: with you. [00:15:47] S1: If you're using Sierra and you're using ElevenLabs on the back end, and Sierra is partnered with United, and you're comparing the experience that United would give you pre and post, it's night and day.
00:16:02 · Mati Staniszewski
[00:16:02] S0: Yeah. But I'm thinking even on the human side. If you are comparing, you know, just... So even let's ignore voice, and and I have my my take on voice as well, where just on the text level, if you're comparing customer support beyond by AI agents versus humans, I think many people wouldn't be able to tell, and it's probably a better result altogether. If if we are thinking about, like, more of a creative case and, like, a narration for a movie or a conversational dialogue, I think there, yes, it's a little bit harder to get on the on, like, a high end part to get something comparable. On the voice side, on the uncanny valley piece, I think in the narration space, I think you can get the results from voices that people won't be able to tell are not narrated by human. And we've seen this across so many so many places where an audiobook created is just so good that people that people are, like, both immersed. And I don't think if you kind of ask them point back, is it better or worse, they would be able to to tell. In the conversational setting, I don't think we passed uncanny value for all types of use cases. I think we passed it in call center application. I think we passed it in in some of the customers that support experience use cases. But let's say in immersive media or gaming or personal AI type typespaces, I don't think it passed down kind of very, like, Turing test in all settings. Like, we are speaking now. We can interrupt each other at any point. We can move pretty quickly, pause. We can... There's, like, a reasoning and and voice that's combined to get us one. That moment has not yet, I think, happened for conversational setup.
00:17:39 · Joubin Mirzadegan, Mati Staniszewski
[00:17:39] S1: But you're convinced it will. [00:17:41] S0: I think it will. Yeah. I think it will. Hopefully, we make it happen. So... Yes. I'm biased as well, I guess. [00:17:47] S1: Yeah. You're biased. That's fine, though. Like [00:17:48] S0: But even VoteVoice. So you think in that conversational setting, you would be able to tell today, I presume, if you are messaging with an AI straight away that this AI or, like, after a few exchanges? [00:17:59] S1: Yeah. So I And you think it will happen? So I have I have two points that I'm making. One is certainly on on... In most cases, I can tell. Okay. Second, I don't even think it's a bad thing that I can tell. I guess what I'm really questioning is why do we want them to sound and feel like a human? Meaning, if I am chatting with United, okay, or if I'm on a phone call with United, I don't actually care if it's a human or an LLM on the other side. In fact, I would rather know. And what I care about is how quickly can you get to resolution. That's all I care about. And in fact, it would maybe give me a little bit of peace of mind if I know what's on the other side of the person that I'm talking to.
00:18:41 · Mati Staniszewski, Joubin Mirzadegan
[00:18:41] S0: Oh, no. I I so 100% agree on on on that part. Like, ideally, you do know. It's preempted, and it's it's a good experience. Right. So I think I think, like, yeah, I I I would decouple those two points too. It's like you want the resolution as quickly as possible. And then two, does being human voice or human experience help you to get to that resolution? In some cases, it does. In some cases, it doesn't. It's like in customer support, no. You want the content. You want the actions. You want the intelligence to be well connected. You want to know the information about you. You want to know you are speaking with AI agent to help you resolve that case. And then let's say you are in a gaming context. You are in an immersive game in Fortnite. That's a kind of example of of of a game we we worked with with Epic Games to bring Darth Vader experience to the game. You want to make it feel as emotional as possible, as real as possible. [00:19:28] S1: Just like Darth
00:19:28 · Mati Staniszewski
[00:19:28] S0: Vader. Just like Darth Vader. Exactly. And... Or, like, you know, by emotional here, you want a, like, pretty pretty raspy, a pretty serious tone of the of the experience. In customer support, you you you you don't, but then there there is a very different experience depending on who you are. So, like, you know, let's say you or I want... You wanna ask ask quick to the resolution, quick responses across across the conversation. We recently worked with a customer support company in health care in in Japan, and they had this case where a lot of their population calling in is older, and then they have a small proportion of people that are younger. It was effectively a widget on top of the of the website, and they had they had the information about who is reaching out for help. And both the voice part, but also just the style of how the information gets delivered was drastically different between those groups and led to much different results between between those groups. So if you had a younger person reaching out for help, you want it relatively quick. You want it relatively short in style, straight to the information. If you get an older person, you wanted to make it a slightly more calmer, less emotions, a lot longer responses, lot slower responses. So in this case, it really matter depending on on the context of who's calling in. And in the same way as you think about a human, we don't... You know, like, you have a human responding on customer support in that same client, what you really wanted to do is to be able to alter depending on who you are speaking with and what type of experience you deliver and then what leads to the resolution. Resolution. And that's where I think it matters that you make agents versatile to help across all those different use cases. I don't think we should force them to be emotional or human in any of those cases. It's more of, like, how does it be able to be... Is able to adapt to the to the use case at hand and kind of bring it to the resolution in a in a way that's understandable.