00:11:32 · Joubin Mirzadegan, Mati Staniszewski
[00:11:32] S1: Mhmm. One of the things that I've been thinking about that I wanted your opinion on, and maybe this is just, like, a very stupid a stupid thought, but, like, we have this concept here in Silicon Valley about, like, crossing what's called the uncanny valley. And the uncanny valley is this concept where basically machines become indistinguishable from humans. Right. You cannot tell if it's an LLM talking to you or writing to you versus a human. Right. And so far, certainly in text, I can, within ten seconds, tell if an LLM has written it or not. It's very, very obvious. Based on [00:12:05] S0: the em dash? [00:12:06] S1: Or Based on the em dash. Based on just the way that it writes. It's not even just the em dash. Based on [00:12:11] S0: No. No. I agree.
00:12:11 · Mati Staniszewski, Joubin Mirzadegan
[00:12:11] S0: No. No. I agree. [00:12:12] S1: It's Like, not even random... Like, forget about the bolding and the em dashes. Right? Because nobody actually writes like that. Just the way that it actually writes does not sound human to me. And it's very obvious the minute that somebody sends me something, immediately, I'm I'm like, you didn't write it. Just like something inside my brain is like, that's not a person that wrote this. Okay? And so maybe audio, if it's coming off of text, is a similar paradigm today. And everybody continues to say, oh, no. No. It's happening. Like, it's going to happen for sure. And one of the questions that I've been asking myself, like, you know, these models have gotten a lot better, and they're still getting better. And everyone's like, well, as long as it keeps getting better, it's definitely gonna not sound... Like, it's gonna sound is... Indistinguishable from us. And, you know, I've listened to the, like, podcast LM from Google. Like, I've listened to all of this stuff. I've done the use ElevenLabs to clone my voice. And I'm not even saying... It's not a knock. Like, I'm not even saying it should or shouldn't sound like me. I'm like, it... Maybe it shouldn't. Like, maybe if you're from India, your dialect should be Indian. Like, maybe if you're from Iran, your dialect should be something else. Maybe if you're from machines, your dialect should be machine. You know? Like, I don't I don't know. And and and on that on
00:13:28 · Mati Staniszewski, Joubin Mirzadegan
[00:13:28] S0: that Lam quick back... Question back, is it across all use cases you can tell, or you can... Like, know, like, it would try... Writes a copy for your email or writes a copy for your post? Or is that... What what... Across all settings or just specific settings? [00:13:41] S1: I think unless they... Somebody has to really I think the only way that I generally can't tell is if somebody writes down what they first thought, then puts it into the LLM to to tweak what they thought, and then goes back and then, like, unbundles the things that sound like an LLM. [00:14:00] S0: Right. [00:14:01] S1: But I think that's across every use case. I I have seen I have seen that. Yeah. Do you agree, by the way? [00:14:08] S0: No. I don't think... I I I think I... Yes. In many settings, it's obvious where somebody is an LM and some of the things you mentioned make it obvious, but, of of course, like, there's something about a style. Like, if somebody goes the lazy effort of, hey. Write this for me. Immediate tell of, like, it's easy. But but I see more common the kind of the last thing of what you mentioned. So somebody will write the thoughts, get the LLM, rewrite the thoughts so they are, like, more human. And and I don't know if we are classifying that in the same category, but in those cases, I think it's very not... Or it can be very good. Like, I'm sorry. [00:14:42] S1: I classify it in that same category. I'm just saying if you tell an LLM to write something for you, just like if you tell an LLM to speak something for you [00:14:51] S0: Got it.
00:14:51 · Mati Staniszewski, Joubin Mirzadegan
[00:14:51] S0: Got it. [00:14:51] S1: Is it always gonna sound like an LLM? [00:14:53] S0: Yeah. The... So I think if you just use, like, a default, likely, yes. I do think now you can kind of, like, preprompt it where it becomes becomes a little bit less of a tell. And in some settings, it's it's it's it's it's as good or better than the the human would. And let... Let's let's take a customer support setting where we work a lot. If you're working in customer support, the baseline you are comparing it to is usually a IVR flow or, like, a decision tree of if this, then that, or a potentially human that has maybe not had as much of the training and and will try to help you but might not be able to help you. In that setting, I find ElevenLabs can provide as good, if not bad... Just on the text level, if better than human human experience where you get the support and, you know, you are comparing it to to something that's not there. [00:15:47] S1: Totally agree [00:15:47] S0: with you. [00:15:47] S1: If you're using Sierra and you're using ElevenLabs on the back end, and Sierra is partnered with United, and you're comparing the experience that United would give you pre and post, it's night and day.
00:16:02 · Mati Staniszewski
[00:16:02] S0: Yeah. But I'm thinking even on the human side. If you are comparing, you know, just... So even let's ignore voice, and and I have my my take on voice as well, where just on the text level, if you're comparing customer support beyond by AI agents versus humans, I think many people wouldn't be able to tell, and it's probably a better result altogether. If if we are thinking about, like, more of a creative case and, like, a narration for a movie or a conversational dialogue, I think there, yes, it's a little bit harder to get on the on, like, a high end part to get something comparable. On the voice side, on the uncanny valley piece, I think in the narration space, I think you can get the results from voices that people won't be able to tell are not narrated by human. And we've seen this across so many so many places where an audiobook created is just so good that people that people are, like, both immersed. And I don't think if you kind of ask them point back, is it better or worse, they would be able to to tell. In the conversational setting, I don't think we passed uncanny value for all types of use cases. I think we passed it in call center application. I think we passed it in in some of the customers that support experience use cases. But let's say in immersive media or gaming or personal AI type typespaces, I don't think it passed down kind of very, like, Turing test in all settings. Like, we are speaking now. We can interrupt each other at any point. We can move pretty quickly, pause. We can... There's, like, a reasoning and and voice that's combined to get us one. That moment has not yet, I think, happened for conversational setup.
00:17:39 · Joubin Mirzadegan, Mati Staniszewski
[00:17:39] S1: But you're convinced it will. [00:17:41] S0: I think it will. Yeah. I think it will. Hopefully, we make it happen. So... Yes. I'm biased as well, I guess. [00:17:47] S1: Yeah. You're biased. That's fine, though. Like [00:17:48] S0: But even VoteVoice. So you think in that conversational setting, you would be able to tell today, I presume, if you are messaging with an AI straight away that this AI or, like, after a few exchanges? [00:17:59] S1: Yeah. So I And you think it will happen? So I have I have two points that I'm making. One is certainly on on... In most cases, I can tell. Okay. Second, I don't even think it's a bad thing that I can tell. I guess what I'm really questioning is why do we want them to sound and feel like a human? Meaning, if I am chatting with United, okay, or if I'm on a phone call with United, I don't actually care if it's a human or an LLM on the other side. In fact, I would rather know. And what I care about is how quickly can you get to resolution. That's all I care about. And in fact, it would maybe give me a little bit of peace of mind if I know what's on the other side of the person that I'm talking to.