00:16:01 · John Collison, Mati Staniszewski
[00:16:01] S0: Mhmm. [00:16:01] S1: And then a lot of audiobook authors or book authors that tried to... Couldn't afford Yeah. Professional innovation and wanted to create an audiobook. However, none of the companies accepted AI audiobooks. [00:16:12] S0: And you can't sell an AI audiobook on Audible or something. [00:16:14] S1: Exactly. So Audible would block AI content. So we had no choice. We need to create an avenue for them to... [00:16:21] S0: There was no distribution Exactly. For AI audiobooks. [00:16:24] S1: Exactly. So we created ElevenReader. Okay. And that kind of came with functionality where you can upload your PDF. You can upload your text and have it read out loud with a number of incredible voices. So whether it's sir Michael Caine all the way through to a state and working together with Sergei Feynman where where you [00:16:41] S0: got you were working with the sir Michael Caines of the world? [00:16:43] S1: Exactly. Okay. And then you can actually read it on that. And that kind of works extremely well. So that works. [00:16:48] S0: Yeah. [00:16:48] S1: Now how can you do it? Think Actually, I do [00:16:50] S0: want everything read to me by Michael Cain. [00:16:53] S1: It's a it's a great voice. Yeah. [00:16:54] S0: Shouldn't you guys have a consumer app where I can just do the common voice things? Like, I want to be able to have an Eleven app on my phone, and then if I upload a PDF to it, it can do the common things that I would like, such as have it read it to me. [00:17:06] S1: Yeah. That's that's exactly a reader. So that works.
00:17:06 · Mati Staniszewski, John Collison
[00:17:06] S1: Yeah. That's that's exactly a reader. So that works. [00:17:08] S0: Okay. The phone makers allow third party keyboards. Do you think they... Do they allow third party transcription engines? Will they, do you think? [00:17:17] S1: The phone makers, you said. Right? [00:17:18] S0: I mean, like Apple [00:17:19] S1: and and Google. Yeah. Yeah. They... So they're OS makers. Yeah. Not not not all of them. Android... And with Android, you can work through it. There's, like, you know, variations of that. Nothing. Tech and others. [00:17:29] S0: But, yeah, I feel like if you had a popular 11 app that allowed for transcription, people would use it a bunch, and maybe eventually Apple would say, oh, we should allow third party transcription engines if that's what people want. [00:17:38] S1: I mean, there is... It seems like they might be going in that direction. Right? And I recently... They announced that we'll open up the LLM ecosystem. Hopefully, they will do the same with voice ecosystem, which is kind of similar.
00:17:38 · Mati Staniszewski, John Collison
[00:17:38] S1: I mean, there is... It seems like they might be going in that direction. Right? And I recently... They announced that we'll open up the LLM ecosystem. Hopefully, they will do the same with voice ecosystem, which is kind of similar. [00:17:48] S0: But again, it's I think rational to do when it's moving so quickly. Yeah. The voice assistant paradigm is one of the oldest paradigm... You know, UI paradigms in computing, like the open the pod bay doors, Hal, from Yeah. '19 Yeah. '69. Yeah. I will claim it's not working yet. So Siri doesn't have the intelligence. And then on Gemini and ChatGPT and those apps, I mean, I want to use the voice mode, but I don't know about you. It just doesn't work. And so, like, sometimes I'll be using my phone, and I'll use the iOS keyboard transcription Yeah. Yeah. Yeah. To type in the field and then, like, say a bunch of stuff and then send it off. But this suggests to me that consumers really want voice mode that works, and yet it's just not working yet for the major LLM apps or for anyone. Why isn't it working yet?
00:18:41 · Mati Staniszewski, John Collison
[00:18:41] S1: It is pretty hard to do because you want you want two things. You want you want to be able to say things that you want, but you want sometimes for it to execute it, sometimes to wait for you to, like, finish and add something in the sentence. Sometimes you want it to be interactive so it asks you questions back to clarify and get some of the additional detail. And all of that is actually pretty hard. That's where kind of the magical... Ideal version of a voice agent for us comes through, where you need the speech to text element, you need the transcription side, unique... You need then the kind of the turn taking mechanism. So, like, when do you finish sentence? When when is it likely based on silence, based ways... Likely on the context? And then sometimes you want it to speak back and clarify or at least give you the text back to clarify and then maybe execute set of instructions. So that problem is still very hard research. So I agree with the claim that this orchestration side has not passed a true conversational agent Turing test where it, like, behaves as you would expect from another person, where you [00:19:37] S0: can say, I that's the simpler way of saying what I'm saying is that we have passed the Turing test with text LLMs a long time ago, and we're actually nowhere near that on voice LLMs. So it's kind of interesting how that's a final frontier. [00:19:47] S1: Yeah. I feel like it's it's it's going to work in, like, specific domains. Like, in customer support, call [00:19:53] S0: Yes.
00:19:53 · John Collison, Mati Staniszewski
[00:19:53] S0: Yes. [00:19:54] S1: Passes the voice tune test. Works well. And, like, let's take another, like, spectrum of that, an interactive gaming experience, like, a truly interactive as you would have with another human in that game, it's, like, you know, so hard and and and and further out there, we haven't passed it yet there. [00:20:10] S0: Yes. Yes. [00:20:10] S1: Sounds good. Yeah. But I think that's a combination of, like, know, even like a simpler version of within that. Sometimes you might give a response immediately back. Sometimes you need a tool call to get additional information from the database, how you orchestrate that. So, like, that's probably the most common thing we see as we work with some of the companies out there is you want those systems to orchestrate extremely well where if it's a conversational use case, pretty simple, you can route the agent to speak with, but if you need to authenticate, if you need to pull additional information from the database, what do you do? How do you handle that graciously? [00:20:41] S0: That's really tricky. [00:20:42] S1: And, yeah, and and to that extent, I would grade that. That's just getting getting getting there. Yes. And we'll hopefully see that we... Our goal is to, like, pass the voice chewing test in all those cases or the chewing test for all conversational agents outside of voice two, and I hope we will all be there in the in the next year or so.
00:21:02 · John Collison
[00:21:02] S0: For subscription businesses, a lot of revenue is lost in that last few seconds before the checkout. Someone has to get up and find their wallet or they mistype their card number or they hit an error. They just get up and you lose the sale. For a company like ElevenLabs, adding hundreds of thousands of subscribers, even a tiny bit of friction like that, it would really add up. But that's why ElevenLabs uses Link from Stripe. Customers save their details once, and then they can check out in seconds across more than a million businesses with saved credentials. So if you want a faster checkout for your customers, you should turn on Link from Stripe. Are you guys working on personalized voice transcription? Where it feels like part of the way we're making it hard for ourselves is when I speak to Siri, I have a bit of an accent, and so it sometimes has a hard time understanding me, but my accent doesn't change. And so it could just get good at listening to John. But my understanding is it's not. It's just like running the global voice recognition model. And I'm guessing it's the same for ElevenLabs where you're running the global voice recognition model. But, again, you have an accent. And so if someone's understand... Like, if you walked up to someone in a coffee shop and said two words, they might have a hard time understanding it because they'd like... They're not putting it through their Mati Polish accent filter. And so where is this going with, like, actually interpreting the person that you know to exist on the other side?