00:05:18 · Joubin Mirzadegan, Mati Staniszewski
[00:05:18] S1: Right. [00:05:19] S0: Final downhill moment. So AI, nobody was working on AI, which gave us an advantage of two things. One was we had a good amount of time to start work before the GPT moment where... Like a true viral moment where that gave us an advantage of working before others. And two, the people we were able to hire in those days were true missionaries and excited people about the space. [00:05:46] S1: You mean you're not competing with Zoc back then?
00:05:49 · Mati Staniszewski
[00:05:49] S0: You are not competing with other big players. Although happy to compete with them, we... I think we we are now... We we we we... I think we have a great spot for a lot of the researchers to do the best work of their lives in in audio and beyond. But the the main thing is that all the people that joined you at the time are, like, a true missionaries. They really believe in a mission, really believe in what the voice can do for the world. That was still, like... Currently, it became normalized that voice is going to be the future interface. It's an important part. Back then, nobody really cared about AI. Nobody really cared about voice. And and me and my cofounder, so we started a company. We knew each other for fifteen years, so we are best friends from high school. Took all the same classes. We both love mathematics Mhmm. And everything around it. And then for the years did everything together. Worked together, traveled together, lived together, and are still best friends, so the time is on our on our side. Mhmm. And and the idea came from from Poland. We are both from Poland, and there's a very peculiar thing. If you watch a movie in Polish language, like a foreign movie in Polish language, all the characters get narrated with one single voice. So whether it's a male speaker or a female speaker in the original movie, in Polish, you get just one voice narrating every Is that unique to Poland? It's unique to the post communist countries. There's, like, five countries in the world that do it this way, and Poland is, like, one of the few, which is... It's almost hard to imagine, but it's a terrible experience. And it still happens for most of the work today, most of the content today. So that was the initial trigger point of like, hey. We know this will change. We know that with the invention of transformer a few years back, with the fusion models, audio will be able to to get created in that way that's actually heavily emotional and has a good intonation. You can recreate any voice you want. And and with knowing what's possible across audio space, decided to start ElevenLabs. And, course, the vision quickly expanded from there where it's not only that that will change. It will be the entire interaction with technology around us that that will change.
00:07:56 · Mati Staniszewski, Joubin Mirzadegan
If you can create synthetic speech that's high quality, the the voice will be such a powerful medium for for interaction and start ElevenLabs in 2022. We also... You know, we we we... Through the years prior... So here's how Google, I was a Palantir. We would meet and do hack weekend projects together across variety of different domains. I'm curious how you star... Stumbled on your on your idea as well. But in our case, we we were exploring new technologies and seeing what what they can do and how we can implement some of the... Mostly ideas around them, less so commercially focused ideas, and one of those was was was in audio. But that gave us a good kind of glimpse into how we can work together. We're so excited about going going full time and and dropping dropping our irrelevant companies and and starting ElevenLabs. [00:08:43] S1: Crazy. What an insane story. And then when the when the... When, I guess, what [00:08:49] S0: was it? GPT three came out? Was a year later? So the ChatGPT came December 2020 till January 2023. [00:08:58] S1: Not long into the journey. [00:08:59] S0: Not long into the journey. It's like Did you basically scrap everything? No. It was it was it was... So in a way, a lot of our ideas were based on the foundational stack anyways. [00:09:11] S1: Yep.
00:09:11 · Mati Staniszewski
[00:09:11] S0: So they were, like, just validated how relevant they are. They they... So, like, it was almost a parallel invention path, just a different technology. We, of course, have... A lot of the transform and diffusion ideas are are coming into our work into our work too. So when we started 2022, we would use some of the most relevant work recreating what should be do... What should be done in text to speech. And and then when chat GPT moment came and and a lot of the technology around us, this was, a, validating that we are doing the right thing because the ideas were giving good results. And, it kind of gave you this amazing overlap of what's possible where you could have ChatGPT now and you could have Voice on top of it. Suddenly, you can do content in such a much easier way Mhmm. Where ElevenLabs has even more advantage. I I guess, you know, like, the... At the time... So in 2022, most of the synthetic speech just was so poor. It was still very robotic. It's still... I don't want to name some of the big other companies, but but you can imagine the the hardware devices around how, like, the... That's like... The... It's still not good. And back in 2020, it was even worse. So it was truly robotic speech, and nobody could figure out how to make the speech sound human. So, like, actually represent emotions. And and that's where the 2022 for us was all focused. Like, how can we get the text, understand the context in a much better way, understand that the emotions around the text. Is it happy text? Is it dialogue? And bring that in the same way that the voice actor would or human would. When you read, you understand that context, and then you bring those emotions across. So that was the big innovation we brought in 2022 where text to speech sounded human. And the second big was being able to recreate voices. So how can you effectively encode a voice and the characteristics of the voice? And then when you produce synthetic speech, it still carries the style of the voice, the accent, the dialect, the... Whether it's narrated voice or conversational voice. How do you bring that all across? And we figured this out... This one out. But a lot of those ideas were based on similar ideas that the LLMs would carry as well, and then, of course, a lot of the the nuance that were required for the audio space.
00:11:32 · Joubin Mirzadegan, Mati Staniszewski
[00:11:32] S1: Mhmm. One of the things that I've been thinking about that I wanted your opinion on, and maybe this is just, like, a very stupid a stupid thought, but, like, we have this concept here in Silicon Valley about, like, crossing what's called the uncanny valley. And the uncanny valley is this concept where basically machines become indistinguishable from humans. Right. You cannot tell if it's an LLM talking to you or writing to you versus a human. Right. And so far, certainly in text, I can, within ten seconds, tell if an LLM has written it or not. It's very, very obvious. Based on [00:12:05] S0: the em dash? [00:12:06] S1: Or Based on the em dash. Based on just the way that it writes. It's not even just the em dash. Based on [00:12:11] S0: No. No. I agree.
00:12:11 · Mati Staniszewski, Joubin Mirzadegan
[00:12:11] S0: No. No. I agree. [00:12:12] S1: It's Like, not even random... Like, forget about the bolding and the em dashes. Right? Because nobody actually writes like that. Just the way that it actually writes does not sound human to me. And it's very obvious the minute that somebody sends me something, immediately, I'm I'm like, you didn't write it. Just like something inside my brain is like, that's not a person that wrote this. Okay? And so maybe audio, if it's coming off of text, is a similar paradigm today. And everybody continues to say, oh, no. No. It's happening. Like, it's going to happen for sure. And one of the questions that I've been asking myself, like, you know, these models have gotten a lot better, and they're still getting better. And everyone's like, well, as long as it keeps getting better, it's definitely gonna not sound... Like, it's gonna sound is... Indistinguishable from us. And, you know, I've listened to the, like, podcast LM from Google. Like, I've listened to all of this stuff. I've done the use ElevenLabs to clone my voice. And I'm not even saying... It's not a knock. Like, I'm not even saying it should or shouldn't sound like me. I'm like, it... Maybe it shouldn't. Like, maybe if you're from India, your dialect should be Indian. Like, maybe if you're from Iran, your dialect should be something else. Maybe if you're from machines, your dialect should be machine. You know? Like, I don't I don't know. And and and on that on