00:02:51 · Joubin Mirzadegan, Mati Staniszewski
[00:02:51] S1: Okay. And so your first... Let's say your first 10. Yeah. How many were each of those responsibilities? [00:02:57] S0: So my cofounder, the smartest person I know, also an incredible researcher. So that's what he used to do at at Google and previously at at university. So so he was one of the researchers, but on top of Scout, we hired two people in the first 10, one, two people, then additional one research engineer. But the research engineer, like, in the early days, who kind of do everything, whether it's data, whether it's improving the model. So it, like, kind of infuses. But I would say from the first stand, including me and my cofounder, four of the people would be doing research or research engineering. Two or three people would do engineering, and three people would do go to market growth. So pretty quickly, we started looking into how we can get the technology out there to to both the self serve distribution and then sales led distribution through working with some of the companies. [00:03:47] S1: Yeah. And before we get too far ahead, like, maybe for those that don't know, what do you all do?
00:03:47 · Joubin Mirzadegan, Mati Staniszewski
[00:03:47] S1: Yeah. And before we get too far ahead, like, maybe for those that don't know, what do you all do? [00:03:52] S0: So we are ElevenLabs. We do combination of research and and product work, product deployment work focused on AI voice space. We are on a mission to be the voice of technology, and we do it for two two key ways. One is in agents platform offering. We help people build conversational agents across customer support for personal AI, across media entertainment to make interactions interactive. And then second is our creative platform offering where we help people create narration, voice over, dubbing, add music to their work in one holistic suite. So really stretching the use cases where voice can empower and now even more on the omnichannel experience in agent and omni modality experience on the creative side. [00:04:34] S1: What's crazy is the company started pre GPT three. Right? And you were, if my math is correct, like, 25 years old at that point, living in Europe and recruiting research researchers. Like, that doesn't... I'm trying to, like... That doesn't really add up to me. Like, how does that how does that, like... What are they even... What were they working on? Like, what research were they even doing? Yeah. How did that how did that work? [00:05:06] S0: No. It is... You know, it's like, in a way, it was also the best... In in many ways, it was the best. We started our company in 2022. It was it was kind of still the year where crypto, metaverse were kind of on their [00:05:18] S1: Right.
00:05:18 · Joubin Mirzadegan, Mati Staniszewski
[00:05:18] S1: Right. [00:05:19] S0: Final downhill moment. So AI, nobody was working on AI, which gave us an advantage of two things. One was we had a good amount of time to start work before the GPT moment where... Like a true viral moment where that gave us an advantage of working before others. And two, the people we were able to hire in those days were true missionaries and excited people about the space. [00:05:46] S1: You mean you're not competing with Zoc back then?
00:05:49 · Mati Staniszewski
[00:05:49] S0: You are not competing with other big players. Although happy to compete with them, we... I think we we are now... We we we we... I think we have a great spot for a lot of the researchers to do the best work of their lives in in audio and beyond. But the the main thing is that all the people that joined you at the time are, like, a true missionaries. They really believe in a mission, really believe in what the voice can do for the world. That was still, like... Currently, it became normalized that voice is going to be the future interface. It's an important part. Back then, nobody really cared about AI. Nobody really cared about voice. And and me and my cofounder, so we started a company. We knew each other for fifteen years, so we are best friends from high school. Took all the same classes. We both love mathematics Mhmm. And everything around it. And then for the years did everything together. Worked together, traveled together, lived together, and are still best friends, so the time is on our on our side. Mhmm. And and the idea came from from Poland. We are both from Poland, and there's a very peculiar thing. If you watch a movie in Polish language, like a foreign movie in Polish language, all the characters get narrated with one single voice. So whether it's a male speaker or a female speaker in the original movie, in Polish, you get just one voice narrating every Is that unique to Poland? It's unique to the post communist countries. There's, like, five countries in the world that do it this way, and Poland is, like, one of the few, which is... It's almost hard to imagine, but it's a terrible experience. And it still happens for most of the work today, most of the content today. So that was the initial trigger point of like, hey. We know this will change. We know that with the invention of transformer a few years back, with the fusion models, audio will be able to to get created in that way that's actually heavily emotional and has a good intonation. You can recreate any voice you want. And and with knowing what's possible across audio space, decided to start ElevenLabs. And, course, the vision quickly expanded from there where it's not only that that will change. It will be the entire interaction with technology around us that that will change.
00:07:56 · Mati Staniszewski, Joubin Mirzadegan
If you can create synthetic speech that's high quality, the the voice will be such a powerful medium for for interaction and start ElevenLabs in 2022. We also... You know, we we we... Through the years prior... So here's how Google, I was a Palantir. We would meet and do hack weekend projects together across variety of different domains. I'm curious how you star... Stumbled on your on your idea as well. But in our case, we we were exploring new technologies and seeing what what they can do and how we can implement some of the... Mostly ideas around them, less so commercially focused ideas, and one of those was was was in audio. But that gave us a good kind of glimpse into how we can work together. We're so excited about going going full time and and dropping dropping our irrelevant companies and and starting ElevenLabs. [00:08:43] S1: Crazy. What an insane story. And then when the when the... When, I guess, what [00:08:49] S0: was it? GPT three came out? Was a year later? So the ChatGPT came December 2020 till January 2023. [00:08:58] S1: Not long into the journey. [00:08:59] S0: Not long into the journey. It's like Did you basically scrap everything? No. It was it was it was... So in a way, a lot of our ideas were based on the foundational stack anyways. [00:09:11] S1: Yep.
00:09:11 · Mati Staniszewski
[00:09:11] S0: So they were, like, just validated how relevant they are. They they... So, like, it was almost a parallel invention path, just a different technology. We, of course, have... A lot of the transform and diffusion ideas are are coming into our work into our work too. So when we started 2022, we would use some of the most relevant work recreating what should be do... What should be done in text to speech. And and then when chat GPT moment came and and a lot of the technology around us, this was, a, validating that we are doing the right thing because the ideas were giving good results. And, it kind of gave you this amazing overlap of what's possible where you could have ChatGPT now and you could have Voice on top of it. Suddenly, you can do content in such a much easier way Mhmm. Where ElevenLabs has even more advantage. I I guess, you know, like, the... At the time... So in 2022, most of the synthetic speech just was so poor. It was still very robotic. It's still... I don't want to name some of the big other companies, but but you can imagine the the hardware devices around how, like, the... That's like... The... It's still not good. And back in 2020, it was even worse. So it was truly robotic speech, and nobody could figure out how to make the speech sound human. So, like, actually represent emotions. And and that's where the 2022 for us was all focused. Like, how can we get the text, understand the context in a much better way, understand that the emotions around the text. Is it happy text? Is it dialogue? And bring that in the same way that the voice actor would or human would. When you read, you understand that context, and then you bring those emotions across. So that was the big innovation we brought in 2022 where text to speech sounded human. And the second big was being able to recreate voices. So how can you effectively encode a voice and the characteristics of the voice? And then when you produce synthetic speech, it still carries the style of the voice, the accent, the dialect, the... Whether it's narrated voice or conversational voice. How do you bring that all across? And we figured this out... This one out. But a lot of those ideas were based on similar ideas that the LLMs would carry as well, and then, of course, a lot of the the nuance that were required for the audio space.