00:03:39 · Mati Staniszewski
[00:03:39] S3: Across the general... Across AI space, what we are seeing is that there's increasing overlap when the foundational models now can create all the modalities. Increasingly, they also shift to that multi model approaches. Yes, you are right. There will be a lot of overlaps. Our key focus is helping people get the best of audio for creative workflow, and then helping deploy conversational agents. And of course, underpinning that, and maybe that's what makes us unique in the space is that we build our own models, where all of the companies that you mentioned, for them, those use cases that use the best of voice and deploy the best of voice are very rare. So whether that's building the models, frequently those companies won't build the models themselves and will choose to partner with others. And then the application side that you need to deploy the agent or the creative work is actually pretty complex. If you're creating, let's say, an audiobook, you need to go for a number of steps. And we spoke this before heading out of scene. You need to correct the pronunciation to make sure the words sound perfect. You need to select the right voices for each of the characters. So the whole exercise of creating and narrating the audiobook is pretty tricky. When we started the company, actually, on that case, when we first released our ElevenLabs interface, it was this tiny box where people could copy paste effectively a tweet length and download the work. And what quickly transpired is it's not enough. However, we still had users that would, we had one of these audio book authors that would copy paste his entire book, download it, stitch it up together. It was like 300 copy pastes, and then release it and it was great. That person returned with number of other people, and we realized, okay, you need a lot more of the interface to actually build The same exact thing happened with agents, where we work deeply with a customer in a healthcare space, a company called Hippocratic. They do this incredible use case where they help automate appointment scheduling for patients calling in to the healthcare units, but then also call them back to remind them about taking medicine and remind them about taking the appointment. And all of that is done for voice agents.
00:05:33 · Mati Staniszewski, Taya
But to be able to do that, they invested so much time to combine the text to speech, the LLabs of the brain, the speech to text, orchestrate that all together, and then bring that to the real actions. And then the same thing appeared across so many of other customers, and we knew that to be able to solve that use case at scale, we need to do a lot more. And I think that's the unique thing that we don't see many other companies do. The application side is very shallow. [00:05:57] S2: Something I've seen in the AI foundation model space is that companies keep trying to get bigger. They keep trying to tackle more. They need more funding, so they need to be more ambitious. I wanna know, you know, you've basically centered your entire company on audio right now, but have you considered focusing on other kinds of models, generating video or images kind of models? [00:06:20] S3: You are right, Taya. The size of the models is scaling in most of the other modalities. In audio, the slight difference is that it's still less about the scale. It's a lot more about the model architecture. I have a lack of having a brilliant co founder. Smartest person I I know who leads a lot of our research has been able to assemble some of the best researchers in the world for audio, and have been able to crack some of the model architecture challenges. And we still think this will be true in the audio space for the next year or two. Over long term, it will commoditize. Over the next couple of years, it's critical. [00:06:50] S2: You think it will commoditize?
00:06:50 · Taya, Mati Staniszewski
[00:06:50] S2: You think it will commoditize? [00:06:52] S3: In the long term, yes. Even if there's differences, which I think will be the true for some voices, some languages, on its own, the differences will be smaller over time. So today [00:07:01] S2: So why make the models if you think they'll commoditize?
00:07:03 · Mati Staniszewski
[00:07:03] S3: Because in the short term, they are the biggest advantage and the biggest step change you can have. Today where the voices still don't sound good, we need to solve that problem. Now when the interactions don't sound good, we need to solve that problem. The only way to solve it is for building the models yourself. And then over long term, there'll be other players that will solve that too. Hopefully, are first time and time again, but the call is to solve those. But to your question, the pattern we are seeing, if you want reliable, scalable use case, you will likely still use different models for different use cases. So best of breed across. So like in the agent space, you want the reliability and scalability across that, the kind of as the key component, you want explainability of the model. So then you are good with using speech to text, LLM text to speech, so you know each of the steps, you can link it up to the tooling in the right way. If it's a slightly different space, what we do expect in the next year or two is kind of what you mentioned, where increasing amount of models will move into a multi model or fused approaches. So you will create audio and video at the same time, or audio and LMs at the same time in a conversational setting. Google with Veo 3 incredible model that they released, is a good example of what you can achieve if you combine those together. And we are planning to do experiments in partnership with other companies and and independently to see whether we can combine our audio expertise with some of the expertise of other models to make that easier for our customers. Because ultimately, in the long long term, yes, it will be needed.
00:08:28 · Taya, Mati Staniszewski
[00:08:28] S2: Other companies training AI video models you're thinking about partnering with? [00:08:32] S3: Either AI, other AI companies or Open Source. [00:08:35] S2: Yeah. What are some of those companies? [00:08:36] S3: They are still figuring this one out. We have close conversations with a few of our close close friends. [00:08:42] S2: That is a trend that we see a lot where these, you know, Sora and v o three, you generate both at the same time. And I imagine creatives are increasingly looking to do that. I mean, there's other challenge that you face of you're competing against these players and they're far more capitalized than you are. So are you thinking more about partnerships or potentially being acquired by someone in the future? Like, do you compete with them?
00:09:05 · Mati Staniszewski
[00:09:05] S3: We are building a generational company, and we we want to stay independent and build something that will create that value in the long term. So the whole thesis of how we think about combining the model and other work is that you cannot just rely on the model. You need to build the models, but then you'd also need that application side. That's why for us, it's so important to treat models as the head start, invest, be the best in the audio space, while at the same time, build a creative platform offering and build agents' platform offering. And that's the combination of building AI and building product. That's that's the magic. The same way like software and hardware was the magic for Apple. We think the product and AI will be the magic in the next generation of the best use cases. Now, what I think is tricky, and that's where where we expect in the next couple of years is essential, is on the audio side itself. So far, we've been able to create a text to speech model that was beating out of some benchmarks recently on speech to text, now on music. Then there's a second step of how you combine that model with, let's say you take an open source video model. That fusing step is actually challenging and tricky. So here too, we hope to be one of the first to figure that out, to do it in both high quality way and reliable way. We don't today plan to train some of the compute or data intensive models that competitors do. And of course, that's just a segment. That's why it's so important to do the product side.