00:01:37 · Mati Staniszewski
[00:01:37] S3: As a company, one thing that we are aiming to solve is how humans and technology interact. So a lot of the work that we do underpins that common mission, and effectively, as you think about a lot of the problems that we solved across the history, it always started, what's the problem? Can we solve it? What's the next problem that we came across the scene? Like you said, we started with voice. So one of the key things that came when we were starting the company is that a lot of the technology around us just didn't sound human. You had a lot of the knowledge, information that just wasn't accessible, and we tried to solve this for creating our own text to speech model, and our own voice models, to make it sound human. And then the next thing came about, which is to be able to create a lot of that incredible work, you need a lot of more than that. You need sounds, you need music, you need understanding of speech through speech to text. So we, time and time again, started building that across, and built effectively our creative platform offering for creatives in the space. And then more recently, and I think that's the biggest focus across the company over the last two years, we realized that there is a big shift from static information into dynamic information where finally you can interact with the technology in ways you could never before. With the rise of agents, conversational agents, voice agents, suddenly you can speak with the devices, they can understand you and speak back. But the main thing to actually make them valuable is bringing all of the information, the knowledge base inside agent, being able to it with the existing systems, whether it's the Google, whether it's the Salesforce data, then figuring out the way to deploy the agent in a way which is testable, where you can evaluate over time, you can monitor, you have the right safeguards in place. So the main thing for us today is helping some of the biggest companies to starting companies deploy in their conversational agents and be able to interact with their users in a new ways.
00:03:21 · Taya
[00:03:21] S2: So I wanna get back to agents, but that's a lot of things. And there's also the new AI generated music platform that you guys launched over the summer. I see you guys as competing with OpenAI in some aspects. I see you guys competing with Adobe on some AI voice tools. How should we think about you?
00:03:39 · Mati Staniszewski
[00:03:39] S3: Across the general... Across AI space, what we are seeing is that there's increasing overlap when the foundational models now can create all the modalities. Increasingly, they also shift to that multi model approaches. Yes, you are right. There will be a lot of overlaps. Our key focus is helping people get the best of audio for creative workflow, and then helping deploy conversational agents. And of course, underpinning that, and maybe that's what makes us unique in the space is that we build our own models, where all of the companies that you mentioned, for them, those use cases that use the best of voice and deploy the best of voice are very rare. So whether that's building the models, frequently those companies won't build the models themselves and will choose to partner with others. And then the application side that you need to deploy the agent or the creative work is actually pretty complex. If you're creating, let's say, an audiobook, you need to go for a number of steps. And we spoke this before heading out of scene. You need to correct the pronunciation to make sure the words sound perfect. You need to select the right voices for each of the characters. So the whole exercise of creating and narrating the audiobook is pretty tricky. When we started the company, actually, on that case, when we first released our ElevenLabs interface, it was this tiny box where people could copy paste effectively a tweet length and download the work. And what quickly transpired is it's not enough. However, we still had users that would, we had one of these audio book authors that would copy paste his entire book, download it, stitch it up together. It was like 300 copy pastes, and then release it and it was great. That person returned with number of other people, and we realized, okay, you need a lot more of the interface to actually build The same exact thing happened with agents, where we work deeply with a customer in a healthcare space, a company called Hippocratic. They do this incredible use case where they help automate appointment scheduling for patients calling in to the healthcare units, but then also call them back to remind them about taking medicine and remind them about taking the appointment. And all of that is done for voice agents.
00:05:33 · Mati Staniszewski, Taya
But to be able to do that, they invested so much time to combine the text to speech, the LLabs of the brain, the speech to text, orchestrate that all together, and then bring that to the real actions. And then the same thing appeared across so many of other customers, and we knew that to be able to solve that use case at scale, we need to do a lot more. And I think that's the unique thing that we don't see many other companies do. The application side is very shallow. [00:05:57] S2: Something I've seen in the AI foundation model space is that companies keep trying to get bigger. They keep trying to tackle more. They need more funding, so they need to be more ambitious. I wanna know, you know, you've basically centered your entire company on audio right now, but have you considered focusing on other kinds of models, generating video or images kind of models? [00:06:20] S3: You are right, Taya. The size of the models is scaling in most of the other modalities. In audio, the slight difference is that it's still less about the scale. It's a lot more about the model architecture. I have a lack of having a brilliant co founder. Smartest person I I know who leads a lot of our research has been able to assemble some of the best researchers in the world for audio, and have been able to crack some of the model architecture challenges. And we still think this will be true in the audio space for the next year or two. Over long term, it will commoditize. Over the next couple of years, it's critical. [00:06:50] S2: You think it will commoditize?
00:06:50 · Taya, Mati Staniszewski
[00:06:50] S2: You think it will commoditize? [00:06:52] S3: In the long term, yes. Even if there's differences, which I think will be the true for some voices, some languages, on its own, the differences will be smaller over time. So today [00:07:01] S2: So why make the models if you think they'll commoditize?
00:07:03 · Mati Staniszewski
[00:07:03] S3: Because in the short term, they are the biggest advantage and the biggest step change you can have. Today where the voices still don't sound good, we need to solve that problem. Now when the interactions don't sound good, we need to solve that problem. The only way to solve it is for building the models yourself. And then over long term, there'll be other players that will solve that too. Hopefully, are first time and time again, but the call is to solve those. But to your question, the pattern we are seeing, if you want reliable, scalable use case, you will likely still use different models for different use cases. So best of breed across. So like in the agent space, you want the reliability and scalability across that, the kind of as the key component, you want explainability of the model. So then you are good with using speech to text, LLM text to speech, so you know each of the steps, you can link it up to the tooling in the right way. If it's a slightly different space, what we do expect in the next year or two is kind of what you mentioned, where increasing amount of models will move into a multi model or fused approaches. So you will create audio and video at the same time, or audio and LMs at the same time in a conversational setting. Google with Veo 3 incredible model that they released, is a good example of what you can achieve if you combine those together. And we are planning to do experiments in partnership with other companies and and independently to see whether we can combine our audio expertise with some of the expertise of other models to make that easier for our customers. Because ultimately, in the long long term, yes, it will be needed.