00:19:54 · S3
[00:19:54] S3: Hey, I'm curious how you're thinking about the need for voice in the future where agents do more and more of the work. So basically, what are the kind of use cases maybe where you mean conversation? I think it's more of a follow-up to the last question.
00:20:09 · Mati Staniszewski
[00:20:09] S1: Like first, you, all of us will have so many different devices around us and step from that, you will have robots around us. So, of course, voice will be such an important interface to to instruct and and be able to interact with those those those devices. In many ways, I feel like the... You know, we see a lot of developments of of intelligence, but then the real bottleneck of the future will be how we communicate with that intelligence. And I have voice and visual part will be a big unlock to be able to actually get the most of that intelligence value in those settings, which isn't yet possible. But on the flip side, the value of the human to human interaction will only increase. So whether that's the events like this one, whether that's events with your favorite artist will increase in value with that ability of having voice all around you. But the trust will be such a big part and something we optimize for in between the agent and human. In the future where all of you will all of us will have a voice agent, for example, to call and book a restaurant or give information to a health care appointment, all of that will require such a high degree of trust that this is you and authenticated you. So there'll be like a level of encoding and decoding for real, then encoding, decoding for watermark opted in human. And then by default, everything else will be fake, which is kind of the opposite of how it is today. You detect for AI, but it will detect for real authenticated AI in the future and assume it's fake. You could pass it.
00:21:50 · S4, Mati Staniszewski
[00:21:50] S4: Andre spoke earlier about jagged intelligence. Do you see similar odd places in audio where models are good and the bad that you might not expect? And yes, what are they? [00:22:02] S1: There's still so much on the bot side. I think we spoke a little bit about where we see the voice agents working, so this combination of the models together. And support settings works really well, works reliably. And early sales starts working, but the moment you start swapping to a true emotional interaction, not yet working. It doesn't get the emotion that well. It's slightly too slow. So that is still, I think, a big step change that should work. Same will apply in a very different domain on the music side. I think in the music side, you can get good production music. You cannot get Top Charts music even with artist input. I think this will change over the next year or two. Can I [00:22:50] S4: just follow-up? [00:22:51] S1: Of course. [00:22:51] S4: Andre's take was that the reason for that was that the labs were basically training for the stuff that had economic value. Where you're training your models, is that true of you? Are you basically training for the things that make the most money? Or is it that there are some challenges that are genuinely harder than others?
00:22:51 · S4, Mati Staniszewski, Andrew Reed, S5
[00:22:51] S4: Andre's take was that the reason for that was that the labs were basically training for the stuff that had economic value. Where you're training your models, is that true of you? Are you basically training for the things that make the most money? Or is it that there are some challenges that are genuinely harder than others? [00:23:09] S1: We try to train the models, build the product and the ecosystem that will derive, of course, the biggest impact for all our customers, all users, which should correlate, of course, with the revenue in the long term. So that long term perspective, it's going to be minimal in the next few years, so not next year. So frequently, we will train the models that might not provide that value in the short term. Or even step before, we'll spend so much time labeling the data, not only the what of audio, but also how of audio, like what emotions did I use, what is my voice described as, what is this music described as. So we assembled a team of now thousand plus people that have been voice coaches, musicians, artists before that can help us annotate that behind the scenes. And that will not provide value in the next six to twelve months, but we think will as well in the next twelve to twenty four. And then you often collect that data, which frequently just isn't that accessible as well. [00:24:03] S0: Okay. Last one, and then we'll go to lunch. [00:24:10] S5: Hey, can you hear me? Thanks. Yes. Big fan of yours and ElevenLabs. [00:24:14] S1: Thank you.
00:24:14 · Mati Staniszewski, S5
[00:24:14] S1: Thank you. [00:24:15] S5: What do you think from the model error perspective, what do you think are the moats here with audio models? The labs are going there, not going there. What are the kind of in this sausage making of making a real good frontier audio model? What are the main defensible parts there?
00:24:36 · Mati Staniszewski
[00:24:36] S1: So, of course, we do a variety of models. And I recently had a pleasure of meeting Jansen, and he was commenting on a few of those models. And he said that our speech to text or speech to text models are technology, and text to speech is artistry, and we are all artists. So he gained a client for life. But, of course, we do believe there is a little bit of that to to really fix text to speech and fix that emotionality. You will need to be really focused on that space. You really need to get in front of users, collect the data, collect the preferences, use that to fine tune their models. And then there's the domain specificity in how you actually bring those models to production. In health care, very different than in financial services, very different than in education or experiences. So that's on the model layer. I think there will be continuous advantage that if you actually care about the quality, spending the time on the model work will help you keep that advantage. But to your point, the models and a lot of use cases will use a model as just a small part of their stack. And that's where we spend a lot of time beyond going beyond the research on the product side of how you understand a user's problem, the workflow that they need. And voice agents is combining the audio models with knowledge and bringing that inside of system. How you bring it outside with telephony systems so you can interact across channels. How you evaluate, test, and monitor. And then as you create, whether that's in the agent space, whether that's in the creative space, that same understanding, you build the ecosystem. And that's what we hope to build across ElevenLabs, a place where, whether that's distribution and brand, that people can trust the platform where you have preexisting set of work that you can start off, whether it's a template for creating an agent, template for creating a workflow in the creative space, or whether that's a voice. And we had a pleasure now of having over 20,000 voices that people created, contributed, that you can use across language styles and voices. And I think that will be an increasingly important layer of how you are able to cater to that diversity, make it easy for people to start and really understand that workflow.