00:32:44 · Mati Staniszewski
[00:32:44] S1: One, breaking down the language barrier, we... You know, the... Kind of the inspiration came from from the movie side, but it also applies in any in any communication setup. Like, could in the future, could I travel to another country and speak Polish or speak English, and that language isn't being understood in the local native language? Like from Hitchhiker's Guide to Galaxy, this version of the babble Exactly. Then you can actually understand the world. Voice, of course, will be an interaction layer, but similarly, all of us will have our own kind of extension and voice agents that can help on our behalf, and there's very clear and great examples of that of people that lost their voice and can get it for the first time back. We see that everywhere, whether that's people that lost it due to ALS or throat cancer that can get it back. Just recently, there was an example of a patient that had Neuralink. We worked with them to bring the voice that that person could speak with their own voice back with the with the family around. We worked with with with with the lady that lost her voice before before she got married, and and then finally, technology became possible. We we were able to recreate that voice, and for the first time, she could replicate the the marriage ceremony and speak the vows together, which was, like, such a heartfelt moment. Probably, like, the most important from all the work that that we do.
00:34:03 · John Collison, Mati Staniszewski
[00:34:03] S0: When you guys you guys talk about voice agents, is a voice agent just the idea that you have some long running or persistent agent that is going out and interacting with the world through voice? And so customer service would be one example of this. You know, in the other direction, you're clogging and making you a rest... A restaurant reservation and actually calling up the restaurant. Is that is that kinda how I should think about voice agents? [00:34:31] S1: That's right. It's it's it's... Exactly. Whether it's like the reactive side of being able to interact with the customer or the proactive to call it back. We recently had a very interesting one, topical because it was a Guinness related one where there was a there was a developer developing a Gindex effectively. [00:34:46] S0: Oh, I saw that. They were calling all the pubs in Ireland, checking the price of a pint. [00:34:50] S1: Yeah. Could ask that and or report information [00:34:53] S0: That was called... The Gindex was built with ElevenLabs too. [00:34:55] S1: It was built with ElevenLabs too, so people could could do both sites, could proactively reach out, reactively reach out Yes. Always capture through voice, and Yeah. Yeah. Yeah. And then kind of 3,003 different entities could report their prices and and and get that across. [00:35:12] S0: Have you, by the way, hooked up your OpenClaw to ElevenLabs? Is the OpenClaw, ElevenLabs combo something that a lot of people at Eleven are doing?
00:35:12 · John Collison, Mati Staniszewski
[00:35:12] S0: Have you, by the way, hooked up your OpenClaw to ElevenLabs? Is the OpenClaw, ElevenLabs combo something that a lot of people at Eleven are doing? [00:35:21] S1: So, yeah, as you know, the OpenClaw will, like, kind of look for the most popular tools frequently where it tries to hook up. So Yeah. Yeah. Yeah. Is one of the recommended ones. The top option for voice. [00:35:30] S0: Can you tell me a bit about the business of voice models where... I think people have an intuition around big LLMs where there are these very expensive training runs. And yes, they kind of depreciate quickly, but the... There's so much usage that all of the models trained to date have, you know, paid off their training runs and then some. And then there's this kind of ever larger CapEx going into... I mean, a lot of it is inference these days, but also training. And so I have some intuitions from the LLM world. I'm curious just how I should think about voice for, one, how expensive is training the voice models? Is the expense in the researchers? Is the expense in the the training runs? And, I mean, the economics is presumably kinda simple, but it's just per per usage, but, yeah, just talk us through the business. [00:36:21] S1: Yeah. Definitely cheaper than the LM and and image video models, It's gonna be smaller models. [00:36:27] S0: Yep. Okay. So the models are smaller? [00:36:29] S1: Smaller. What's [00:36:30] S0: the parameter count for a leading edge voice model? [00:36:32] S1: Few billion to low tens of billion parameter models. Yeah.
00:36:32 · Mati Staniszewski, John Collison
[00:36:32] S1: Few billion to low tens of billion parameter models. Yeah. [00:36:37] S0: So so... And for context, I think the... I mean, kinda like, you know, CPUs moved away eventually from gigahertz as, like, the metric as they move to more cores. I think we've mostly moved away from just raw parameter count, but I think the leading edge LMs are in the hundreds of billions of parameters. [00:36:52] S1: I think the leading ones, yes. But, of course, you know, you have the variations that you will use at at lower scale. Yep. So CapEx is still pretty high. Yeah. We've, of course, raised recently a 500,000,000 at at 11,000,000,000 valuation to, like Makes sense. Makes sense. To continue to continue being able to build the best models in the in in the world. Researchers, you know, of course, you want to... You want the best people in the world. I think we have we have those people working in in in audio in in in micro founder who who is who is leading that work. So that's that's that's definitely a big piece of, like, not financially, but even, like, how you keep the ambitious [00:37:28] S0: Mhmm.
00:37:28 · John Collison, Mati Staniszewski
[00:37:28] S0: Mhmm. [00:37:29] S1: About deployment, deployment, so you kind of continue building leading models helps you attract more talent and and building that. And then on the... How we service, of course, inference is correlated with how the models are used, and and for us, like, we've seen incredible incredible growth across the work. Mostly, this is charged per... If it's input text or text to speech, it's usually per per per text token. If it's voice agent or or transcription, then then it's per minute, and we see that kind of being the the bigger part of it. Usually, like, broadly, it's per token basis, and, course, as we work with businesses, it's like an annual agreement. The bigger the spend, the bigger the comments, the bigger the discount to Yeah. Gonna get that across. The way we usually do is, like, when we have a new model, we try to give it at cost to a lot of the customers so they can experience the best. It's still usually, like, not as But just [00:38:19] S0: the newest thing is often the most expensive, whereas you make the newest thing the most economically attractive one? [00:38:25] S1: We try to make it attractive so the customers are... Like, you know, like, it's it's more expensive for us than any previous generation. We don't like... [00:38:32] S0: Yes. [00:38:32] S1: Oh, the quality is higher, so we try to keep the prices still still competitive I used to to [00:38:37] S0: subsidize it, but it's inherently more expensive
00:38:37 · John Collison, Mati Staniszewski
[00:38:37] S0: subsidize it, but it's inherently more expensive [00:38:38] S1: to figure model. Exactly, exactly. And over time, we might do some tricks to optimize it, but we want the customers to experience... Because of research, the big thing that we've seen is the reliability of the model in the early days might not be there, and then two, people don't even know what's possible with that model. So you kind of want the widest set of distributions so people can show the world what's possible, so you can have it, of course, as a distribution mechanism, learn yourself what to improve, what to change Yes. And then and then get it out there. [00:39:10] S0: Are the voice models just getting bigger and bigger? Like, will we have voice models in the hundreds of billions of parameters, or have we found... [00:39:18] S1: Like, it [00:39:19] S0: seems like for certain types of model architectures, there's, like, an upper limit on, like, the natural size. Have we found that upper limit for voice models?