AI Tinkerers - "One-Shot"
00:46:58 · Joe Heitzeberg, Luke Harries
[00:46:58] S1: So I'm I'm really excited about that. You can definitely achieve it with ElevenLabs by cloning, but if we're profitable on that stuff, I think it's a game changer. [00:47:05] S0: Yeah. Yeah. Hope... Some cool stuff in the works. And just to show you my own voice clone and your audience may see... Enjoy seeing the quality of the voice clones as well. Yeah. Hey there. I'm Luke Harries from ElevenLabs support. How can I help you today? Hey, Luke. I'd like you to help me... Can you tell me a bit more about conversational AI or... Actually, what is text to speech? Tell me what is text to speech? Alright. I can definitely help with that. Text to speech or TTS is a technology that converts written text into spoken words. It's basically what allows Perfect. Thank you, Luke. Text aloud. Does that And so that that works. And then what we can do is jump back. And if we click on this top right, we should be able to go to conversation history for this agent. And then you can see the full transcript. It gives a summary of it. You can also write your own evals as well. Mhmm. So like, let's say someone like well, the evals, they're actually just questions and prompts. Yeah. So you can say like, I'd like to extract this information and grab it all. Okay. And and then it can... You can define like success and... [00:48:16] S1: So you have evals built into the system when you're going. That's amazing. Yeah. That's incredibly useful. And you're running the evals and you're evaluating the eval output. [00:48:22] S0: Yes. Yeah. Yeah. And the way I would... I mean
00:48:22 · Luke Harries, Joe Heitzeberg
[00:48:22] S0: Yes. Yeah. Yeah. And the way I would... I mean [00:48:25] S1: That's cool. [00:48:25] S0: The interesting thing about... People are often asking me about voice evals. My contrarian take is like, largely, you know, it is getting better, the term taking and the interruption and those bits. But, like, I think the vast majority of the evals you need to do is actually around the LLM and the brain. And I think most of the just normal, like, text platforms is where where the kind of eval should be doing. And also then [00:48:52] S1: Makes sense. [00:48:53] S0: So much cheaper, so much quicker. You're not doing Yeah. All the like voice in voice [00:48:56] S1: That out makes total sense. In fact, you'd probably wanna partition those at scale. [00:48:59] S0: Yes. Yeah. [00:49:00] S1: So we talked about latency. You're already really pretty dang fast. In fact, I met a... Someone... Oh gosh. A hypo... Hypocrite... Hippocratic. Yeah. Yes. Yeah. So founder there did a meetup in Palo Alto for Tinkerers and [00:49:15] S0: Yes. [00:49:16] S1: Showed his... This is at the time... This is a year and a half ago. Okay? And it was definitely the fastest voice... And I was pretty deep on all the things open source and different... Including ElevenLabs. And when he showed that, I was like, this is the best I've seen in terms of naturalness, speed. I was like... And so we asked him what he was using, and when he said ElevenLabs, I was surprised. [00:49:34] S0: Yes.
00:49:34 · Luke Harries, Joe Heitzeberg
[00:49:34] S0: Yes. [00:49:35] S1: I was like, what? Because I was that familiar with the latencies that I was like, you must have a special dose. Like, oh, yes. We have a... You know, we got... This is a long time ago. But, clearly, all that that speed has come to the platform because it's it's just as fast now. So we talked about latency, how important this... We haven't talked about... We talked about expressiveness. Sounds like you got stuff coming. I won't press you on it to get you in trouble. Turn detection. What are you doing there? That seems like a really important too. [00:50:00] S0: Yeah. We've just shipped a new, so we're releasing very soon our, like, conversational AI two point o [00:50:09] S1: K. [00:50:09] S0: Which is there's been a whole bunch of new features we've shipped, namely the built in RAG and the better turn taking model. And what we've moved to is our own custom model, which feeds in two layers of input. So one is voice and audio [00:50:24] S1: Mhmm.
00:50:24 · Joe Heitzeberg, Luke Harries
[00:50:24] S1: Mhmm. [00:50:24] S0: And and the other is, like, semantic word level. Mhmm. So what's quite interesting... I do this thing where most weekends, I walk around a park and I speak to our conversational AI system, and I get to tell me, like, reflect on my week. Oh, cool. And with the first couple of versions, it's like, what was the, you know, what was the highlight of your week? And you're like, yeah. Great question. And then it goes, Luke, are you still there? And you're like, no. I'm like Right. I'm trying to think Right. Like, clearly, I'm still thinking. But you know that from, like, the semantic word layer of it. [00:50:57] S1: Good. That's a huge improvement. Yeah. I know that feeling. It's attention. Yes. It's worked for me to kinda keep the AI to be with me. [00:51:04] S0: I got really good at saying, and then it wouldn't interrupt me. [00:51:07] S1: It feels like walking a dog that's not quite trained. You have to be too much in the moment. [00:51:12] S0: Yeah. But, no, that
00:51:12 · Luke Harries, Joe Heitzeberg
[00:51:12] S0: Yeah. But, no, that [00:51:13] S1: that can be solved. That's wonderful to hear. Okay. Talking about the... Go back to popping up to ElevenLabs, the tremendous growth. I can imagine early on that a lot of it was just tinkering and playing, and it's like almost like a new form of entertainment to use voices and AIs and interact with them. Yeah. But now we've got clearly people using it for customer support. What are the growth drivers right now? What are the actual real world use cases where you're seeing... You know, is it the stuff that I would expect to hear support? Are there any unexpected unexpected modes modes that that are are interesting? Interesting? [00:51:42] S0: The one which I think is really surprise... Yeah. So the obvious ones are like support and sales. Yeah. The one which I think is really surprising is education. Oh, yeah. Because I think for years in, like, tech VC space, just no one view no one view education companies as, like, actually venture backed investable businesses. Mhmm. But what's fascinating is as soon as you start integrating proper AI and voice Mhmm. The quality level improves so much. You can deliver on so much more. You can, like, properly have a, like, hour long experience or or get way better at language learning. [00:52:18] S1: Does that does that look like people just using it directly as consumers or or baking it into education platforms as a mode?
00:52:18 · Joe Heitzeberg, Luke Harries
[00:52:18] S1: Does that does that look like people just using it directly as consumers or or baking it into education platforms as a mode? [00:52:25] S0: So for us, we... With our conversational AI is nearly always we sell to businesses who create these incredible end experiences. Yeah. So an example would be chess.com. Okay. So they have a... An app with, professor Wolf, which will give you, like, interactive chess lessons and speak you through it. Fun. So that's a really cool one. There's another one, like, a whole bunch of the language learning apps are using us. Yeah. And so like Practica is one example, but some of the biggest use us and like creating properly interactive Yeah. Things. And my wife is French. I've spent thousands of pounds Yeah. Thousands of dollars having like one on one French lessons. Yeah. And every time it was like, oh, you need to be there at a certain time. Yeah. Their Internet's bad and blah blah blah. And now it's just like on demand. You have a fantastic French tutor.