AI Tinkerers - "One-Shot"
00:45:29 · Joe Heitzeberg, Luke Harries
[00:45:29] S1: And we didn't talk about that. You can import a Twilio number. So get a number on Twilio. There's some off workflow that makes it easy to put in. That's really cool. [00:45:36] S0: And you can do both inbound calls. You can do outbound calls. We have Oh, that's cool. Batch batch outbound calls. So say you wanted to ring up [00:45:44] S1: Yeah. [00:45:45] S0: All the people going to the AI Tinkerers thing tonight to get them get them hyped for it. [00:45:49] S1: Sure. I think, well, for, like, the VIP dinner, experience, I think it's a better way. And you could do a quick interview like, hey. You know, what are some topics that you'd like to talk about and just stream that through back to the sponsor with a summary. I'm gonna build that. That's what I'm doing on the [00:46:02] S0: Yeah. [00:46:03] S1: Building that on the airplane when I go back. [00:46:04] S0: The the other cool thing is what I really love about voice is it's so much quicker for data collection [00:46:09] S1: Yeah. [00:46:10] S0: And so much easier. Do you imagine if we had done this podcast and it was just in a Google doc? Yeah. Taking us hours so much more boring. Yeah. Instead, you could phone people ahead of time. What the topic's top of mind? [00:46:20] S1: Yeah. What are the [00:46:20] S0: things you... How could you be helped out during this dinner? And then, like, synthesize that, match make everyone. But we
00:46:20 · Luke Harries, Joe Heitzeberg
[00:46:20] S0: things you... How could you be helped out during this dinner? And then, like, synthesize that, match make everyone. But we [00:46:26] S1: have two guys. I'm I'm pretty bullish on the emotional stuff. I can't wait for that from ElevenLabs. And, you know, there have been some other experimental things. And even back in the day, I don't know if you remember, what was it? It was tortoise. Do you remember tortoise? Yes. Yeah. I mean, tortoise was incredible Yes. Before ElevenLabs. [00:46:44] S0: Yeah. That was a step change too. [00:46:45] S1: Yes. And it was... You were able to do emotional stuff if you gave it inputs. For me, that was just exciting. But when you're talking about, like, engaging a human to, like, provide a bunch of useful information, that engagement is super important for conversion. Right? [00:46:58] S0: Yeah. [00:46:58] S1: So I'm I'm really excited about that. You can definitely achieve it with ElevenLabs by cloning, but if we're profitable on that stuff, I think it's a game changer.
00:46:58 · Joe Heitzeberg, Luke Harries
[00:46:58] S1: So I'm I'm really excited about that. You can definitely achieve it with ElevenLabs by cloning, but if we're profitable on that stuff, I think it's a game changer. [00:47:05] S0: Yeah. Yeah. Hope... Some cool stuff in the works. And just to show you my own voice clone and your audience may see... Enjoy seeing the quality of the voice clones as well. Yeah. Hey there. I'm Luke Harries from ElevenLabs support. How can I help you today? Hey, Luke. I'd like you to help me... Can you tell me a bit more about conversational AI or... Actually, what is text to speech? Tell me what is text to speech? Alright. I can definitely help with that. Text to speech or TTS is a technology that converts written text into spoken words. It's basically what allows Perfect. Thank you, Luke. Text aloud. Does that And so that that works. And then what we can do is jump back. And if we click on this top right, we should be able to go to conversation history for this agent. And then you can see the full transcript. It gives a summary of it. You can also write your own evals as well. Mhmm. So like, let's say someone like well, the evals, they're actually just questions and prompts. Yeah. So you can say like, I'd like to extract this information and grab it all. Okay. And and then it can... You can define like success and... [00:48:16] S1: So you have evals built into the system when you're going. That's amazing. Yeah. That's incredibly useful. And you're running the evals and you're evaluating the eval output. [00:48:22] S0: Yes. Yeah. Yeah. And the way I would... I mean
00:48:22 · Luke Harries, Joe Heitzeberg
[00:48:22] S0: Yes. Yeah. Yeah. And the way I would... I mean [00:48:25] S1: That's cool. [00:48:25] S0: The interesting thing about... People are often asking me about voice evals. My contrarian take is like, largely, you know, it is getting better, the term taking and the interruption and those bits. But, like, I think the vast majority of the evals you need to do is actually around the LLM and the brain. And I think most of the just normal, like, text platforms is where where the kind of eval should be doing. And also then [00:48:52] S1: Makes sense. [00:48:53] S0: So much cheaper, so much quicker. You're not doing Yeah. All the like voice in voice [00:48:56] S1: That out makes total sense. In fact, you'd probably wanna partition those at scale. [00:48:59] S0: Yes. Yeah. [00:49:00] S1: So we talked about latency. You're already really pretty dang fast. In fact, I met a... Someone... Oh gosh. A hypo... Hypocrite... Hippocratic. Yeah. Yes. Yeah. So founder there did a meetup in Palo Alto for Tinkerers and [00:49:15] S0: Yes. [00:49:16] S1: Showed his... This is at the time... This is a year and a half ago. Okay? And it was definitely the fastest voice... And I was pretty deep on all the things open source and different... Including ElevenLabs. And when he showed that, I was like, this is the best I've seen in terms of naturalness, speed. I was like... And so we asked him what he was using, and when he said ElevenLabs, I was surprised. [00:49:34] S0: Yes.
00:49:34 · Luke Harries, Joe Heitzeberg
[00:49:34] S0: Yes. [00:49:35] S1: I was like, what? Because I was that familiar with the latencies that I was like, you must have a special dose. Like, oh, yes. We have a... You know, we got... This is a long time ago. But, clearly, all that that speed has come to the platform because it's it's just as fast now. So we talked about latency, how important this... We haven't talked about... We talked about expressiveness. Sounds like you got stuff coming. I won't press you on it to get you in trouble. Turn detection. What are you doing there? That seems like a really important too. [00:50:00] S0: Yeah. We've just shipped a new, so we're releasing very soon our, like, conversational AI two point o [00:50:09] S1: K. [00:50:09] S0: Which is there's been a whole bunch of new features we've shipped, namely the built in RAG and the better turn taking model. And what we've moved to is our own custom model, which feeds in two layers of input. So one is voice and audio [00:50:24] S1: Mhmm.
00:50:24 · Joe Heitzeberg, Luke Harries
[00:50:24] S1: Mhmm. [00:50:24] S0: And and the other is, like, semantic word level. Mhmm. So what's quite interesting... I do this thing where most weekends, I walk around a park and I speak to our conversational AI system, and I get to tell me, like, reflect on my week. Oh, cool. And with the first couple of versions, it's like, what was the, you know, what was the highlight of your week? And you're like, yeah. Great question. And then it goes, Luke, are you still there? And you're like, no. I'm like Right. I'm trying to think Right. Like, clearly, I'm still thinking. But you know that from, like, the semantic word layer of it. [00:50:57] S1: Good. That's a huge improvement. Yeah. I know that feeling. It's attention. Yes. It's worked for me to kinda keep the AI to be with me. [00:51:04] S0: I got really good at saying, and then it wouldn't interrupt me. [00:51:07] S1: It feels like walking a dog that's not quite trained. You have to be too much in the moment. [00:51:12] S0: Yeah. But, no, that