AI Tinkerers - "One-Shot"
00:40:53 · Luke Harries, Joe Heitzeberg
[00:40:53] S0: Well, there's... Yeah. Well, actually, I was thinking... So I think you should be able to solve your problem just through the system prompts because it's a... You can select any model. You could do Anthropic, you can do Gemini, you can do OpenAI. [00:41:05] S1: You know what I did with the system prompts? So rather than actually solve the problem I described is I was like, okay. Now I want you to spell every word in in a Boston accent. I use I use at ElevenLabs, like, gruff voice, and I tried to make it sound. [00:41:18] S0: Voices remain. [00:41:19] S1: Just as fun. [00:41:19] S0: Finding a voice. Well, Joe, you should cloak your I don't know. Genuinely. And use that for... We can actually demo. I've got my voice in this panel. [00:41:26] S1: Well, you know, I have a lot of emails for the organizers that are that are me. Yes. Aren't me. [00:41:31] S0: Yeah. I I I am sure we're getting to it. But the... To show you the, it's a pretty cool demo. I'll just show you if you want, which is, and then we can jump into how you build these after. Yeah. Is there's actually two modes of tools. We have, like, server side tools [00:41:46] S1: Yeah. [00:41:47] S0: And... But we also have client side tools. [00:41:49] S1: Mhmm.
00:41:49 · Joe Heitzeberg, Luke Harries
[00:41:49] S1: Mhmm. [00:41:49] S0: And so this can run, execute JavaScript on your behalf Mhmm. Or like cool. Well, it just returns which tool choose and the args. Yeah. But like, you can have it do stuff like switch the page. Mhmm. So you can be like, oh, I'd love to go to a meetup in Warsaw, and they will like take you immediately to the page. And then it goes great, like give me a quick summary of yourself. Yeah. And then they could like pre fill the entire thing for you. Yeah. Which could be pretty cool. But just to show people that one. Yeah. Hey there. I'm Alexis from ElevenLabs support. How can I help you today? Yeah. Hey, Alexis. Can you tell me more about conversational AI? Sure thing. ElevenLabs Conversational AI is a platform that allows you to deploy customized conversational voice agents. It combines speech to text language models Alexa. Text to speech, and Can you tell me a bit more? Can you show me the the client side tools? Okay. I can tell you more about client side tools. Client tools enable your assistant execute. Perfect. I'm now now actually take me to the page. Or sending... Oh, take me to the page. Do you wanna be taken to the documentation page about client side tools? [00:43:03] S1: Yeah. Is this part baked in with the JavaScript widget, the the navigation aspect? Or do I have to build that?
00:43:03 · Joe Heitzeberg, Luke Harries
[00:43:03] S1: Yeah. Is this part baked in with the JavaScript widget, the the navigation aspect? Or do I have to build that? [00:43:09] S0: I think I may be... The tricky... Where it's still... Let me just get it. Okay. Yes. Take me to the page. Let's see. There we go. And now it's jumped to the jumped to the actual page. Oh, isn't that cool? Yeah. I think... I'm not sure. We definitely have in our code, in our docs. We actually have that how to implement [00:43:32] S1: that. Okay. [00:43:33] S0: So you can just grab it and paste [00:43:34] S1: it in. [00:43:34] S0: Okay. I think we do probably a little bit of asking you to do that thing if you want to to set something. [00:43:41] S1: This is the exciting thing for me. Yeah. I've I've thought a lot about the platform we built for Tinkerers has so many functions from invoicing and scheduling and emails and screening and data and that... Why build more UI? It's just getting more complex. If I can expose the primitives through tools to something like that, you know, our users are familiar with what's available. And if they have a good experience, I think they'll quickly ask to do the right things. Right? Does that make sense? [00:44:08] S0: 100%. [00:44:08] S1: 100%. Very interesting. And it's, know, voice is the best mobile app experience if you really have a great one that's smart.
00:44:08 · Joe Heitzeberg, Luke Harries
[00:44:08] S1: 100%. Very interesting. And it's, know, voice is the best mobile app experience if you really have a great one that's smart. [00:44:14] S0: 100%. I'll just show very briefly to create one. What you can do is you can name your agent. So say we've got... So we do a... Should we do a AI Tinkerers? We can try and do a quick... Well, we do we do a support agent, maybe bring in a bit of AI Tinkerers stuff in it. Sure. And then you click create. And straight away there, you actually have a support agent, which is ready to go. Mhmm. You can select the voice. So should we try doing it with my voice? Luke Harries. So I did my own voice clone recently, which is pretty fun. And then, hi, I'm Luke from ElevenLabs. Luke Harries from ElevenLabs support. And you'll just... I'm just entering this information into the system prompt in the first message. You can also do, as we discussed, like, client side tools. You can do a end call. So at the end of call, it sends an... [00:45:14] S1: Oh, [00:45:14] S0: cool. Oh, yeah. Sorry. Gives the ability to end the call, but you can also do, like, a web hook when a call ends. [00:45:19] S1: You can [00:45:19] S0: send transcript to your own own knowledge base. Nice. And and then any other bit? I think that's probably... Oh, yeah. Here's the widget if you want to add it to your own site. [00:45:29] S1: And we didn't talk about that. You can import a Twilio number. So get a number on Twilio. There's some off workflow that makes it easy to put in. That's really cool.
00:45:29 · Joe Heitzeberg, Luke Harries
[00:45:29] S1: And we didn't talk about that. You can import a Twilio number. So get a number on Twilio. There's some off workflow that makes it easy to put in. That's really cool. [00:45:36] S0: And you can do both inbound calls. You can do outbound calls. We have Oh, that's cool. Batch batch outbound calls. So say you wanted to ring up [00:45:44] S1: Yeah. [00:45:45] S0: All the people going to the AI Tinkerers thing tonight to get them get them hyped for it. [00:45:49] S1: Sure. I think, well, for, like, the VIP dinner, experience, I think it's a better way. And you could do a quick interview like, hey. You know, what are some topics that you'd like to talk about and just stream that through back to the sponsor with a summary. I'm gonna build that. That's what I'm doing on the [00:46:02] S0: Yeah. [00:46:03] S1: Building that on the airplane when I go back. [00:46:04] S0: The the other cool thing is what I really love about voice is it's so much quicker for data collection [00:46:09] S1: Yeah. [00:46:10] S0: And so much easier. Do you imagine if we had done this podcast and it was just in a Google doc? Yeah. Taking us hours so much more boring. Yeah. Instead, you could phone people ahead of time. What the topic's top of mind? [00:46:20] S1: Yeah. What are the [00:46:20] S0: things you... How could you be helped out during this dinner? And then, like, synthesize that, match make everyone. But we
00:46:20 · Luke Harries, Joe Heitzeberg
[00:46:20] S0: things you... How could you be helped out during this dinner? And then, like, synthesize that, match make everyone. But we [00:46:26] S1: have two guys. I'm I'm pretty bullish on the emotional stuff. I can't wait for that from ElevenLabs. And, you know, there have been some other experimental things. And even back in the day, I don't know if you remember, what was it? It was tortoise. Do you remember tortoise? Yes. Yeah. I mean, tortoise was incredible Yes. Before ElevenLabs. [00:46:44] S0: Yeah. That was a step change too. [00:46:45] S1: Yes. And it was... You were able to do emotional stuff if you gave it inputs. For me, that was just exciting. But when you're talking about, like, engaging a human to, like, provide a bunch of useful information, that engagement is super important for conversion. Right? [00:46:58] S0: Yeah. [00:46:58] S1: So I'm I'm really excited about that. You can definitely achieve it with ElevenLabs by cloning, but if we're profitable on that stuff, I think it's a game changer.