AI Tinkerers - "One-Shot"
00:39:01 · Joe Heitzeberg, Luke Harries
[00:39:01] S1: Mhmm. [00:39:01] S0: To handle the, like, I mean, it abstracts it all to text in text out. Yep. So let's say you're a big enterprise, you spend... You have a thousand per cent engineering team building out all your text based solutions, you can very quickly wrap this and suddenly you have a voice system. And then the other bit is we have the much more feature complete version where you can do proper rag and we have evals. But, yeah, I can show you... Talk you through some of that. [00:39:24] S1: Yeah. Almost think of that other case, the simpler one is like something that SMB might use. [00:39:29] S0: Yes. [00:39:29] S1: Like, I've... Yeah. We've got a WordPress blog and [00:39:31] S0: Yes. [00:39:32] S1: A knowledge base and and like your tools makes it super easy just to kinda, like, load up a rag Yes. Personality define an agent and then actually ship it with JavaScript on the website. Yep. Right? All of it. Yeah. Yeah. Pretty cool. I built I built a couple last night. [00:39:45] S0: Oh, nice. Yeah. What did you build? [00:39:47] S1: Well, I built an AI Tinkerers frequently asked questions. You know? And it was... I have to admit it. There's a little bit of tuning Yes. Needed, you know, in the system prompt and that I didn't quite... Because I I I put out our frequently asked questions. I don't know if you've read. It's very specific. It's very opinionated. [00:40:01] S0: Yes. [00:40:02] S1: You know? And there's a question [00:40:03] S0: That's all the best community
00:40:03 · Luke Harries, Joe Heitzeberg
[00:40:03] S0: That's all the best community [00:40:03] S1: to Questions like, you know, literally, like, you know, I'm a Harvard MBA. I sold my company for 500,000,000. I'm not technical, but I've got a groundbreaking idea, and I would be a real asset to the community. I'm super well connected. I just wanna go, and you should make an exception for me. And it's like, no. [00:40:20] S0: Yes. And then... [00:40:21] S1: So I built an agent with that whole repo of, like, questions, and and I asked that question or something similar. And it was like, well, maybe you might be of value to us. Let's see. Why don't you... Why don't we try once? And I was like, no. You know? And so it made me feel like there's some rag tuning [00:40:37] S0: Yes. [00:40:38] S1: That either I have to do or or And you have [00:40:40] S0: full access to the system prompt to do that. Yeah. That's... Did you... By the way, did you integrate client tools as well? [00:40:47] S1: I I looked at the webhook for that and I I thought about... And I think if I did that, I'd get exactly what I want. [00:40:53] S0: Well, there's... Yeah. Well, actually, I was thinking... So I think you should be able to solve your problem just through the system prompts because it's a... You can select any model. You could do Anthropic, you can do Gemini, you can do OpenAI.
00:40:53 · Luke Harries, Joe Heitzeberg
[00:40:53] S0: Well, there's... Yeah. Well, actually, I was thinking... So I think you should be able to solve your problem just through the system prompts because it's a... You can select any model. You could do Anthropic, you can do Gemini, you can do OpenAI. [00:41:05] S1: You know what I did with the system prompts? So rather than actually solve the problem I described is I was like, okay. Now I want you to spell every word in in a Boston accent. I use I use at ElevenLabs, like, gruff voice, and I tried to make it sound. [00:41:18] S0: Voices remain. [00:41:19] S1: Just as fun. [00:41:19] S0: Finding a voice. Well, Joe, you should cloak your I don't know. Genuinely. And use that for... We can actually demo. I've got my voice in this panel. [00:41:26] S1: Well, you know, I have a lot of emails for the organizers that are that are me. Yes. Aren't me. [00:41:31] S0: Yeah. I I I am sure we're getting to it. But the... To show you the, it's a pretty cool demo. I'll just show you if you want, which is, and then we can jump into how you build these after. Yeah. Is there's actually two modes of tools. We have, like, server side tools [00:41:46] S1: Yeah. [00:41:47] S0: And... But we also have client side tools. [00:41:49] S1: Mhmm.
00:41:49 · Joe Heitzeberg, Luke Harries
[00:41:49] S1: Mhmm. [00:41:49] S0: And so this can run, execute JavaScript on your behalf Mhmm. Or like cool. Well, it just returns which tool choose and the args. Yeah. But like, you can have it do stuff like switch the page. Mhmm. So you can be like, oh, I'd love to go to a meetup in Warsaw, and they will like take you immediately to the page. And then it goes great, like give me a quick summary of yourself. Yeah. And then they could like pre fill the entire thing for you. Yeah. Which could be pretty cool. But just to show people that one. Yeah. Hey there. I'm Alexis from ElevenLabs support. How can I help you today? Yeah. Hey, Alexis. Can you tell me more about conversational AI? Sure thing. ElevenLabs Conversational AI is a platform that allows you to deploy customized conversational voice agents. It combines speech to text language models Alexa. Text to speech, and Can you tell me a bit more? Can you show me the the client side tools? Okay. I can tell you more about client side tools. Client tools enable your assistant execute. Perfect. I'm now now actually take me to the page. Or sending... Oh, take me to the page. Do you wanna be taken to the documentation page about client side tools? [00:43:03] S1: Yeah. Is this part baked in with the JavaScript widget, the the navigation aspect? Or do I have to build that?
00:43:03 · Joe Heitzeberg, Luke Harries
[00:43:03] S1: Yeah. Is this part baked in with the JavaScript widget, the the navigation aspect? Or do I have to build that? [00:43:09] S0: I think I may be... The tricky... Where it's still... Let me just get it. Okay. Yes. Take me to the page. Let's see. There we go. And now it's jumped to the jumped to the actual page. Oh, isn't that cool? Yeah. I think... I'm not sure. We definitely have in our code, in our docs. We actually have that how to implement [00:43:32] S1: that. Okay. [00:43:33] S0: So you can just grab it and paste [00:43:34] S1: it in. [00:43:34] S0: Okay. I think we do probably a little bit of asking you to do that thing if you want to to set something. [00:43:41] S1: This is the exciting thing for me. Yeah. I've I've thought a lot about the platform we built for Tinkerers has so many functions from invoicing and scheduling and emails and screening and data and that... Why build more UI? It's just getting more complex. If I can expose the primitives through tools to something like that, you know, our users are familiar with what's available. And if they have a good experience, I think they'll quickly ask to do the right things. Right? Does that make sense? [00:44:08] S0: 100%. [00:44:08] S1: 100%. Very interesting. And it's, know, voice is the best mobile app experience if you really have a great one that's smart.
00:44:08 · Joe Heitzeberg, Luke Harries
[00:44:08] S1: 100%. Very interesting. And it's, know, voice is the best mobile app experience if you really have a great one that's smart. [00:44:14] S0: 100%. I'll just show very briefly to create one. What you can do is you can name your agent. So say we've got... So we do a... Should we do a AI Tinkerers? We can try and do a quick... Well, we do we do a support agent, maybe bring in a bit of AI Tinkerers stuff in it. Sure. And then you click create. And straight away there, you actually have a support agent, which is ready to go. Mhmm. You can select the voice. So should we try doing it with my voice? Luke Harries. So I did my own voice clone recently, which is pretty fun. And then, hi, I'm Luke from ElevenLabs. Luke Harries from ElevenLabs support. And you'll just... I'm just entering this information into the system prompt in the first message. You can also do, as we discussed, like, client side tools. You can do a end call. So at the end of call, it sends an... [00:45:14] S1: Oh, [00:45:14] S0: cool. Oh, yeah. Sorry. Gives the ability to end the call, but you can also do, like, a web hook when a call ends. [00:45:19] S1: You can [00:45:19] S0: send transcript to your own own knowledge base. Nice. And and then any other bit? I think that's probably... Oh, yeah. Here's the widget if you want to add it to your own site. [00:45:29] S1: And we didn't talk about that. You can import a Twilio number. So get a number on Twilio. There's some off workflow that makes it easy to put in. That's really cool.