AI Tinkerers - "One-Shot"
00:11:51 · Joe Heitzeberg, Luke Harries
[00:11:51] S1: Just for people who don't know what we're talking about, you install the the server and run it, does its indexing, and then it literally prints on the terminal output a QR code. [00:11:59] S0: Yes. [00:11:59] S1: And you scan, and that's the auth for WhatsApp. [00:12:01] S0: Yes. [00:12:02] S1: Okay. So now I have a secure connection, and I'm connecting via that secure connection from my laptop directly to WhatsApp [00:12:08] S0: Yeah. Just like they the [00:12:10] S1: Their own API. [00:12:11] S0: Browser WhatsApp web. Okay. [00:12:12] S1: Yeah. And then... Okay. So then when I issue a query in in, in Claude Yes. And it says, okay. I need to call this tool or do this Yeah. MCP call. What's happening then? [00:12:22] S0: Yeah. So the vast majority of read calls Mhmm. It's already got all the messages saved in this local SQLite. [00:12:29] S1: Makes sense. [00:12:30] S0: And so it can just clear... And so the cool thing is, like, you know, WhatsApp's very precious and fantastic service Mhmm. But you really don't want to get blocked by it. Yeah. And so you can do, like, as much querying as you want this local Yeah. SQLite bit. And that's all local. And then if... When you come to send messages, that's when it's actually, like, fires back through WhatsApp web. There's a few other cool bits, which is [00:12:53] S1: What about the info... If I said, you know, summarize my conversations, are you taking all that SQLite cache, pushing it up to somewhere?
00:12:53 · Joe Heitzeberg, Luke Harries
[00:12:53] S1: What about the info... If I said, you know, summarize my conversations, are you taking all that SQLite cache, pushing it up to somewhere? [00:12:59] S0: So it's... The way that the MTP interface works is you expose tools. Mhmm. And so one of the tools is query my messages. Yep. And it, like, literally just... You can pass in filters. So it's like, who's it from, date range, keywords, blah blah blah. And all the... And so your MTP client, like Claude Claude. Will write that query. Yep. It will get those messages, but it just goes back to your Claude. [00:13:26] S1: So Claude in the cloud though is gonna compute on my website. So those are the trust factors I have to trust. [00:13:31] S0: But you're actually specifying, like Sure. Yes, run this query. Yeah. And then it gets in, returns it. And the step, like, I've left as an exercise to the reader is, like, you can make it fully secured by doing a... Like, instead of using clause, you could use a local LLM Yep. Which then like... They're never leaving your personal computer. But yes, at the moment, it does it does... The messages you explicitly tell it to search. It will search and expose to call. [00:13:57] S1: I'm jumping jumping a couple steps ahead for a second. The thing that I immediately saw from it, going back to your original tweet around email, WhatsApp, and and calendar, was to build a cloud... Or an MCP server. Yes. Or client. Sorry. I wanna build a a client in my server infrastructure that uses this MCP WhatsApp interface behind the scenes as part of an app that is providing a service.
00:13:57 · Joe Heitzeberg, Luke Harries
[00:13:57] S1: I'm jumping jumping a couple steps ahead for a second. The thing that I immediately saw from it, going back to your original tweet around email, WhatsApp, and and calendar, was to build a cloud... Or an MCP server. Yes. Or client. Sorry. I wanna build a a client in my server infrastructure that uses this MCP WhatsApp interface behind the scenes as part of an app that is providing a service. [00:14:24] S0: Yeah. [00:14:24] S1: So wanna automate my... Part of my WhatsApp interfacing with my community or my business or for my business Yeah. As me [00:14:32] S0: Yes. [00:14:32] S1: Natively as me, but with agents. Right? [00:14:34] S0: So what's... That's why I actually built first. Yeah. And so the way I built it is I was... If you look at the code... [00:14:46] S1: We should look at the Yeah. [00:14:47] S0: I can show you the code briefly. So, yeah, what you've got here, you've got this bridge, which is all the Go files, and you've got the MTP server, and so these are the Python files. And if you look, I've actually have an extraction whereby whatsapp dot py is all the, like, business logic. But then there's a very thin wrapper in main dot py, which just takes each of the functions in WhatsApp dot py, adds a nice arg string, and adds a at MTP dot tool, which is just like a Python line Mhmm. Python client to expose it. But but... And what this does is it makes it very easy for your MTP clients to access your MTP tools. Mhmm. But what's interesting about this MTP system is it's actually mainly set up for, like, human interaction in the loop doing approvals.
00:14:47 · Luke Harries, Joe Heitzeberg
[00:14:47] S0: I can show you the code briefly. So, yeah, what you've got here, you've got this bridge, which is all the Go files, and you've got the MTP server, and so these are the Python files. And if you look, I've actually have an extraction whereby whatsapp dot py is all the, like, business logic. But then there's a very thin wrapper in main dot py, which just takes each of the functions in WhatsApp dot py, adds a nice arg string, and adds a at MTP dot tool, which is just like a Python line Mhmm. Python client to expose it. But but... And what this does is it makes it very easy for your MTP clients to access your MTP tools. Mhmm. But what's interesting about this MTP system is it's actually mainly set up for, like, human interaction in the loop doing approvals. [00:15:42] S1: Yeah. [00:15:42] S0: I don't think really anyone's using MTP clients in a fully automated way. But why I've done it this way is you can actually wrap all these whatsapp.whatsapp.py tools, and instead just expose them to any other LLM framework. Framework. [00:16:00] S1: Mhmm. [00:16:00] S0: So another, like, experiment I did with this was you can connect it to the OpenAI agents SDK Mhmm. And you can just pass those in. I think the... And then it's like a very clear, you know, it's like any agentic system can call it in the loop. [00:16:15] S1: Sure. Yeah. Cool. Which, I mean, makes sense. I I don't wanna always be in the loop for things that are clearly trivial.
00:16:15 · Joe Heitzeberg, Luke Harries
[00:16:15] S1: Sure. Yeah. Cool. Which, I mean, makes sense. I I don't wanna always be in the loop for things that are clearly trivial. [00:16:20] S0: No. No. What... What's quite nice is, like, when most of the usage is is exactly as you said. It's like people normally in group chats often in places like Brazil or India where lots of, like, businesses or groups are running on it. So you can imagine you have, like, a fitness WhatsApp group Yeah. Or a voice AI WhatsApp group. Mhmm. And you want to basically, like, do a roundup of all the messages of the last week, like, you can kind of run that. But exactly, like, someone like you who's a a Tinkere would probably actually like, okay. I've done that once. Now how do I actually put that in the loop? [00:16:55] S1: Yeah. Yeah. It makes sense. And I want sort of something that's, you know, especially in WhatsApp, there's lots of groups I'm part of that I'm not gonna be spending my entire day watching every thread. Yes. But occasionally, there's a thread that breaks out that I definitely wanted to pay attention to. [00:17:10] S0: And what's very cool as well is, like, the advanced search. Yeah. So you could be... You're in a bunch of WhatsApp groups, and you're like, oh, like, what's the, coolest things happening about MTP or conversational conversational AI? AI? And you just ask it. Yeah. We'll do the search and then do the roundup for you and then synthesize it all together. [00:17:28] S1: What... So one of the groups of mine is doing that. I'm wondering if they're using your MTP.
00:17:28 · Joe Heitzeberg, Luke Harries
[00:17:28] S1: What... So one of the groups of mine is doing that. I'm wondering if they're using your MTP. [00:17:31] S0: It it may well may well be. Yeah. But the... I mean, the hard stuff truthfully is this what's meow bit under the hood. This is... Basically just done the nice [00:17:40] S1: Do worry about being blocked? So I You won't be blocked. The the user of it will be [00:17:46] S0: Yeah. The bit I need to add a risks bit to this Uh-huh. Repo. But the... I think read is very, you know, pretty safe. Yeah. And it's also... If you think about the spirit of WhatsApp, it's like not actually going Yeah. You know, you're reading your messages and querying your message. Do do... I think the mass sending though, that is scary. Yeah. And you should be careful if you're trying to do WhatsApp. [00:18:07] S1: And I think... I mean, some kinda high level conceptual view, these businesses, WhatsApp and etcetera, were designed for human interaction. [00:18:15] S0: Yes. [00:18:16] S1: And, you know, we may build agentic systems that use those pretending to be humans, but the business is disrupted. Right? Yeah. At some level. So they're not gonna like that, or they're gonna have to adapt. [00:18:25] S0: Yes. [00:18:25] S1: And there's gonna be a mess. You know, actually, two weeks ago, the first time I... You know, we've all seen lots of captchas. I saw a crazy one. This captcha made me listen to three bits of sound [00:18:35] S0: Mhmm.