AI Tinkerers - "One-Shot"
00:10:48 · Luke Harries, Joe Heitzeberg
[00:10:48] S0: Yes. [00:10:48] S1: Yeah. Yeah. Worked perfectly. And then first thing I did was, like, summarize all my recent conversations. There it was, you know, you know, you're on a crypto spam list, and you... Looks like you're talking about this with these people and, you know, breakfast with Luke tomorrow or whatever. And it was, it was great. And I was a little scared to use it to send messages. I'm not a heavy user, and I was, but I could tell, like, this is this is interesting, the simplicity of these things. A bunch of questions, technically, guess, for Yeah. The motivations are like, where is the data going? You know? When it's in my WhatsApp app and it's encrypted, now I'm on MCP. I've installed your thing. It's sub purpose running on my laptop. Part of it's inferring in the cloud. [00:11:26] S0: Well, no. Actually, the only thing it's running is... So it uses the WhatsApp device API. Yeah. So it speaks directly to WhatsApp. [00:11:34] S1: Okay. [00:11:34] S0: And all your messages are on your computer. [00:11:36] S1: They So be the exact same. The auth was a great step, by the way. The QR code that spits out in the terminal. That was cool. Yeah. Yeah. Worked great. [00:11:43] S0: I don't know how how I cursor did that, but worked out. It was... So it's like At first, it printed out a raw string, and I was like, turn it into turn it into QR code, and it did. [00:11:51] S1: Just for people who don't know what we're talking about, you install the the server and run it, does its indexing, and then it literally prints on the terminal output a QR code.
00:11:51 · Joe Heitzeberg, Luke Harries
[00:11:51] S1: Just for people who don't know what we're talking about, you install the the server and run it, does its indexing, and then it literally prints on the terminal output a QR code. [00:11:59] S0: Yes. [00:11:59] S1: And you scan, and that's the auth for WhatsApp. [00:12:01] S0: Yes. [00:12:02] S1: Okay. So now I have a secure connection, and I'm connecting via that secure connection from my laptop directly to WhatsApp [00:12:08] S0: Yeah. Just like they the [00:12:10] S1: Their own API. [00:12:11] S0: Browser WhatsApp web. Okay. [00:12:12] S1: Yeah. And then... Okay. So then when I issue a query in in, in Claude Yes. And it says, okay. I need to call this tool or do this Yeah. MCP call. What's happening then? [00:12:22] S0: Yeah. So the vast majority of read calls Mhmm. It's already got all the messages saved in this local SQLite. [00:12:29] S1: Makes sense. [00:12:30] S0: And so it can just clear... And so the cool thing is, like, you know, WhatsApp's very precious and fantastic service Mhmm. But you really don't want to get blocked by it. Yeah. And so you can do, like, as much querying as you want this local Yeah. SQLite bit. And that's all local. And then if... When you come to send messages, that's when it's actually, like, fires back through WhatsApp web. There's a few other cool bits, which is [00:12:53] S1: What about the info... If I said, you know, summarize my conversations, are you taking all that SQLite cache, pushing it up to somewhere?
00:12:53 · Joe Heitzeberg, Luke Harries
[00:12:53] S1: What about the info... If I said, you know, summarize my conversations, are you taking all that SQLite cache, pushing it up to somewhere? [00:12:59] S0: So it's... The way that the MTP interface works is you expose tools. Mhmm. And so one of the tools is query my messages. Yep. And it, like, literally just... You can pass in filters. So it's like, who's it from, date range, keywords, blah blah blah. And all the... And so your MTP client, like Claude Claude. Will write that query. Yep. It will get those messages, but it just goes back to your Claude. [00:13:26] S1: So Claude in the cloud though is gonna compute on my website. So those are the trust factors I have to trust. [00:13:31] S0: But you're actually specifying, like Sure. Yes, run this query. Yeah. And then it gets in, returns it. And the step, like, I've left as an exercise to the reader is, like, you can make it fully secured by doing a... Like, instead of using clause, you could use a local LLM Yep. Which then like... They're never leaving your personal computer. But yes, at the moment, it does it does... The messages you explicitly tell it to search. It will search and expose to call. [00:13:57] S1: I'm jumping jumping a couple steps ahead for a second. The thing that I immediately saw from it, going back to your original tweet around email, WhatsApp, and and calendar, was to build a cloud... Or an MCP server. Yes. Or client. Sorry. I wanna build a a client in my server infrastructure that uses this MCP WhatsApp interface behind the scenes as part of an app that is providing a service.
00:13:57 · Joe Heitzeberg, Luke Harries
[00:13:57] S1: I'm jumping jumping a couple steps ahead for a second. The thing that I immediately saw from it, going back to your original tweet around email, WhatsApp, and and calendar, was to build a cloud... Or an MCP server. Yes. Or client. Sorry. I wanna build a a client in my server infrastructure that uses this MCP WhatsApp interface behind the scenes as part of an app that is providing a service. [00:14:24] S0: Yeah. [00:14:24] S1: So wanna automate my... Part of my WhatsApp interfacing with my community or my business or for my business Yeah. As me [00:14:32] S0: Yes. [00:14:32] S1: Natively as me, but with agents. Right? [00:14:34] S0: So what's... That's why I actually built first. Yeah. And so the way I built it is I was... If you look at the code... [00:14:46] S1: We should look at the Yeah. [00:14:47] S0: I can show you the code briefly. So, yeah, what you've got here, you've got this bridge, which is all the Go files, and you've got the MTP server, and so these are the Python files. And if you look, I've actually have an extraction whereby whatsapp dot py is all the, like, business logic. But then there's a very thin wrapper in main dot py, which just takes each of the functions in WhatsApp dot py, adds a nice arg string, and adds a at MTP dot tool, which is just like a Python line Mhmm. Python client to expose it. But but... And what this does is it makes it very easy for your MTP clients to access your MTP tools. Mhmm. But what's interesting about this MTP system is it's actually mainly set up for, like, human interaction in the loop doing approvals.
00:14:47 · Luke Harries, Joe Heitzeberg
[00:14:47] S0: I can show you the code briefly. So, yeah, what you've got here, you've got this bridge, which is all the Go files, and you've got the MTP server, and so these are the Python files. And if you look, I've actually have an extraction whereby whatsapp dot py is all the, like, business logic. But then there's a very thin wrapper in main dot py, which just takes each of the functions in WhatsApp dot py, adds a nice arg string, and adds a at MTP dot tool, which is just like a Python line Mhmm. Python client to expose it. But but... And what this does is it makes it very easy for your MTP clients to access your MTP tools. Mhmm. But what's interesting about this MTP system is it's actually mainly set up for, like, human interaction in the loop doing approvals. [00:15:42] S1: Yeah. [00:15:42] S0: I don't think really anyone's using MTP clients in a fully automated way. But why I've done it this way is you can actually wrap all these whatsapp.whatsapp.py tools, and instead just expose them to any other LLM framework. Framework. [00:16:00] S1: Mhmm. [00:16:00] S0: So another, like, experiment I did with this was you can connect it to the OpenAI agents SDK Mhmm. And you can just pass those in. I think the... And then it's like a very clear, you know, it's like any agentic system can call it in the loop. [00:16:15] S1: Sure. Yeah. Cool. Which, I mean, makes sense. I I don't wanna always be in the loop for things that are clearly trivial.
00:16:15 · Joe Heitzeberg, Luke Harries
[00:16:15] S1: Sure. Yeah. Cool. Which, I mean, makes sense. I I don't wanna always be in the loop for things that are clearly trivial. [00:16:20] S0: No. No. What... What's quite nice is, like, when most of the usage is is exactly as you said. It's like people normally in group chats often in places like Brazil or India where lots of, like, businesses or groups are running on it. So you can imagine you have, like, a fitness WhatsApp group Yeah. Or a voice AI WhatsApp group. Mhmm. And you want to basically, like, do a roundup of all the messages of the last week, like, you can kind of run that. But exactly, like, someone like you who's a a Tinkere would probably actually like, okay. I've done that once. Now how do I actually put that in the loop? [00:16:55] S1: Yeah. Yeah. It makes sense. And I want sort of something that's, you know, especially in WhatsApp, there's lots of groups I'm part of that I'm not gonna be spending my entire day watching every thread. Yes. But occasionally, there's a thread that breaks out that I definitely wanted to pay attention to. [00:17:10] S0: And what's very cool as well is, like, the advanced search. Yeah. So you could be... You're in a bunch of WhatsApp groups, and you're like, oh, like, what's the, coolest things happening about MTP or conversational conversational AI? AI? And you just ask it. Yeah. We'll do the search and then do the roundup for you and then synthesize it all together. [00:17:28] S1: What... So one of the groups of mine is doing that. I'm wondering if they're using your MTP.