AI Tinkerers - "One-Shot"
00:24:37 · Joe Heitzeberg, Luke Harries
[00:24:37] S1: Got the community to build and and talk, but more questions than answers at this point. Yeah. It's very early. [00:24:44] S0: There's an interesting thing whereby... Actually, I think MTPs are, you know, if you think about this WhatsApp MCP, everything's on your own computer. Mhmm. It's actually way more secure than the vast majority of cloud applications where the developers read access to the DB. Yeah. So like what's pretty cool is it's kind of somewhat private by default and like a lot more than than lots of things. [00:25:07] S1: Sure. [00:25:07] S0: And when I was starting to think like, okay, how could I actually do like a hosted version of this so you're not needing to like, you know, basically non technical users can do it. [00:25:17] S1: So you don't to install Go. [00:25:19] S0: Yes. You don't have to install Go or or UV. And and what I landed on was what I would really love is a cloud environment, which is like end to end encrypted on the user's behalf where you can execute the code. And I don't I don't think it's... I mean, this is still an early thought. But what's cool is when you run something in your own AWS or Azure or GCP, you have full control of the execution and the data. And I would love if with MTPs or some way where you almost have your own hosted environment that you fully run Yeah. That you're like approving the updates, that you do some reasonable payrolls. So... But [00:25:58] S1: The the meetup I mentioned that was security focus, the the top rated presenter [00:26:03] S0: Yes.
00:26:03 · Luke Harries, Joe Heitzeberg
[00:26:03] S0: Yes. [00:26:03] S1: Was a a person, from a... Tanya, from a person... A company called TinFoil. They do this. [00:26:09] S0: Oh, really? [00:26:10] S1: Secure end and enclaves, cryptographically secured to the hardware. Yeah. [00:26:15] S0: That's very cool. [00:26:16] S1: And with a cofounder from MIT, he's a world expert in that and knows everyone in that niche within PhD computer science, and they're building the company on that. They were a few months away from... Later, I caught up with them. We actually did an episode of One Shot with Tanya because it was so good to hear about this, and that that seems to be the itch that they're scratching. Mean, I exactly what you described. Yeah. And then, we sat down to do the interview, they're like, we just got into YC yesterday. It was a great timing. So pulling them full circle on a lot of these things. Yeah. That's an interesting and a strange thing. [00:26:46] S0: I think either that or, like, could we just get running running on smartphones? And actually, the company I'd love to see. Yeah. [00:26:53] S1: Yeah. Of course. The company [00:26:54] S0: I'd love to see the most just doing this is Apple. Yeah. Because I think Apple has, like, everything set up for them, but we're we're not there just yet. [00:27:04] S1: It's a laugh. I mean, I am frustrated using wonderful tools like Whisper and Super Whisper on my Mac to transcribe, and I'm addicted to that. [00:27:13] S0: Well, that's [00:27:13] S1: I use it for everything. [00:27:14] S0: That maybe takes us nicely into the second hack I'd love to show.
00:27:14 · Luke Harries, Joe Heitzeberg
[00:27:14] S0: That maybe takes us nicely into the second hack I'd love to show. [00:27:17] S1: Well, now we get... ElevenLabs get in there too, [00:27:19] S0: but Yes. [00:27:20] S1: I... And we will, but, I'm so addicted to using transcription in that modality now that I can't use Apple. It's terrible. No. Apple cannot spell the word hackathon. [00:27:29] S0: Yes. [00:27:29] S1: Apple will take the word there and use the wrong version of it in a sentence. I know. It just... An LLM can't even make that mistake if you try to get [00:27:39] S0: it more time, like, tapping around in the message [00:27:41] S1: to try and read the script. Yeah. It's terrible. Now let's talk about then... Let's go to the next topic. Yeah. We're talking about voice
00:27:41 · Joe Heitzeberg, Luke Harries
[00:27:41] S1: to try and read the script. Yeah. It's terrible. Now let's talk about then... Let's go to the next topic. Yeah. We're talking about voice [00:27:48] S0: now. So I was literally ran into the exact issue we're talking about, and I haven't actually demoed this to Joe just yet. So ElevenLabs, we have... We were initially famous for our text to speech models where we were the first company to cross this threshold of human like speech. Mhmm. We more recently shipped the best speech to text model, which is the harsh quality translation. Mhmm. And so what I... I was actually having lunch with my wife, and we're waiting for the bill, and I was like, how is iPhone dictation so bad? Mhmm. And I was like, surely there's a way where I can just speak into it. I can transcribe it with ElevenLabs. Mhmm. I can then just like clean up the raw transcript Mhmm. And then paste it back in. Mhmm. And so what I thought was interesting, I was like, oh, okay. How could I do that? Do I need to build an app? [00:28:39] S1: Do I [00:28:39] S0: have it? I was like, well, Apple Shortcuts is the thing. Oh, yeah. How far can I push Apple Shortcuts? Yeah. So the way I've set it up is all you do is you hold down the action button [00:28:47] S1: Wow. You assign that to be a shortcut. Yeah. Okay. Yeah. [00:28:50] S0: Cool. So you hold down the action button. Hey, Joe. Really excited to do our podcast called two shot... Sorry, one shot, this coming week. Can't wait to see you on Thursday. Friday. Can't wait to see you on Friday. Best wishes, wishes, Luke. [00:29:08] S1: 10AM, not 9AM.
00:29:08 · Joe Heitzeberg, Luke Harries
[00:29:08] S1: 10AM, not 9AM. [00:29:10] S0: And the way it works, you can see in the top right here, it's now processing that shortcut. Cool. It then sends it to ElevenLabs, sends it to to OpenAI to clean up the transcript, and then you can just paste it in. And here we've got, hey, Joe. Really excited to to do our podcast called OneShot this coming week. Can't wait to see you on Friday. Best wishes, Luke. And it's all cleaned up. Great communication. You can call and send. [00:29:35] S1: Well, I'm gonna definitely get that set up. [00:29:37] S0: Yes. Because [00:29:37] S1: Siri is driving me crazy. [00:29:39] S0: The one... I mean, this is maybe a brief tangent, but, like, Apple Shortcuts is so powerful. Yeah. And the the latest hack I've been trying to do is like, I've realized, well, one is like, if you can just hold down the button, do a voice note and then execute actions. Wow. That's powerful for so many follow on Yep. Tasks. The other really cool one though is the vision models have got so good. Like Mhmm. Why not hold it down? That triggers a shortcut, which takes a screenshot Mhmm. Sends that to an LLM to work out the next step. Mhmm. So you basically have this, like, full context, super smart AI assistant. [00:30:16] S1: That's a good idea. [00:30:16] S0: So you could do, like [00:30:17] S1: Good idea. [00:30:17] S0: We're chatting. You're like, I need it to draft a reply. You just hold it down, takes the screenshot Yeah. Does the reply for you, paste it back in.
00:30:17 · Luke Harries, Joe Heitzeberg
[00:30:17] S0: We're chatting. You're like, I need it to draft a reply. You just hold it down, takes the screenshot Yeah. Does the reply for you, paste it back in. [00:30:25] S1: You know those apps... Know those apps that'll do a long screenshot of a scroll all the way? Yes. So imagine that. Now I'm giving the LLM the context, and then I'm the voice of instruction. [00:30:33] S0: Yeah. Yeah. [00:30:33] S1: Hey. Take this page. Make an email to this. Send it to Luke. Ask him a couple questions about the blog post. That'd be very [00:30:39] S0: good. Maybe we should hack that Let's [00:30:41] S1: do it. [00:30:41] S0: Ten minutes after. But honestly, you you could... The only trick... I don't know if you can get that long screenshot in most apps. I only seen some apps actually. [00:30:50] S1: Yeah. Yeah. [00:30:51] S0: But even the stuff on the screen is probably enough. [00:30:53] S1: And probably enough for most cases. You could always zoom out or Yeah. Pinch to zoom. [00:30:56] S0: One one other side point is I worked out that ChatGPT can actually generate Apple Shortcuts. Yeah. And and there's there's an annoying bit whereby you have to sign it. So you have to generate it on your Mac. Mhmm. You then have to run a sign command. Mhmm. But then if you click the link, you then have it instantly loaded Yeah. In your Mac and in your phone. [00:31:17] S1: Yeah. Nice. [00:31:18] S0: And what's cool about Apple Shortcuts as well is that exact workflow I showed you on my phone. [00:31:23] S1: It works on [00:31:23] S0: your Mac. [00:31:24] S1: That's cool.