AI Tinkerers - "One-Shot"
00:00:15 · Joe Heitzeberg, Luke Harries
[00:00:15] S1: This conversation with Luke was so interesting and so inspiring. Here's someone who's at the very heart of one of the hottest companies, ElevenLabs, a company revolutionizing voice AI right now. Yet he's constantly experimenting, building, and sharing practical tools that anyone can learn from. He's built some of the most popular MCP servers that are out there, and he takes us under the hood of one of them, the WhatsApp MCP server, the tech, the launch, how he built it, the feedback he's gotten. He also shows a very useful iPhone shortcut that can really enable all sorts of useful applications. It's elegant and surprisingly powerful. And then finally, we dive into ElevenLabs latest APIs, which I think most people aren't familiar with, that allow you to create a full, like, rag stack with your own data and a webhook back to do virtually anything you need. So, like, voice agents in a few clicks, honestly, it was really cool and very worthwhile. I think you're gonna love this conversation. Here we go. Hello, AI Tinkerers, and welcome to OneShot. This is our global stage where we talk to people who are working on the very frontiers of generative AI in some way. I'm thrilled we're here today with Luke Harries of ElevenLabs, who itself is a leader, obviously, in Gen AI and a a huge growth story and an amazing innovator. But but Luke himself also is an incredible tinkerer with an enormous, toy box of cool contraptions and things he's built. So we'll look at a couple of those today. But welcome, Luke. [00:01:49] S0: Thank you. Excited to be here, Joe.
00:01:49 · Luke Harries, Joe Heitzeberg
[00:01:49] S0: Thank you. Excited to be here, Joe. [00:01:51] S1: It's really wonderful. Let's dive in. Like, you lead growth at ElevenLabs, and I immediately think marketing, but you are not a marketer. You're a pretty hardcore techie. [00:01:59] S0: Yeah. So I've been coding since I was 12, 13. I have a weird background where I ended up in medical school for [00:02:06] S1: a few [00:02:06] S0: years, but spent all my time building. Yeah. And and then, yeah, basically was was hacking around on my own projects, and then Matti phoned me up. Matti, the founder of ElevenLabs, and I actually met during a hackathon ourselves about nine years ago, stayed in touch. Oh, wow. And he phoned me up going, Luke, ElevenLabs is growing very quickly. We need someone to join and do the growthy things. Yeah. And I was like, I've never really done the growthy things before, but I'm up for learning it. [00:02:32] S1: I... You'd clearly are trying to figure out from the outside and what's on the web is like it's a renaissance story. Right? It's like you're studying pre med, but you also created, like, some sports platform with code. It was like, obviously, you're a builder. And tell me more about that one just really quick. [00:02:49] S0: I mean, yeah, that one was... And so I was playing rugby. And [00:02:54] S1: You were at Cambridge. At Cambridge. Right. [00:02:55] S0: Yeah. Yeah. And I was, playing rugby, captaining the... What's called the lions team, the, like, third team playing for the second. [00:03:03] S1: Okay.
00:03:03 · Joe Heitzeberg, Luke Harries
[00:03:03] S1: Okay. [00:03:03] S0: And and it was my job to organize all the rugby leagues. Yeah. And when I was handed over this, and it was literally before we were playing, like, finals matches, not realizing the points different mattered. And I first, like, MVP of it was, like, an incredibly complex Google Sheets with, like, Apps Scripts running in the background. And then I was like, this is actually quite useful. So it turned into a, like, full platform web app. Yeah. Scaled that across... [00:03:29] S1: Was that your first... I mean, if you were coding since a kid, was it the first time you got the sort of endorphin rush feedback loop from real usage and... Or would you have that? [00:03:37] S0: I think that was the first one. I built a bunch of, like, iPhone apps for myself. Yeah. But none of them I, like, properly got out in front of users. [00:03:45] S1: Yeah. And then you went on to do some... You graduated and did not go into med at all. You went and did software engineering [00:03:51] S0: Yes. [00:03:52] S1: Consulting, it looks like. You also did some cancer... Some ML. You started getting into the deeper end of of the the things. It's cancer research Yes. [00:04:01] S0: Yeah. Biotech stuff. [00:04:02] S1: Biotech stuff, but ML. [00:04:04] S0: Yes. Yeah. Yeah. [00:04:04] S1: Yeah. Fascinating. And [00:04:05] S0: then using image models. [00:04:07] S1: Was that around the time that you met, the ElevenLabs founders at the hackathon? [00:04:11] S0: Yeah. Yeah. When I was 19.
00:04:11 · Luke Harries, Joe Heitzeberg
[00:04:11] S0: Yeah. Yeah. When I was 19. [00:04:12] S1: Well before ElevenLabs. Okay. And then you also did, so, again, this Renaissance background with lots of interest, but also united by building and tinkering. You did, you were a YC founder? You were [00:04:25] S0: Yeah. [00:04:25] S1: Head of growth or product at PostHog? Yes. Okay. And that's one where it's like, I think PostHog's story is partly like every Y Combinator company just used them. That was [00:04:34] S0: like Yeah. Was was moving on from my previous company. Yeah. I phoned up Dalton, our YC group partner, and I was like, Dalton, what should I do? I'm working out my next move. He's like, postdocs in London, they're doing pretty cool stuff. You should chat with them. [00:04:49] S1: There you go. The power of community. [00:04:50] S0: Exactly. You know, jump in [00:04:52] S1: with a community where great things are happening. You can always jump ship or do something or combine forces. So you don't know what's gonna take off. No. Right? But you're in the middle of it. That's really cool. Let's talk about the tinkering. I wanna see something like... I know you've done a bunch of stuff that blew up pretty recently on Twitter with MTP. [00:05:07] S0: Yes. [00:05:08] S1: You know, again, it became popular to kinda talk about MTP, and you were like, oh, I need to go build something. So what did you build? That's
00:05:08 · Joe Heitzeberg, Luke Harries
[00:05:08] S1: You know, again, it became popular to kinda talk about MTP, and you were like, oh, I need to go build something. So what did you build? That's [00:05:14] S0: that's Yeah. Well, a little bit of the backstory. Yeah. I I did a tweet. The the funky thing I think about the current wave of AI is there's so much innovation on, like, the business and b to b side. Mhmm. But the consumer side is terrible. And I think it's just because there's companies like Meta and Google who own the consumer interfaces, who are fantastic at AI, but just, like, seemingly aren't shipping the product. So I did this tweet, which is like, I think you can basically solve consumer AI by connecting a AI agent to your WhatsApp, your calendar, and your Gmail. And if you do that, you basically have the entire context on someone's life as well as actually being able to execute actions because you could create group chats, you could send emails, you have all their invoices. So I tweeted this being like, why has no one done it? And I thought about it a little more, and I was like, well, the hard bit's actually the WhatsApp bit. So, like, how do you connect WhatsApp to AI? So that's why I was like, okay. I wonder if I can actually do this. [00:06:13] S1: What what makes that hard? [00:06:14] S0: But the hard bit is WhatsApp has business APIs, but doesn't have personal APIs. And so just being able to query your messages is actually surprisingly difficult to send messages. [00:06:26] S1: Interesting. They don't have just a basic API for, like No. Search
00:06:26 · Joe Heitzeberg, Luke Harries
[00:06:26] S1: Interesting. They don't have just a basic API for, like No. Search [00:06:29] S0: No. It's very like walled garden. Interesting. [00:06:31] S1: So how did you get in? [00:06:32] S0: Yeah. So I went down a couple of different routes, like, basically, you know, did the long tail search of GitHub repos powered by by Deep Research. And the first time I tried, there's this protocol called Matrix Okay. And which tries to abstract all the different messaging apps into one. It's like very cool open source project. Yeah. Beeper was built on top of it. Okay. And Yeah. I tried to get that running and build on top of that abstraction, but I'm not sure if it was me or the docs, but that was a disaster. Okay. And and this was all on a Saturday, by the way. So I did, like, our trend matrix. And then I was like, well, okay. What's actually underneath matrix? Oh, there's this cool Go library, which is written Okay. And called what's meow. Uh-huh. And I and was like, oh, like, I wonder if that works. Yeah. And so I pulled up Cursor, start hacking away. I've never written go in my life. Right. And yeah, just like with Cursor, managed to get the fur... My first goal was like, can I get a local copy of my WhatsApp messages on my computer so I can do all the querying? Mhmm. And so we wrote... I say we, me and cursor, me and Claude, point 3.5. Yeah. Wrote this thing so when it connects... The library connects to the WhatsApp device API. Mhmm. So it pretends to be the browser client. And actually, when you start the client, the first thing it does is load all your messages.