Luke Harries built a WhatsApp MCP server over a Saturday with Cursor and Claude 3.5, letting AI agents read messages from a local SQLite copy and send through WhatsApp Web.
Luke Harries argues most MCP vulnerabilities sit in the protocol itself and calls for a split between read-only tools and tools that create or update data.
Luke Harries says ElevenLabs' Scribe speech-to-text model still beats the latest OpenAI model and Gemini, and he admits he was wrong to think voice agents would never work due to latency.
ElevenLabs' Luke Harries on wiring AI into WhatsApp and voice
Consumer AI can be built by wiring agents into apps people already use, and voice is becoming the way to talk to them. Luke Harries' weekend WhatsApp MCP server proved the demand and exposed security gaps in the MCP protocol. At ElevenLabs, sub-second latency and stronger speech models turned voice agents from a dead end into something customers kept requesting.
Takeaways
Consumer AI can ride on apps people already use. Luke Harries argued that connecting an agent to personal apps like WhatsApp and Gmail could solve consumer AI. Meta and Google own those interfaces but are not shipping AI products, so outside builders can fill the gap.
MCP needs a read versus write split. Luke thinks his WhatsApp server is probably the most used MCP server, which drew security researchers. He says most flaws sit in the protocol itself. Reading is fairly safe, but sending is dangerous, and a malicious server could forward messages elsewhere.
Privacy depends on which model reads messages. Messages live in a local SQLite copy but go to Claude when queried. Luke suggests a local LLM for full privacy and proposes a hosted, end-to-end encrypted environment so non-technical users can run MCP servers safely.
Voice agents work once latency drops below a second. Luke once believed voice agents would never work and now says he was completely wrong. About 20 customers in two months asked for combined speech tools, leading to an engineer mode where developers supply the brain through webhooks or websockets.
Evaluate the LLM brain, not the voice. Luke argues most evals should target the reasoning layer. ElevenLabs' conversational AI 2.0 adds built-in RAG and a turn-taking model that uses both voice audio and semantic word-level input to decide when to speak.
Scribe beats OpenAI and Gemini on transcription. Luke says ElevenLabs' Scribe model offers speaker diarization, character-level timestamps, tagging of laughter and coughs, and languages such as Gujarati and Serbian. His iPhone action-button Shortcut transcribes with Scribe and cleans up with OpenAI cheaply via raw API usage.
Education is a surprise growth market for voice. Customers include Perplexity, Time Magazine, Practica and chess.com, which built interactive lessons around a Professor Wolf character. Luke also expects conversational characters to change gaming completely, and ElevenLabs is building a studio product that turns PDFs into produced audiobooks.
How the WhatsApp MCP server was built and launched
Luke found WhatsApp offers business APIs but no personal ones.
Tap a name to go there · tap a line to see why they are linked
On this episode 2
Show 1
He tried the open-source Matrix protocol, which he calls a disaster, then switched to the whatsmeow Go library.
Over a Saturday with Cursor and Claude 3.5 he built a server that stores messages in local SQLite and sends through WhatsApp Web.
He spent an hour on a quick video and tweet thread. Garry Tan's retweet drove the first spike and a Hacker News post pushed it past 3,000 GitHub stars.
Luke says personal launches are harder than company launches because of ego, and credits Simon Willison's advice to write about every project.
How ElevenLabs runs on AI tools
Gong analyses sales calls to feed the product roadmap.
Granola records half-hour customer calls and the notes go into a Claude project that drafts case studies.
A custom ChatGPT GPT checks customer communications before they go out.
Engineers code in Cursor, and marketing videos use ElevenLabs voices and sound effects, which Luke calls the most underrated model for video.