Lenny's Podcast: Product | Career | Growth
00:01:08 · Lenny Rachitsky, Brendan Foody
[00:01:08] S1: I talk to a lot of people about what should I be studying? Where should I be getting better? [00:01:12] S0: How can they leverage this technology to do so much more? We'll give people interviews where we say, use whatever tools are available to build a website, and let's see what product you're able to build in an hour.
00:01:24 · Lenny Rachitsky
[00:01:24] S1: Today, my guest is Brendan Fudi, CEO and cofounder of Merkor. Merkor is the fastest growing company in history to go from 1 to $500,000,000 in revenue. They did this in seventeen months, less than a year and a half. Brendan is also the youngest Unicorn founder ever. They just raised a $100,000,000 at $2,000,000,000 valuation. Merkor, if you haven't heard of them, helps AI labs and AI companies hire experts to help them train their models using AI. They've never had a customer churn. Their net retention is over 1600%, and they're on a 9 figure revenue run rate. In our conversation, we talk about the increasing value and importance of evals, the landscape of AI training companies like Merkor and why they've become so important and valuable, how Brendan discovered this opportunity, his insights on what product market fit looks like, the core tenets he's instilled within his organization that have allowed him to build the fastest growing company in history, what people writing evals for labs are actually doing day to day, which skills and jobs are gonna last the longest with the rise of AI, why he doesn't think we'll see AGI or super intelligence anytime soon, and so much more. This episode is incredible. You need to hear this. If you enjoy this podcast, don't forget subscribe and follow it in your favorite podcasting app or YouTube. It helps tremendously. Also, if you become an annual subscriber of my newsletter, you get 15 incredible products for free for one year, including Lovable, Replit, Bolt, n eight, and Linear Superhuman, whisper flow, gamma, perplexity, warp, granola, magic patterns, raycast, chappier d, and mobbin. Check it out at lenny'snewsletter.com and click product pass. With that, I bring you Brendan Foody. This episode is brought to you by WorkOS. If you're building a SaaS app, at some point, your customers will start asking for enterprise features like SAML authentication and SCIM provisioning. That's where WorkOS comes in, making it fast and painless to add enterprise features to your app. Their APIs are easy to understand so that you can ship quickly and get back to building other features. Today, hundreds of companies are already powered by WorkOS, including ones you probably know, like Vercel, Webflow, and Loom. WorkOS also recently acquired Warrant, the fine grain authorization service.
00:03:43 · Lenny Rachitsky
Warrant's product is based on a groundbreaking authorization system called Zanzibar, which was originally designed for Google to power Google Docs and YouTube. This enables fast authorization checks at enormous scale while maintaining a flexible model that can be adapted to even the most complex use cases. If you're currently looking to build role based access control or other enterprise features like single sign on, SCIM, or user management, you should consider WorkOS. It's a drop in replacement for Auth0 and supports up to 1,000,000 monthly active users for free. Check it out at workos.com to learn more. That's workos.com. This episode is brought to you by Jira Product Discovery. The hardest part of building products isn't actually building products. It's everything else. It's proving that the work matters, managing stakeholders, trying to plan ahead. Most teams spend more time reacting than learning chasing updates, justifying roadmaps, and constantly unblocking work to keep things moving. Jira Product Discovery puts you back in control. With Jira Product Discovery, you can capture insights and prioritize high impact ideas. It's flexible, so it adapts to the way your team works and helps you build a road map that drives alignment, not questions. And because it's built on Jira, you can track ideas from strategy to delivery all in one place. Less chasing, more time to think, learn, and build the right thing. Get Jira product discovery for free at atlassian.com/lenny. That's atlassian.com/lenny. Brendan, thank you so much for being here. Welcome to the podcast.
00:05:17 · Brendan Foody, Lenny Rachitsky
[00:05:17] S0: Thank you so much for having me, Lenny. I'm a huge fan and so excited to have a conversation. [00:05:22] S1: I'm really excited to have this conversation as well. I'm a huge fan of yours. I'm excited for more people to learn about you and what you're building. I wanna start with a tweet that you have pinned at the top of your Twitter feed right now. And here's the tweet. Quote, we are now working with six out of the magnificent seven, all of the top five AI labs, most of the AI application layer companies. One trend is common across every customer. We're entering the era of evals. The reason this caught my attention is that's one of the most recurring trends on this podcast. People talking about the increasing value of learning how to do evals well and the value of evals for companies. Feels like still people don't know what the hell this is, what we're talking about, why this is so important. Talk about just what you think people are still missing, what they need to know, what this era of evals means.
00:06:11 · Brendan Foody, Lenny Rachitsky
[00:06:11] S0: If the model is the product, then the eval is the product requirement document. And the way that researchers' day to day looks is that they'll run dozens of experiments where they'll make small improvements on an eval set. And reinforcement learning is becoming so effective that once they have an eval, they can help climb it, right? If you look at just how fast people were able to saturate Olympiad math once they focused on it, how fast we're even saturating SWE bench once we focus on it. And so in many ways, the barrier to applying agents, the entire economy to automate every workflow is how do we measure success? How do we eval it and write the PRDs for everything that we want agents to do, which Merkor is obviously a huge part of doing. [00:06:56] S1: So people hearing this are like, okay, okay, shit, I gotta really pay attention to this eval stuff. Any advice about learning how to do this well? What companies that are doing this well are doing differently? Like help people get better at this thing.
00:06:56 · Lenny Rachitsky, Brendan Foody
[00:06:56] S1: So people hearing this are like, okay, okay, shit, I gotta really pay attention to this eval stuff. Any advice about learning how to do this well? What companies that are doing this well are doing differently? Like help people get better at this thing. [00:07:10] S0: Yeah, I think that for enterprises especially, the core way to think about it is how can they build a test or a systematic way to measure how well AI automates their core value chain? So if it's an architecture firm that's producing these architecture diagrams of what they provide to their end customer, how can they effectively measure that? And each company has its own value chain, or maybe a handful of them if it's a multi product company. And just thinking about how they measure that is the prerequisite to really effectively applying AI throughout their entire business. [00:07:52] S1: I saw you talking about this on the No Priors podcast with Sarah and Elad, and I don't know if it was after this or before this, but Sarah tweeted, Evals equals your new marketing. What do you what does that mean? What do you think she's saying there?