Lenny's Podcast: Product | Career | Growth
00:33:22 · Lenny Rachitsky, Brendan Foody
[00:33:22] S1: It's such a good opportunity for me to learn so much about this. I'm just going to ask questions. Yeah, sure. So interesting to me. How much of the experts are focused on specific concrete knowledge versus personality and like softer skills? How much of it is like, here's how you do an exam, here's how you do an x-ray? [00:33:40] S0: It depends on the lab. It's a lot of both. I think that previously it might've been more softer skills, but now a lot of the labs are focused on their business models of what are the economically valuable capabilities that drive revenue and leaning a lot into these professional domains. But I think the creative side is also still really important to everyone. And so we're seeing a meaningful amount of both. Like we hired all the people from the Harvard Lampoon a couple months ago, their comedy club to help with making models funnier. And so do all sorts of stuff like that, hiring Emmy award winning screenwriters and everything across the board on creative capabilities that you'd look for. [00:34:23] S1: That is amazing. What a cool story. Yeah. I'm excited for this to kick in. How fast do these things turn around? Like, say you hired this team, like how fast are we going to see the impact potentially? Is it like months? Is it years? [00:34:34] S0: Well, it depends because some models, or some labs will release iteratively where they'll just improve the model behind the scenes, sort of every [00:34:42] S1: Without announcing a new model.
00:34:42 · Lenny Rachitsky, Brendan Foody
[00:34:42] S1: Without announcing a new model. [00:34:44] S0: Exactly. As sort of every couple of weeks versus others do these big releases. And so it depends a lot. We're behind all of them, but the, I mean, we move really fast. It would be a customer gives us a request of we need these, you know, award winning screenwriters. And within twenty four hours, we'll turn around, the experts. And there's also this really interesting dynamic where in a set of a 100 people that we hire, oftentimes the top 10% of people will drive majority of the model improvement. It's sort of like a company, right? If you have a 100% company, oftentimes the top 10% of the company, will drive majority of the impact. And what that means is that when we're able to build proprietary advantages in identifying who are those top 10 of people, both insofar as how do we have them on our platform, but also identify and match them effectively, it creates so much value for customers that it's difficult to compete against. And so it really does tie back to the founding thesis of the company, which is how do we find these extraordinary people and identify them so that we can reliably deliver these top 10% or top 10x experiences for our customers. [00:35:56] S1: So on that, so is the idea you hire Jane, she's incredible at coding, and she now works for Anthropic, and that's her full time job doing this, or is this like a part time thing? Is this a project thing mostly?
00:35:56 · Lenny Rachitsky, Brendan Foody
[00:35:56] S1: So on that, so is the idea you hire Jane, she's incredible at coding, and she now works for Anthropic, and that's her full time job doing this, or is this like a part time thing? Is this a project thing mostly? [00:36:09] S0: It would sometimes be part time, sometimes it would be full time. I would say most often it's part time, where it's like, you know, someone might work at fan a company where they're underemployed, maybe one of the ones that's moving slower, where they have an extra twenty hours a week, and then they're able to do this on the side, or, you know, whatever the equivalent is sort of across a bunch of different industries. But we also do a lot of, you know, forty hour a week roles as well. [00:36:39] S1: And these, how much are they making? Is it like, like meaningful enough for Fannie engineers to spend time on this?
00:36:39 · Lenny Rachitsky, Brendan Foody
[00:36:39] S1: And these, how much are they making? Is it like, like meaningful enough for Fannie engineers to spend time on this? [00:36:45] S0: Yeah, very meaningful. I mean, so our median pay rate in the marketplace 95 an hour, but it can flex up well up into like $500 an hour, based on the depth of someone's expertise. And one thing that highlights this difference relative to a lot of the crowdsourcing companies is if you look at the economics of the crowdsourcing companies, oftentimes they would pay like $30 an hour to Talon as sort of the average. And so think about the, you know, people that you can hire, the undergrads for $30 an hour versus the you know, Goldman bankers, the McKinsey analysts, the, FANG software engineers. And ultimately it comes down to what are the capabilities that labs want their models to have? And it much more falls in the latter bucket than the former one. [00:37:33] S1: I know there s only so much you can talk about with this stuff, but so Anthropic Claude has been so good at coding, so much better historically than other models. I also use it for writing, giving feedback on writing. What is it that allowed them to get so good at this and continue to be so good at this? [00:37:51] S0: Well, can't go too much into detail about customer work, but I think that it's this trend of reinforcement learning and being very thoughtful about defining the right rewards that we're really seeing across the board, and how we can mitigate reward hacking, set up the right rewards. That's super impactful. [00:38:14] S1: Evals, [00:38:14] S0: again,
00:38:14 · Brendan Foody, Lenny Rachitsky
[00:38:14] S0: again, [00:38:15] S1: Back evals is [00:38:15] S0: all you to evals. One of my favorite quotes from customers is that models are only as good as their evals, which has always held true. [00:38:24] S1: I think Greg Brockman tweeted this once, evals are all you need. [00:38:28] S0: Yeah, no, truly. [00:38:29] S1: Let's talk about Marcour a little bit more. One of the maybe... Not even maybe. I believe, the data tells us it's the fastest growing company in history. [00:38:39] S0: Yeah. [00:38:40] S1: I wanna understand what you did to make this happen. So let me just ask, what do think are some of the core tenets of how you built Morcord that most contributed to being this successful?
00:38:51 · Brendan Foody
[00:38:51] S0: I think the most important thing is looking at the leading indicators in fast moving markets. Like I remember when I used to think everyone in venture talks about the why now. And I used to think about the why now of how, from a product standpoint, less from a market standpoint of like, now we can automate the way that we review resumes or the way that we conduct interviews, etcetera. But ultimately, there is this legacy market that has all these incumbents and is relatively stagnant. What matters a ton is actually figuring out what are the new markets, the new pockets of demand that are changing very quickly, where the wealthiest customers in the world are willing to pay whatever it takes to improve model capabilities. And how do we focus on the leading indicators of those markets to make sure that we have the best solution for the flagship customers in the market and optimize everything around that? And that's what I found has been most impactful in building the business. I think that's, maybe that's one thing is like leading indicators in markets. If I had to choose another, it's customer obsession. Like we have had for the last, we're starting to like have a couple of, product managers help out with go to market, but like for the last year and a half of the business, we've had no one in sales and marketing. And so we're sort of like immature from a sales and marketing standpoint because we've focused a 100% of company resources on how do we build great products and experiences for our customers. Just getting word-of-mouth, the people that have worked with us at other businesses want to keep working with us and leaning into creating those great experiences. And so that's where I spend all my time. I think that some founders can get caught up in like, how do they get really good at marketing before they've figured out the thing that really drives a lot of customer love and creates the six star experiences that you're used to building.