Lenny's Podcast: Product | Career | Growth
00:31:02 · Brendan Foody, Lenny Rachitsky
[00:31:02] S0: Exactly, yeah. It's really unlocking, allowing the model to focus on all the right tokens from pre training all the right things in model context, upgrading the effective reasoning chains, to enable the models to reason better in a more generalized way. [00:31:19] S1: What's the scale of people just working on this stuff? Is it like thousands, tens of thousands, hundreds of thousands? [00:31:24] S0: Tens of thousands at any given time, hundreds of thousands more generally. I mean, it's huge. And the most exciting thing is that it's growing really quickly. I mean, I think that to your question also about the competitive landscape, historically there were all these crowdsourcing companies that would get these super high volumes of low skilled people. I think like Scale and Surge were the primary companies that pioneered that industry. And then in this transition to higher skilled labor, what people realized is that actually you can go a lot further with just getting higher caliber people, even in smaller amounts initially, and now subsequently scaling that back up once they're able to meet the quality bar. I think that there's a bunch of companies that after our success and very rapid revenue growth that sort of started early last year have chased after that, which makes sense, right? And seeing that the market was changing very quickly, we were taking off and trying to pursue a similar thesis on the market.
00:31:24 · Brendan Foody, Lenny Rachitsky
[00:31:24] S0: Tens of thousands at any given time, hundreds of thousands more generally. I mean, it's huge. And the most exciting thing is that it's growing really quickly. I mean, I think that to your question also about the competitive landscape, historically there were all these crowdsourcing companies that would get these super high volumes of low skilled people. I think like Scale and Surge were the primary companies that pioneered that industry. And then in this transition to higher skilled labor, what people realized is that actually you can go a lot further with just getting higher caliber people, even in smaller amounts initially, and now subsequently scaling that back up once they're able to meet the quality bar. I think that there's a bunch of companies that after our success and very rapid revenue growth that sort of started early last year have chased after that, which makes sense, right? And seeing that the market was changing very quickly, we were taking off and trying to pursue a similar thesis on the market. [00:32:27] S1: It's interesting. There's always been these companies, AlphaSight and, GLG, that like sort of did this before AI, or is like paid to connect to an expert and ask them questions about stuff. And essentially, okay, it turns out this is really useful for models. We don't need the person in the middle.
00:32:27 · Lenny Rachitsky, Brendan Foody
[00:32:27] S1: It's interesting. There's always been these companies, AlphaSight and, GLG, that like sort of did this before AI, or is like paid to connect to an expert and ask them questions about stuff. And essentially, okay, it turns out this is really useful for models. We don't need the person in the middle. [00:32:42] S0: Exactly. Yeah. Well, the one core difference is that alpha sites would generally be a one off call versus a lot of our work is really hiring people for projects, right? Of how do they work on something for a longer period of time? And so that's, I think, one of the reasons that some of the traditional expert networks have struggled to get into this. And also, how do you retain those people and think about all the incentives where it actually looks more similar in some ways to one of the traditional labor marketplaces of an Uber or DoorDash, just with much higher skilled talent that's treated exceptionally well. [00:33:22] S1: It's such a good opportunity for me to learn so much about this. I'm just going to ask questions. Yeah, sure. So interesting to me. How much of the experts are focused on specific concrete knowledge versus personality and like softer skills? How much of it is like, here's how you do an exam, here's how you do an x-ray?
00:33:22 · Lenny Rachitsky, Brendan Foody
[00:33:22] S1: It's such a good opportunity for me to learn so much about this. I'm just going to ask questions. Yeah, sure. So interesting to me. How much of the experts are focused on specific concrete knowledge versus personality and like softer skills? How much of it is like, here's how you do an exam, here's how you do an x-ray? [00:33:40] S0: It depends on the lab. It's a lot of both. I think that previously it might've been more softer skills, but now a lot of the labs are focused on their business models of what are the economically valuable capabilities that drive revenue and leaning a lot into these professional domains. But I think the creative side is also still really important to everyone. And so we're seeing a meaningful amount of both. Like we hired all the people from the Harvard Lampoon a couple months ago, their comedy club to help with making models funnier. And so do all sorts of stuff like that, hiring Emmy award winning screenwriters and everything across the board on creative capabilities that you'd look for. [00:34:23] S1: That is amazing. What a cool story. Yeah. I'm excited for this to kick in. How fast do these things turn around? Like, say you hired this team, like how fast are we going to see the impact potentially? Is it like months? Is it years? [00:34:34] S0: Well, it depends because some models, or some labs will release iteratively where they'll just improve the model behind the scenes, sort of every [00:34:42] S1: Without announcing a new model.
00:34:42 · Lenny Rachitsky, Brendan Foody
[00:34:42] S1: Without announcing a new model. [00:34:44] S0: Exactly. As sort of every couple of weeks versus others do these big releases. And so it depends a lot. We're behind all of them, but the, I mean, we move really fast. It would be a customer gives us a request of we need these, you know, award winning screenwriters. And within twenty four hours, we'll turn around, the experts. And there's also this really interesting dynamic where in a set of a 100 people that we hire, oftentimes the top 10% of people will drive majority of the model improvement. It's sort of like a company, right? If you have a 100% company, oftentimes the top 10% of the company, will drive majority of the impact. And what that means is that when we're able to build proprietary advantages in identifying who are those top 10 of people, both insofar as how do we have them on our platform, but also identify and match them effectively, it creates so much value for customers that it's difficult to compete against. And so it really does tie back to the founding thesis of the company, which is how do we find these extraordinary people and identify them so that we can reliably deliver these top 10% or top 10x experiences for our customers. [00:35:56] S1: So on that, so is the idea you hire Jane, she's incredible at coding, and she now works for Anthropic, and that's her full time job doing this, or is this like a part time thing? Is this a project thing mostly?
00:35:56 · Lenny Rachitsky, Brendan Foody
[00:35:56] S1: So on that, so is the idea you hire Jane, she's incredible at coding, and she now works for Anthropic, and that's her full time job doing this, or is this like a part time thing? Is this a project thing mostly? [00:36:09] S0: It would sometimes be part time, sometimes it would be full time. I would say most often it's part time, where it's like, you know, someone might work at fan a company where they're underemployed, maybe one of the ones that's moving slower, where they have an extra twenty hours a week, and then they're able to do this on the side, or, you know, whatever the equivalent is sort of across a bunch of different industries. But we also do a lot of, you know, forty hour a week roles as well. [00:36:39] S1: And these, how much are they making? Is it like, like meaningful enough for Fannie engineers to spend time on this?