Lenny's Podcast: Product | Career | Growth
00:23:16 · Brendan Foody, Lenny Rachitsky
[00:23:16] S0: Yeah, think that there's a lot of things where the whole value chain of building companies has a lot of these variable costs, even large portions of operations or consulting, right? Like imagine if we could have 10 times as many McKinsey consultants, what would be possible insofar as the research we could do, the analysis, etcetera. But I think the companies and people that are going to succeed are those that lean into this narrative of abundance, of how do we do so much more, rather than fighting back against it of how do we try to stop displacement. [00:23:52] S1: So along those lines, think about your second bucket, which is the people that will be most successful. It's not like a specific skill, but it's being good with AI, using AI to become more, become better at what you're already doing. This reminds me of Elon's whole thing with Neuralink, which I don't know if this is how he put it, the way I've always heard it is he wanted to build Neuralink because in the future when AGI and super intelligence is around, we need a way to compete and the best way to compete is plug our brains into a super intelligence so we have a chance. And it feels like that's what AI is, like getting good at AI tools is essentially, is having this super power. [00:24:30] S0: Figuring out how to leverage them and incorporate it will definitely be of paramount importance. [00:24:35] S1: Yeah, just comes back to this almost cliche quote now. It's, AI won't replace you. People that are really good with AI will replace you.
00:24:35 · Lenny Rachitsky, Brendan Foody
[00:24:35] S1: Yeah, just comes back to this almost cliche quote now. It's, AI won't replace you. People that are really good with AI will replace you. [00:24:41] S0: I think it's totally spot on. And I've definitely seen this at the enterprise level as well, where there are certain enterprises we talk to that are almost like fearful, wanting to engage, not wanting to, you know, eval their businesses because that'll provide the evidence that their value chain is being automated. There's others that, I mean, like, you know, some of the most recognized sophisticated fortune 500 businesses that, that have this mentality. And there's others that are leaning into it of, if we have the ability to do 10 or a 100 times more, what will that mean? And how do we lean into that future? Because there's so many things that are gonna change over the next ten years. And I think those are the kinds of businesses that are gonna be successful. [00:25:26] S1: Let's talk about labor markets more broadly. You you guys so it's interesting, though. You started not feeding people to AI labs, training models. It was just, help people find jobs, help companies hire, and then you're like, oh, wow. This whole opportunity. You have this really interesting view on the future of just labor markets and hiring. Talk about that.
00:25:45 · Brendan Foody
[00:25:45] S0: Yeah, it's interesting. I remember when we started the company, as I mentioned, we were 19 and just had this gut intuition that it felt so wildly inefficient that labor markets are so disaggregated. And what I mean by that is when we would hire someone internationally, they would apply to a dozen jobs. When we, as a company in the Bay Area, were considering candidates, we would consider a fraction of a percent of candidates that were available in the market. And the reason for that is that there is this matching problem that everyone's solving manually, where they'll manually review resumes, they'll manually conduct interviews, and manually decide who to hire. But when we're able to automate that matching problem at the cost of software, it makes way for this global unified labor market that every candidate applies to and every company hires from facilitating a perfect flow of information in the economy. And I think that that future is undoubtedly what we're heading towards. But what we've realized over time is that the nature of work is also changing dramatically. And part of building that future over a ten year time horizon creating that future of work. And all of the more tactical things we do and building these incredible data sets across evals and RL environments for our customers.
00:27:06 · Lenny Rachitsky, Brendan Foody
[00:27:06] S1: What I've seen in how hiring has changed, I'm doing research on this with a partner Gnome, it s so much easier to apply for companies that everyone is just applying out to hundreds of companies. AI is just making it easy to adjust their resumes and cover letters and make it feel like, oh I applied to more courses very specifically, but it was one of 100 places. And then on the flip side, hiring managers are getting flooded with applications. And so now they need AI to filter. So even if we didn't want to get to this place, we're almost being pushed into this direction of so much volume on both sides, and we need something really smart at filtering and helping us hire and select. And this is exactly what you guys have been building for a long time. [00:27:45] S0: Precisely. Yeah. And the fascinating thing, like a lot of people ask, are we, do we think about ourselves as a labor marketplace or do we think about ourselves as a data company? And I think that the reason it's an interesting question is our realization from what the labs need is that they actually need a labor marketplace. They actually need these exceptionally high caliber people. And of course, we'll layer on some project management and some software platform associated with it. But the really core thing that they want is how do they find these extraordinary professionals across all of these different domains that can measure model capabilities and work to build that future work together.
00:28:28 · Lenny Rachitsky
[00:28:28] S1: This episode is brought to you by Interpret. Interpret is a customer intelligence platform used by leading CXN product orgs like Canva, Notion, Perplexity, Strava, Hinge, and Linear to leverage the voice of the customer and build best in class products. Interpret unifies all customer conversations in real time from Gong recordings to Zendesk tickets to Twitter threads and makes it available for your team for analysis and for action. What makes Interpret unique is its ability to build and update a customer specific knowledge graph that provides the most granular and accurate categorization of all customer feedback and connects that customer feedback to critical metrics like revenue and CSAT. If modernizing your voice of customer program to a generational upgrade is a 2025 priority, like customer centric industry leaders like Canva, Notion, Perplexity, and Linear, reach out to the team at interpret dot com slash Lenny. That's e n t e r p r e t dot com slash Lenny. Going back to just how this all works and what you guys do for models, I was talking to a friend who had a ankle sprain or her... His foot was hurting, and he got an X-ray, and he fed the X-ray into Chachi PT and then asked him, like, give me this specific X-ray, and he's like, okay. Sure. And then it gave him here's what you have. And he was talking to me, is out there on the internet to train this model to know this stuff? I was like no, it's actually somebody sitting there helping the model understand this once they recognize it doesn't fully understand this. Like humans are actually helping them learn these things.
00:30:01 · Brendan Foody, Lenny Rachitsky
[00:30:01] S0: Exactly. Well, so the way it works, at least what most people's understanding is there's a lot of complexity in how the models work, that pre training gets a lot of the knowledge into the model of what are all the different things that sort of see it in the world, and then post training and reinforcement learning is for all of the reasoning of what are the pieces of knowledge that are accurate, what are inaccurate, and what to prioritize, at any given time to make a decision. And so behind that, there would have been radiologists that, worked on the post training data set to create some stasis point for here's the diagnosis and rewards and penalties associated with it. And it's really the quality of those people that went into the quality of the decision and recommendation that ChatGPT ultimately made. [00:30:46] S1: So let's actually follow that thread, because that's really interesting, and I don't know how many people understand it. I sort of understand it. So the work that you do and these experts do is post training. It's not feeding data into the model that it's trained on, it's, we have this model GPT-five, now here's all the things it's missing, let's add to it. [00:31:02] S0: Exactly, yeah. It's really unlocking, allowing the model to focus on all the right tokens from pre training all the right things in model context, upgrading the effective reasoning chains, to enable the models to reason better in a more generalized way.