Y Combinator Startup Podcast
00:10:57 · Max Hodak
These are the people that already speak your language, are already familiar with it, but there's never enough of them to really fill an entire company. You have to hire from the general public. So there's... Different companies hire in different ways. There's different processes that make sense to different founders. This is the thing that really is gonna be matched to who the founders are and how they view the world, and there's no one right answer. But this is a thing where you need a really defined process. There is no right answer, but a wrong answer for sure is not having something that you do very religiously as a company. And this is an area where reality has a surprising amount of detail. It seems really straightforward like, oh, you'll you'll, like, have a job board. You'll get applications. You'll review them. This very quickly becomes a huge, huge drag on the rest of your team. You can easily spend almost all of your time recruiting if you're not doing it efficiently to get to suboptimal outcome. And so for us, again, we've built a lot of software to do this. There are four steps to our process. The first is that we've built a software interface for users to apply... For applicants to apply online where we can capture some structured information from them upfront, including the ability to apply to multiple jobs in parallel. And so we originally used a commercial applicant tracking system. We've moved this to our internal tools, and one of the reasons we did that is because this allowed us to do something that we couldn't find in any the commercial ATSs. So when we... The first step of our process when users apply is it goes to company wide voting. This is a heavily redacted version of of the internal interface, but hopefully, can make out the idea of what's going on here. So the the applicant's resume is in the middle. We collect a little bit of other structured information, but the most important thing is on the the far right, you see this... There's a question. Like, how would you vote for this candidate? Are they known good, strong yes, yes, no, strong no? And so when a person applies, the system picks out seven or eight current employees that it thinks look something like their backgrounds, and it pings them all for votes.
00:13:05 · Max Hodak
And so we can distribute the voting across a lot of the company for this initial review, which is essential because if you're doing anything cool by the time you get a couple years into it, that top of funnel is overwhelming. And if you place any in... Any small group of employees or any one person in the way as a bottle... Bottleneck on this, they will absolutely bottleneck the whole rest of the organization. And it's also you want to, I think, average over judgment. I think there's different people that are better or worse at hiring and have different perspectives on what you're looking for at that stage. So And in the beginning, as the founder, you can meet with everybody. You should... That will take you quite far. You should definitely interview everybody for quite a while. But even beyond that, you still wanna... You want ways to average over the judgment of the rest of your team, and voting mechanisms are usually a really good way to do that. So these are these are our actual statistics over the last couple years. So 17% of the top of funnel applications that we get go to a phone screen. Again, that first initial voting stage is drawn from a company wide pool so that we can get fat... That we can get the voting done quickly within usually twenty four or forty eight hours and not bottleneck that on any small group of people. The phone screen is, again, drawn from a company wide pool of people. This is not team specific. This is a company wide bar really looking for three things judgment, horsepower, and agency. Like, if we throw you into a complex, vaguely defined situation, will you tell... Tend to make good decisions, or will you create diplomatic incidents? Like, do you meet just a basic hurdle for technical competence and demonstrated ability to learn things? And are you effective at causing the world to look like you wish it were? Like, how does your life look or not like whatever ambitions you had? And, like, do you have specific ambitions for your life? And so this, we can distribute over the entire company. And then half of those tend to go to homework.
00:15:03 · Max Hodak
Ideally, we'd be using entirely AI resistant homeworks now, so our favorite types of homeworks are things that don't saturate, have a very high ceiling, and are naturally scorable to two or three numbers that we can put on a plot so that when we get responses to homework, we can plot them all, and then it's very obvious when someone has really beaten the Pareto frontier, and we otherwise don't care whatever AI models they use. Like, that can make you better. In cases where that's not possible for homework right now, we're doing increasing number of technical phone calls or on-site practical tests. But ideally, we would have an AI resistant take home for each of these. Anthropic had a really interesting take on the AI resistant homework where they've had a couple tasks where there's like... It's like kernel optimization, like what is the minimum number of cycles you can get it down to, and this is naturally adjusting. Like, the hurdle for a while was... I think it was Sonnet's performance. If you could beat that, then you could get an interview. I think that there's a bunch of ways to construct AI resistant homeworks. And then by the time you get to the interview, it is important that you have, from there, reasonably high, like at least 25% conversion to an offer because otherwise it is just you're gonna waste too much of your time doing onsites for employees that don't convert. You can't get that down. And so this is... There's four steps to this, initial voting, the phone screen, homework, and a full interview. And this is, as far as... Like, from what... My experience, this is the minimum set of information that we need to make a, like, a full decision, and I don't think that there's a a more efficient way to elicit this. Like, I don't think there's a smaller number of steps that we could use. So this has become our process. So you're hiring people. They're coming into work. They're starting. They're incurring payroll. But how do you know that you're good at this? Like, eventually, you'll get feedback from the market on how good you are at hiring because the company will work or it won't. Like, your team will be capable of accomplishing the stuff that you've set out, and they'll help you course correct through that. But this is a very, very long feedback, and it's very poorly behaved loss function.
00:17:14 · Max Hodak
And so it's kind of your job as management to design synthetic gradients that allow you to find out earlier and along the way how recruiting is going and if need a course correction. A conventional answer to this is the three sixty review process. So once a year, you send out a lot of forms, you gather up a bunch of feedback around each employee, you set up a bunch of meetings with HR and with the various managers, and you can do the conventional performance review cycle, which, based on my experience, is this is a very disruptive process that doesn't tend to surface issues that you don't already know about but haven't acted on because you knew that thing was there, but firing people is hard, and so people drag their feet on it, and this is kind of reinforcing things you already knew. And it only happens once a year, maybe twice a year if you split up the company into cohorts. But I think, really, what would be nice to have is a signal that gives you this kind of natural feedback from across the company about who's good and who isn't and what's working and what's not in a way that is largely unbiased and is more continuous. Imagine if you could get feedback kind of every few weeks on where there are issues and where things are going well. And so the process that I had developed, which I've now used for the last six or seven years, is every few... Like, every couple weeks, every four to six weeks, it's not that often, people around the company get pinged with a question through the software, through Helix. And there's a form, but there's only one question that really matters, which is knowing how this person turned out, would you vote again today for their hire? It's the same questions we use on the initial voting. And so you'll get a prompt to say, like, this person you work with, like, how would you vote for their hire today? And then what we can do is we construct a graph over the company of all of the feedback. And so the basic intuition is that, like, your vote should be weighted more highly if everybody else has rated you highly. And the astute may notice that this looks a lot like the original Google algorithm PageRank, which is an idea called eigenvector centrality where you can create a weight over the graph by looking at how the graph points together. This is a little bit different than actual...
00:19:26 · Max Hodak
Literally, eigenvector centrality, but it's very similar, and so we call this technique eigenreviews. And I've become convinced that this is more or less the right way to do performance reviews. There's some other tricks that you have to apply to get this to work really well. For example, we apply dropout where we'll run a thousand iterations where we'll randomly remove some percentage of the edges each iteration. And then when you look at the distribution of scores that you get out of that, if you see additional peaks, example, this is included. There could be voting clicks that need further investigation. But as a whole, this is... It distributes the judgment across the company, updates more or less continuously with about a month lag, and gives you just way better insight into what's going on really around the company. So this is... And it also totally gets rid of that kind of traumatic, super heavy, once a year HR driven performance review process. So the point of this talk is not that you should use this in particular, although you should consider it. And if you actually roll this out at your company, you can email me, I'll send you a doc with more specific tricks on how to actually get this to work well. But the real theme of the talk is that rate of iteration separates success from failure. And if you can get a fast iteration loop, that really overcomes many other things you're gonna run into. And this effect is so severe. I mean, if you can learn one thing every week and there's a competitor that's learning a thing every month, they will never matter. Overwhelmingly, if there's... You're looking at different way... Like, two different approaches to solve a problem, if there's one that allows you to compound, like, in half... In a much shorter amount of time than the other, even if the other approach has significant redeeming characteristics, you should really consider going with the shorter iteration cycle because the compounding effect is just so dramatic. And so speed determines success and failure, and speed is determined by infrastructure. This is driven by really boring sounding things like how well do your purchasing and recruiting and spending processes work. This is as important as how well do you understand the object level technical content of the thing that you're building.
00:21:37 · Max Hodak
I see companies founded by just, like, stellar pedigree scientists and engineers all the time that die on the vine because this execution is tough to follow through. When your job is to organize, it's like... It's it's uncommon that these deep tech companies fail because the technology doesn't work. They fail because once you end up with this organization of hundreds of people and hundreds of thousands of square feet of physical infrastructure, you haven't built the systems to manage that and becomes unwieldy, and then you can't connect strategy to execution. So we heavily lean towards things that have shorter iteration cycles, kind of etcetera, all else equal, but that's not a blanket rule. Like, there are no blanket rules in startups. You're looking at each new fact pattern that comes in as its own unique thing and then making decisions that make sense to you. And one of the harder lessons as a startup founder, one of the harder things, I think, to to really deal with is the fact that you cannot delegate your judgment. As the CEO, you must always make decisions that make sense to you no matter how much momentum or inertia the alternatives seem to have. So in school, if you're... Let's say there's somebody sitting next to you and you cheat on the test by looking over at them, your grade will... All else equal, your grade will be dragged towards the average of the class. That is not good enough to succeed in startups. You have to do things that are at the long tail that you're the successful companies are the exceptions. By becoming an average, that is not good enough. And so in order to succeed, your judgment has to be differentiatedly good. Now, the reality might be that you don't know if your judgment is good yet. And so one way or another, you will have to find out. And that means making decisions that make sense to you even when you are totally alone in that realization. That is the only way to get to to the really big outcomes. Now, it's not that often that everyone else will think one thing, and you'll be like, you're all totally wrong, but it is a really eerie feeling. Like, you'll get to a point four or five years into the company when there's hundreds of millions of dollars on the line, and there's some really high stakes decision, and only you can make it, and then you will look around for advice.