E79: Now at $2B Valuation: Mercor CEO On How They Started

00:12:59 · Usman HanifPlay from here

[00:12:59] S1: I say, wow. I I mean, I think it's really cool. You mentioned you guys are even taking into account facial expression during these AI interviews. I suppose a human does the same thing. Right? And so, you know, touching on that a little bit more since it sounds pretty interesting, is this a matter of sort of collecting all this data, whether it be facial expression plus what they said and then feeding it to an LLM and letting it reason? Or how are you guys approaching this?

00:13:24 · Brendan Foody, Usman HanifPlay from here

[00:13:24] S2: Yeah. So we started out with just textual data, like just the transcript of the interview. And then we expanded to also do visual stuff. But I will say that we're just starting to dip our toes into a lot of the more complicated visual stuff just because models are only now beginning to have really precise, you know, ability to extract features from these interviews. But I find it to be such like an interesting technical problem, because it's something that humans struggle immensely with. And models are actually superhuman. Like MIT released a paper that they were able to predict the heart rate of an individual and an interview with a reasonably high degree of competence based on the recording of that interview. And I believe that there's all of these subtle nuances and features of the heart rate of the individual, the facial expressions, have the tone, all of these, you know, little nuances that actually go a long way in predicting that person's attitude, their passion, their competence, and all these different characteristics and how they'll perform for a role. And so our focus now is in digging in beyond just what they're saying to all the other characteristics of that individual and how all of these subtle things that are often overlooked can can be priced in reasonably to predict their performance in a given task or job. [00:14:47] S1: And and have you seen any promise with that, like being able to take their facial expression or, you know, their voice inflection, whatever it may be? Is that actually a pretty reasonable predictor?

00:14:58 · Brendan FoodyPlay from here

[00:14:58] S2: So we found that audio is certainly something that's really, really important and has way more signal than people's transcript. However, the visual evaluation is just starting to take we found some signals and visual evaluation. There's interesting things like the blurriness of someone's camera actually contains some signal. Wow. Yeah. And, and there's like little features like that. We're really just dipping our toes into the facial expression stuff. But I believe that as we get more data about this, that there's going to be a lot of signals that emerge. One thing we talked about a lot internally is that like, this fascinating problem of predicting human ability is extremely high dimensional, right? There are so many things that go into what make you good at a given task or job, everything from, you know, all the different tangentially relevant experiences to your motivation to the manager in that job, whether they're like incentivizing you and you want to do a good job. And because the problem is so high dimensional, similar to predicting the next token and a transformer model, it requires an immense amount of data to actually help you meaningfully extract the right features. And so one of the reasons that we're so excited about the business and all of the progress that we're making, is that it allows us to operate at a much larger scale with a lot more data, where we're actually able to make sense of these, you know, small features that we previously weren't able to. And so as the performance data expands, it makes all that possible.

00:16:37 · Usman Hanif, Brendan FoodyPlay from here

[00:16:37] S1: Yeah, absolutely. And just to probe a bit more here, just because I find this technology to be so exciting. You know, when when you're collecting this information, you know, now are you taking into account more, say, multimodal models, vision language models? Is is this where you guys are headed now if you're expanding beyond just the textual space? [00:16:56] S2: Completely. Yeah. So that's why the frontier models that can do good multimodal evaluation are extremely compelling to us. And we've been using vision models for a while and are starting to, you know, explore a lot of what's possible with audio models as well as more sophisticated features with vision models. There also are other kinds of custom models we run experiments with, like the facial expression stuff that we were previously discussing as an example of that and beyond beyond just language models. But there's there's all sorts of models that go into extracting good features. And then it's about, you know, mapping those features to the right performance metrics that we wanna optimize for.

00:16:56 · Brendan Foody, Usman HanifPlay from here

[00:16:56] S2: Completely. Yeah. So that's why the frontier models that can do good multimodal evaluation are extremely compelling to us. And we've been using vision models for a while and are starting to, you know, explore a lot of what's possible with audio models as well as more sophisticated features with vision models. There also are other kinds of custom models we run experiments with, like the facial expression stuff that we were previously discussing as an example of that and beyond beyond just language models. But there's there's all sorts of models that go into extracting good features. And then it's about, you know, mapping those features to the right performance metrics that we wanna optimize for. [00:17:38] S1: Yeah. And and I'm curious in the early days. I mean, you know, the sort of the... You know, it it makes complete sense that once you get a bit more data on one of the candidates, say they've done a couple jobs through the the platform, the reviews are pretty valuable. But when it comes to that first almost, you know, zero shot, one shot approach of sort of classifying this candidate, how did you guys sort of test this? Was it just like, pray that this works and and hopefully the the the company like it? Or how did you go about that?

00:17:38 · Usman Hanif, Brendan FoodyPlay from here

[00:17:38] S1: Yeah. And and I'm curious in the early days. I mean, you know, the sort of the... You know, it it makes complete sense that once you get a bit more data on one of the candidates, say they've done a couple jobs through the the platform, the reviews are pretty valuable. But when it comes to that first almost, you know, zero shot, one shot approach of sort of classifying this candidate, how did you guys sort of test this? Was it just like, pray that this works and and hopefully the the the company like it? Or how did you go about that? [00:18:10] S2: It started with a lot of doing things that don't scale and was, you know, of doing human processes and think how we can replace them, where, you know, my co founders and I have done thousands, not exaggerating ourselves. And we obviously would be able to reach the scale that the models are at now for us at hundreds of 1000s of interviews. But you gain an immense amount of appreciation for like all the different ways to optimize an interview experience, and the importance of automating all of those processes. And so it started with, you know, manually reviewing resumes, and then figuring out how we can automate at least the shortlisting, even if not the final review. And then I remember for a time, we automated conducting the interview, but not yet evaluating the interview because it would actually be way faster. We could just like watch interviews on two x speed and then, you know, and and stop an interview if someone wasn't good. And then and then pushing towards complete automation, which is which is where we're