Founders in Arms
00:26:05 · Rajat Suri, Jean-Denis Greze, S3, Immad Akhund
[00:26:05] S1: of AI? You can't do the coding, deep coding anymore. Right? [00:26:09] S2: Yeah. We... I mean, we do an architecture interview and a previous work interview. And then on the coding, we have, like, a two and a half to three hour assignment where, like, in person, where we actually put you in our code base with Claude or whatever agent you wanna use, and we ask you to add something to the product. And what we're testing actually is on how you use AI along the way, and how are you making the right trade offs between not looking at the code and looking at the code. And so usually the feature will have some places where it doesn't matter if you look at the code, but have a couple of places where you actually do need to look at the code to understand how to prompt to to get the right outcome. Yeah. We're just trying to find people who have that, you know, like the right judgment about one, [00:26:51] S3: and other. [00:26:51] S0: This is like on your full code base or some sort of simplified version [00:26:55] S2: of [00:26:55] S0: the [00:26:56] S2: code base? It's like a stuck in time a while back version of that code base. Yeah. We can't do it on a movie code base, so I don't wanna expose my entire code base to candidates. Yeah. So it's it's, like, six months old or something [00:27:06] S1: like the same feature for all candidates? [00:27:09] S2: There's a couple of variants. Couple of variants. [00:27:11] S1: Yeah. Yeah. Pretty similar. Yeah. Right? And and then you'll see how we how we do it.
00:27:16 · Jean-Denis Greze
[00:27:16] S2: I'll tell you, though. I mean, I've always been very skeptical of interviewing. Like, just philosophically for me, it's you're trying to judge someone, you know, a complex human being by spending, like, eight hours with them. It never seems really the the fair way to go about things. I think with AI, I wouldn't say that our current process is anywhere close to foolproof. And I I would say sometimes we're like, oh, the person didn't inspect the code enough, or they wrote too much by hand as opposed to AI. But in their head, they're in an interview. Right? So they're making a set of trade offs about how to spend, like, the two hours with us, and maybe they were thinking, like, oh, show off how good I am at writing code. You know? Because what's the opt... What are they optimizing for? It's less clear. I think, you know, with a whiteboard interview or, like, a coding interview where something has to work, it's like, I'm not saying objectively in a good way, but like you kind of have a binary function. Is it working or not? But I think with AI, I really... We wanna see the thought process of the person, and I don't think we're foolproof there. I'm also a big believer... Sorry. I always get in trouble for saying this one. I'm glad I said this one, and then, like, the concierge would be like, what are you doing? It was like, I'm a big believer in, like, are you hiring? Are you firing? Right? And, like, you can hire well. That's really helpful in life. But also, you can fire quickly, which is, like, a humane thing to do if it's not working. So we've not had to do much of that at Town. But I think because it's not clear what makes someone a great engineer in the current environment, I think it's okay for us to be like, we'll make mistakes in interviewing, and then we should be really quickly about identifying those and and moving forward.
00:28:58 · Immad Akhund, Jean-Denis Greze, Rajat Suri
[00:28:58] S0: Yeah. Especially for you because the other side is trying to get the job. So they're not necessarily, like, beef out of the people over time. Mhmm. [00:29:07] S2: Yeah. And I I think [00:29:08] S1: I struggle with this too. It's like how to identify good engineers at this age. You know, all the previous filters, many of them have gone away. But I think I really like what you said early on, Scott. You want engineers who are easy to man... [00:29:18] S2: You know, high agency, easy to manage. [00:29:20] S1: They'll look like a lie. Talking to talking to customers, good communication. Those seem to be, like, the most important things. And maybe in this idea and age, you also want engineers. I think you said, like, how many customers? That's really important. And and, you know, we'll have a good product sense. Right? Because that's gonna be also very important. But I think my best engineers now, you know, they are very much end to end engineers. Like, they can talk to customers. They're deep in the product, but they can actually make stuff happen as well in in the coach side. What what have you [00:29:49] S2: changed actually in Mercury? We
00:29:49 · Jean-Denis Greze, Immad Akhund, Rajat Suri, S3
[00:29:49] S2: changed actually in Mercury? We [00:29:51] S0: just had a whole engineering week. So some of the people at the back, the people from engineering group, this thing has a crazy echo. Can people hear me okay? Don't know. Okay. So... Yeah. Actually, we're kind of emerging to something similar to you. Like, give them some some, like, project down to... We actually have, like, a somewhat interesting thing we're trying to have where we actually don't tell them the feature to build. We show them, like, customer, like, transcripts of, like, what experience they had with this AI bot type thing and, you know, where it makes some mistakes and things like that. We tell them to go kinda look at that and figure out what to what to to kinda build. [00:30:35] S1: So So normally, you would have a product manager do that, but now you have the engineer kind of, like, interpret the customer. [00:30:39] S0: I mean, I would... I think it's an engineering job to look at, like, you know, the AI bot is reacting in these ways, [00:30:45] S3: like Okay. Okay. So there's a lot [00:30:46] S2: of things. [00:30:46] S0: Yeah. Yeah. It's it's not up to us. Actual conversation. But then we have a bunch... We do have, like, [00:30:54] S2: an architecture type interview as [00:30:55] S0: well where we, [00:30:56] S2: you know, try to get
00:30:56 · Jean-Denis Greze, Immad Akhund, Rajat Suri
[00:30:56] S2: you know, try to get [00:30:57] S0: a sense of... But I do think it's much harder than it used to be. But, also, like, engineers are a lot more productive with the actual. So it's a it's a really interesting world where it's changing all the time. I mean, yeah, we would talk about whether in one year's time, you really [00:31:12] S2: ever need to even read [00:31:14] S0: the code as an engineer. And then, you know, to some extent, we're not testing for quite the right things. But, yeah, you have to get ahead of all [00:31:22] S2: of these things. [00:31:23] S0: Yeah. I think [00:31:24] S1: it's really challenging before I ask the question. It's like we're all struggling.
00:31:24 · Rajat Suri, Jean-Denis Greze, Immad Akhund
[00:31:24] S1: it's really challenging before I ask the question. It's like we're all struggling. [00:31:27] S2: Yeah. One one thing that's been interesting for us is I think the product engineers are looking at the code less and less. But we do have a few people on the team who really work on the architecture and all the... Like, they're just literally the interfaces and all the things that we have so that we don't, like, foot down ourselves. And those folks are pretty deep actually in the code in terms of understanding the continuous, like, code rot that is being introduced by the AI. And the engineers that we have doing that are, like, one of them is one of the best five engineers I've worked with in my career, like a like a senior principal kind of level person, at least. The other two are, like, senior staff or principal. And they, you know, their skill set... I'm actually... I'm, like, curious how you would develop their skill set in in the new world. Because I think they have a skill set around, like, abstractions and and understanding how AI is gonna, like, evolve your code base in bad ways and how you make sure the right instructions and skills are exactly in the right place. [00:32:34] S0: You don't think in two years, AI will be able to mimic the past skill set?