The Frontier with Brendan Foody
00:38:20 · Andy Fang, Brendan Foody
[00:38:20] S1: The way I would frame it is there's workflows that we can automate much more. And there are things that we can get those people to do that like we didn't have time for them to do previously. Yeah. Yeah. So I think that's more the way we're framing it right now. [00:38:31] S0: Another thing that I've been super interested in is the origins of the WeDash program because we launched something similar called We Task, where we have our team members go in and create tasks on the Mercor platform so that they could build more empathy for the user experience. It was, of course, inspired by you guys. And so what was the early motivation for that? I uh aside from I guess your being one of the first delivery drivers.
00:38:31 · Brendan Foody, Andy Fang
[00:38:31] S0: Another thing that I've been super interested in is the origins of the WeDash program because we launched something similar called We Task, where we have our team members go in and create tasks on the Mercor platform so that they could build more empathy for the user experience. It was, of course, inspired by you guys. And so what was the early motivation for that? I uh aside from I guess your being one of the first delivery drivers. [00:38:56] S1: I mean, that really was the motivation, is like we did the first couple hundred deliveries ourselves. Like it's just like stuff you want to think about, like if you're thinking about it theoretically. It's like, what do I park in downtown Palo Alto? And then it's like if I'm at a restaurant, like the busy salad place sprouts, it's like like the the takeout orders are like at a different place than like if you're ordering in person. It's like as a dasher, you need to like learn that stuff. And then also if I'm a merchant receiving orders, not understanding how hectic it can be at a restaurant, unless you've actually like been there and during peak hour. For us, it's like all that type of stuff. And then with groceries, it's like, okay, how do I go to the right aisle and find the right thing and like how difficult and stressful it can be? No better way to like empathize than to do it yourself. So, you know, I think it's something that started pretty much um from day zero at DoorDash. [00:39:41] S0: Do you still have every employee in the company doing it on what cadence?
00:39:41 · Brendan Foody, Andy Fang
[00:39:41] S0: Do you still have every employee in the company doing it on what cadence? [00:39:44] S1: Every quarter. So every employee every quarter has to do at least one dash. [00:39:48] S0: I've definitely found that it's so easy for teams to just get lost in the numbers and the metrics of like what's happening. Yeah. And get totally disconnected from the actual user or experiences that matter. And it's fascinating how that works. And you really need these empathy building X uh exercises to effectively solve that disconnect. [00:40:07] S1: Yeah. And I think one thing we like to say is like look at the anecdotes, not the averages, you know, because the averages can be misleading. It's like, you know, on average, things are fine. But then it's like, even if you're messing up like 10% of the time, that's like awful. You know, it's like you got to look at where those 10% is happening. [00:40:22] S0: The other thing that um I'm also curious about regarding what you were saying about failure of tasks in accounting environments. Sure. Is any insights that you've had about how it performs in like the real accounting environments versus simulated accounting environments and any of the problems you encountered there?
00:40:22 · Brendan Foody, Andy Fang
[00:40:22] S0: The other thing that um I'm also curious about regarding what you were saying about failure of tasks in accounting environments. Sure. Is any insights that you've had about how it performs in like the real accounting environments versus simulated accounting environments and any of the problems you encountered there? [00:40:42] S1: When we dumb down the tasks, you know, into these like, you know, RL environment type things. It's like, oh, the models do great. But I was like, the accounting team's like, why can't the model do my work? There's a couple hypotheses we have that we're still testing through. It's like, okay, there's probably maybe there's a context issue where it's like, okay, maybe in real life you don't have connections to like the pyramid platform or whatever. But I suspect there's also complexities in the data that are being stripped out when we kind of try to have them go through these like RL environments. [00:41:11] S0: What are the largest things that are getting stripped out and you think might be lost in the simulation? [00:41:16] S1: Well, there's like all the PII stuff, obviously. There's also like the real like accounting numbers. Like we can't like put our real accounting numbers in there. And so like there's the ways you fudge them, but then if you fudge one number, that will affect another number, which will affect another number. It's like not trivial to fudge, you know? Yeah. Like to make it like pseudo-real is like not trivial.
00:41:16 · Andy Fang, Brendan Foody
[00:41:16] S1: Well, there's like all the PII stuff, obviously. There's also like the real like accounting numbers. Like we can't like put our real accounting numbers in there. And so like there's the ways you fudge them, but then if you fudge one number, that will affect another number, which will affect another number. It's like not trivial to fudge, you know? Yeah. Like to make it like pseudo-real is like not trivial. [00:41:36] S0: All of these are very top of mind because we're obvious we're doing a ton of PII scrubbing. Okay. And it's super hard to balance this tension between the realism of the data set like you're describing, but also the legal risk or the uh security risks associated with mitigating all the PII. I don't think any company in the world has cracked this because it's such a challenge. It might not be possible to solve in a lot of cases. [00:42:04] S1: I don't think they are. And so I'm like getting it to like be RL able, is that like gonna be proximal enough to like the real world use case? Or is it just gonna be a fact where it's gonna reach the ceiling and then like to get to the realism, you're just gonna like enterprises are just gonna need to figure out themselves.
00:42:04 · Andy Fang, Brendan Foody
[00:42:04] S1: I don't think they are. And so I'm like getting it to like be RL able, is that like gonna be proximal enough to like the real world use case? Or is it just gonna be a fact where it's gonna reach the ceiling and then like to get to the realism, you're just gonna like enterprises are just gonna need to figure out themselves. [00:42:20] S0: It's fascinating because for the labs, simulating those environments is the primary bottleneck to automating them. And so they'll probably be willing to spend a lot of money and how do they build better and better simulations. I think simulations get all the some combination of contractors and synthetic that can they might be buying enough like dead companies data to like extrapolate. [00:42:40] S1: Yeah, we'll see. [00:42:42] S0: Just got a full company exclusive license to previously $15 billion public company. Oh my gosh. And so there's like some giant, yeah, company. [00:42:50] S1: You guys are just about to scrape the graveyard, huh? [00:42:52] S0: Yeah, so hopefully your your accounting problems will be solved soon. Um we'll see. There's definitely so many really challenging things in how you bridge this real-to-sim gap between what evals and environments the labs are hill climbing versus the real things you see in the messy enterprise. [00:43:07] S1: I I guess what the thing I remembered is like even just like figuring out what the right data to query is. [00:43:12] S0: Yeah.