Dream Machines: Physical AI, Robotics, Being Human
00:04:58 · Evan Helda, Charles Wong, Aravind Kandiah
[00:04:58] S1: Didn't say groupies. I'm gonna steal that term. [00:05:00] S2: We've been thinking about this since we were in college. Right? And like working on it in various forms. Right? The other thing, you know, like, think to Aravind's point as well, the... We have a unique vantage point where because we work with so many different autonomy teams, we see a lot. Right? And we see the common failure modes, we see the success paths. And that's an interesting place to be as a a SIM tool, as a dev tool across the stack. Right? So just so I think we want people to know that about us. [00:05:34] S1: Yeah. That's awesome. Let's dive into all that. Step into all that. Double Right, [00:05:38] S0: clicking to what Charles said. It's like every customer deploying robots, they just fall into the same traps. Yeah. And it's like, for us, it's just like, guys, you will fall into this. Just don't fall into it. It's like, no. But I wanna try it. And then they're gonna, like, fall into the trap. It's like, you know, if we can, like, hey, guys. These are the the common failure modes. Avoid them the best you can and, like, do these things right before. So it's like just, like, helping them course correct earlier. Because we just see these failure modes again and again and again at all stages. You may be a hyperscaling startup that raised, like, $2,000,000,000. You may be a giant MNC. You may be a small startup. They all make the same mistakes, which is crazy.
00:05:38 · Aravind Kandiah, Evan Helda
[00:05:38] S0: clicking to what Charles said. It's like every customer deploying robots, they just fall into the same traps. Yeah. And it's like, for us, it's just like, guys, you will fall into this. Just don't fall into it. It's like, no. But I wanna try it. And then they're gonna, like, fall into the trap. It's like, you know, if we can, like, hey, guys. These are the the common failure modes. Avoid them the best you can and, like, do these things right before. So it's like just, like, helping them course correct earlier. Because we just see these failure modes again and again and again at all stages. You may be a hyperscaling startup that raised, like, $2,000,000,000. You may be a giant MNC. You may be a small startup. They all make the same mistakes, which is crazy. [00:06:20] S1: Well, let's keep pulling that thread because I think it's a really interesting place to maybe start. What is that most common failure point? Is there one big failure point they're all repeating? Is there a a basket of them based upon certain verticals? What what are you guys seeing currently?
00:06:36 · Aravind Kandiah
[00:06:36] S0: The biggest one is actually how these robotics teams think about testing. Right? So the the very traditional... There's two groups of folks. Right? One group of folks is like, okay. We're gonna just go out and collect as much real world data from some kind of test platform. Right? Maybe it's a humanoid. Maybe it's a a Yam arm. Maybe it's something in a very controlled environment, and then they'll use that as their, like, test set. The other one is they just say, okay. We're just gonna manually roll this out and get a human babysit the system while it's rolling out these policies. And then they will get, like, a certain a certain score from this. Like, you know, maybe it's, a 80% success rate. And then they will then take that 80% success rate, then they'll go to their management and say, like, hey, guys. You know, we think we have achieved, like, a significant level of performance. Let's start rolling this up. But they don't have the discussion, which is, like, setting the expectation of, hey. These are all the scenarios that I need to handle in the actual wild and discussing them with the end users. Right? So let's just take, for example, let's say you are doing industrial manufacturing. Right? You need to be able to pick and place very specific parts and put them on, like, specific places in the factory. There's actually a a finite number of scenarios that you need to be able to handle, but most teams only test, like, maybe 30% of them before actually going out into the wild. Mhmm. Because they just don't have, like, the ability to test the other 70%.
00:08:11 · Evan Helda, Aravind Kandiah
[00:08:11] S1: Mhmm. [00:08:11] S0: But, like, sitting down, setting the expectations with the management folks, and, okay. Let's define the level of autonomy that we wanna roll out and the number of scenarios we wanna handle. And, like, having all of that discussion beforehand, I think it's something a lot of teams skip as well. So as a result, you get this expectation mismanagement where people think that the system can do all these things, but in actuality, it only can do, like, 30%. And it's like, oh, why is it failing in production? You have all these, like, back and forth. [00:08:39] S1: That's really... That's a useful insight. So when you guys talk to these customers, and this kinda kicks off the first question I wanna open with, how is it that you guys are helping them avoid these failure modes? Like, when you go in to talk to these customers, what is your story? Perhaps within this answer, you can kind of give us the thesis of the business as well, we can kind of pack a couple answers into one here. But when you go talk to a customer, these guys probably look at you and they think, what do these guys know that I don't? And they also probably have a bias towards real world data oftentimes and not synthetic data. And they look at you and they say, these guys haven't built robots. What are they? Again, same kind of idea. How do you quickly earn the credibility and trust and then convince them to avoid those pitfalls? And then from there, what is the general story and thesis of Bifrost that you try to land with them in that first interaction?
00:08:39 · Evan Helda, Aravind Kandiah
[00:08:39] S1: That's really... That's a useful insight. So when you guys talk to these customers, and this kinda kicks off the first question I wanna open with, how is it that you guys are helping them avoid these failure modes? Like, when you go in to talk to these customers, what is your story? Perhaps within this answer, you can kind of give us the thesis of the business as well, we can kind of pack a couple answers into one here. But when you go talk to a customer, these guys probably look at you and they think, what do these guys know that I don't? And they also probably have a bias towards real world data oftentimes and not synthetic data. And they look at you and they say, these guys haven't built robots. What are they? Again, same kind of idea. How do you quickly earn the credibility and trust and then convince them to avoid those pitfalls? And then from there, what is the general story and thesis of Bifrost that you try to land with them in that first interaction? [00:09:30] S0: Yeah. Maybe I can answer the first part, then Charles can answer the the second part and the thesis. [00:09:35] S1: Sure. [00:09:36] S0: So the interesting part is when folks actually come to us, a lot of the times, they're just like, hey. We just need more data of x, like a specific scenario. So they aren't even going into the part of, like, hey. Have you guys built robots or or not? And fun fact is, when me and Charles first started the company, we actually built robot. [00:09:56] S1: Perfect.
00:09:57 · Aravind Kandiah
[00:09:57] S0: So we actually built what looked like physical intelligence, like, first product. And it's, like, a whole bunch of, like, stuff that we posted online, but we used, like, a UR arm. And synthetic data was just the way we trained it. We tried to sell it to factories, but all the factories were like, you know, if you want to sell us this whole thing, you have to build an entire production line, and then we realized it wasn't like a venture scalable business at that time. Right? So coming back, when teams comes to us come to us, their first thing is like, okay. You know, we're building this autonomy system. The performance is not good enough. We are failing in a whole bunch of scenarios. We just need data for x y z scenarios. So that's usually, like, the opening point. Like, they'll come to us and they're like, oh, can you just give me more data for x? But then we tell them, okay. First of all, like, let's, like, slow down. We can give you guys the data, the tool to generate all that data, but let's first look at your evaluation stack. Right? Like, how are you actually determining where the models are failing? And this is actually the most critical part where a lot of teams fail. It's because the evaluation set is either a, super biased, or it's just very, very low coverage. For example, let's say you're building an autonomous boat. What most teams do is they go... They bring the autonomous boat to a specific port in the world and then just collect all the data from that specific port. Right? So their test data is only of that port of specific weather conditions.