Invest Like the Best with Patrick O'Shaughnessy
00:39:46 · Eric Vishria
[00:39:46] S2: And they're all over the place and they're on assembly lines and manufacturing lines and all these things where you're doing repetitive tasks in controlled environments. Repetitive task in controlled environments is like more or less solved and like, you know, that'll continue to happen. But having tweaked tasks in real-world environments, that's where you need the AI, that's where you need the AI plus the robots. The trick with it all is, you need a model That can do that. Of course, the first problem is there is no internet-scale data to bootstrap the whole thing, right? Algorithms are all bootstrapped on the internet, which is a ton of human knowledge, and the equivalent for that for robots doesn't exist. And people are trying different things with videos and simulations and teleoperation, so like there's a lot of different ways. But if you kind of think about, take teleop as an example, how much you need to teleop robots? To get to an internet scale data, this first step is like getting a good set of data to bootstrap the model. One of the, I think, insights that you can have is with the internet, there's like a lot of slop data. Even before AI generated all this stuff, there was also a bunch of junk data, and some data was more valuable than others. Like you may value, okay, certain things on Reddit more valuable than other things, you might value Wikipedia more than like other forums, you might value GitHub more than other things, and all of the model Companies did that, right? They prioritized data that was more valuable and less valuable and ran that through the model. I think one of the most interesting things that these AI robotics companies are doing is saying, "Okay, well, let's just go after the high value data to start. If we go after the high value data, then we can kind of bootstrap this model." Now, once you do that, then can you, through that pre-training process, get to a place where you can have Very small auxiliary examples of data that you add in post-training and all of a sudden it works for that. That's the magic we have with LLMs, is you have this giant pre-trained base and then you add a little magic on it in RL and post-training and you teach it a new thing that it wasn't in the pre-training set and you kinda go from there. And I think the exact same thing is happening in robotics.
00:42:02 · Eric Vishria
We're investors in Sunday Robotics, which is going after this exact pipeline company. It's a cool company, and they're doing household robots, but the key part, before you get to household and all that stuff, matters much less actually than can you get this training pipeline to work and how do you do that? And one of the lessons I learned, and I look back, I try to learn from history, 'cause it doesn't repeat but it rhymes, and if I look at autonomous vehicles as an example, which is they're robots basically, they're AI plus robots. You look at Waymo and you look at Tesla. They were both designed vertically integrated in their own way. You have a Tesla, you have a fleet of Teslas that everybody owned, it was collecting data on the Teslas, and it was used to train the models to drive the Teslas. Same thing with Waymo. I think part of the lessons is they gathered very, very high quality data, they did their pre-training, and then they worked off of that. That was a simplification to allow you to get to a complete. Product or complete solution, which of course is gonna continue to improve and ultimately will generalize, I'm sure it'll generalize in some way so you'll be able to strap it onto any car and everything else. So I think the same thing's happening in robotics where you have companies like Sunday and others who are using techniques where they're vertically integrating the robot and the model, the data collection around the robot. Sunday uses gloves That are designed with the robot hand, so they're perfect, so you get very good data transferability from one to another. You do this pre-training, and you have a great pre-training dataset, you have a good model, and then you start adding these examples in your RL and post-training on top, and you get cool emergent behavior. Do you have a most visceral moment when we invested in Sunday the first time? We saw them, partner Peter arranged. A demo, we went down into the basement at the Stanford lab, and they had like this totally janky cardboard glove thing. You see a few evolutions of it. The last time we saw a demo, we went down in the basement of their now office building, and there was like a dozen robots just folding arbitrary laundry.
00:44:11 · Eric Vishria, Patrick O'Shaughnessy
It wasn't in the demo, it was just trial and error, and then they have people like taking the clothes and then measuring them to make sure that they were folded properly and creating a rigorous baseline and evaluation criteria. And I was like, "Oh my god, this is happening. [00:44:27] S0: " Do you ever worry about how to pick the right customer for these companies? Every one of these things folds laundry, which, like, I don't think anyone likes folding laundry, so it seems like a good use case, but it feels like we don't actually understand the demand for what these things will be used to do. You're worried about that they were sort of building solutions that will then be in searches of problems.
00:44:49 · Eric Vishria
[00:44:49] S2: I don't, and I'll explain why, and it's certainly informed by watching the LLM evolution. Some of the people who were involved in the early LLM at OpenAI really understood that code was gonna be important, and obviously, you know, the product team had this perspective that you could get to RSI if you got code generation going and automating AI research and whatnot. If you think about it, the first use cases were very much language oriented, they were very much essay writing and editing and marketing. Like, I think the first application that really took off was Jasper, which was just writing marketing copy. It's gonna evolve a lot basically. I think laundry is kind of a good task because it's arbitrary, it's complex, it requires dexterous manipulation, it isn't time sensitive. If it takes three times longer, so be it, who cares? Doesn't matter, just let it run all day. So I think it has like some of those properties. I don't think the task is actually that important. What I think is much more important is, are you pre-training an amazing model than being able to post-train on top of it and get that flywheel going? If you get that fly Then the task capability will just
00:45:57 · Patrick O'Shaughnessy
[00:45:57] S0: keep multiplying. Maybe this question will be annoying or slightly uncomfortable for you, but if you ask basically every founder and critically other investors of your type, almost everyone, if I ask like "Who's the best board partner? " will say "You. " You come up way more often than anyone else that I've come across. And I'm curious why you think that is. What it is that you're doing that other people, the incentives are there to do a great job as a board partner. What do you think you're doing on the boards of these companies, partner with the founders, that's actually different from other really talented investors who are also nominally doing the same job but don't come up nearly as often when asked that question?
00:46:37 · Eric Vishria
[00:46:37] S2: One of the things that I've realized is we each are attracted to different types of entrepreneurs where we have chemistry. I'm an investor second, and I try to be a partner first. We'll see companies come in, we had one come in yesterday, and it's what I would call an investment grade opportunity. You can invest, it probably works, you make money, it's good, and an investor would do that. A partner wouldn't, because that's not sufficient for a partner. Unless you have real chemistry with that person, where you feel like you're gonna be able to work together really effectively, and I'm gonna learn a ton from them, and they're gonna learn something from me, together we're gonna just feed each other's loops. Unless you feel that way, you can't be a partner. And so you pass on that. That's a really important like fit element to me. It kinda starts with this mutual selection actually, weirdly. They wanna partner with us, and I wanna partner with them, and really like looking forward to working with them together. And I'll give you a really good example of where this comes into play for me. If I take Saajing and Benchling. Benchling Is life sciences SaaS companies absolutely crushed, done really, really well, and then of course, you have this biotech crash and everything else, and the company, it became grindy. The company had never had any churn for the longest time. Such that even on their reports for every SaaS company, you'd have this like, okay, gross ARR added, churn line, net ARR added, like everybody does the same thing. They never had a churn line, never reported it for the first of like six years that I worked with the company. So then they got seven years of churn in like twelve months. Turns out life sucks when you get seven years of churn in twelve months. Through that grind and through it all, and related to the whole, wait a minute, the goalpost moved, we have to do something different, everything else. They kept thinking about how to apply AI for these biotech and pharma customers, which they're very close to. How can we make it better for them? How can we apply these models in their world in a way that they're excited about? And continued to iterate, and it was grindy. And I was there for it.