Invest Like the Best with Patrick O'Shaughnessy
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: You've obviously been in and around – and are – great researchers. What are these people like as people? How do they tend to be distinctive from the average person? [00:00:00] S1: I think they're just the same. I have a very hard time thinking of a single set of personality traits. There is no constant basically. There might be a commonality in that to do effective science, you have to be very passionate about that. But even that passion can come from many different places. I've worked with people that were remarkably effective that are just driven purely by the desire for novelty. They don't give a damn about what their technology does. They don't give a damn about whether it's useful. They just want cool new ideas. I've also worked with other people that really want to solve a particular thing. And they're just as happy building stuff as they are testing out experiments, as they are hammering away at things. Whatever it takes. [00:00:00] S0: You mentioned the difference between research and engineering, which also makes me think of manufacturing. Elon would be fond of saying that the factory is the product. The hardest part of this whole equation is actually the scale up of – whatever this thing ends up looking like – making 100 million of those. How do you think about that part of the equation, or is it too remote at this stage?
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: You mentioned the difference between research and engineering, which also makes me think of manufacturing. Elon would be fond of saying that the factory is the product. The hardest part of this whole equation is actually the scale up of – whatever this thing ends up looking like – making 100 million of those. How do you think about that part of the equation, or is it too remote at this stage? [00:00:00] S1: I think it's an important part of the equation. I'm not sure it's the part of the equation that we most need to figure out right now, but it's certainly part of it. A lot of how I prefer to think about this is to figure out the hard part and then enable a lot of experimentation on the other parts. Making a robot at scale is difficult. Making a robot at scale is even more difficult if you don't know what kind of software is going to run on it afterwards, and you're not even sure whether it's the right kind of robot. One of the really valuable things we can get out of general purpose AI tools like robotic foundation models is the ability to get a lot of the other stuff figured out so that at least some of the uncertainty goes away, so that when you scale things up, you have some confidence that this is really going to work.
· Sergey Levine, Patrick O'Shaughnessy
[00:00:00] S1: I think it's an important part of the equation. I'm not sure it's the part of the equation that we most need to figure out right now, but it's certainly part of it. A lot of how I prefer to think about this is to figure out the hard part and then enable a lot of experimentation on the other parts. Making a robot at scale is difficult. Making a robot at scale is even more difficult if you don't know what kind of software is going to run on it afterwards, and you're not even sure whether it's the right kind of robot. One of the really valuable things we can get out of general purpose AI tools like robotic foundation models is the ability to get a lot of the other stuff figured out so that at least some of the uncertainty goes away, so that when you scale things up, you have some confidence that this is really going to work. [00:00:00] S0: A lot of people that listen to this are entrepreneurs, people that run companies. A very popular question has become, how should a traditional company begin to think about using LLMs or preparing itself for the ongoing improvement of these models? How would you answer the same question for robotics? How would you encourage companies to think about this?
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: A lot of people that listen to this are entrepreneurs, people that run companies. A very popular question has become, how should a traditional company begin to think about using LLMs or preparing itself for the ongoing improvement of these models? How would you answer the same question for robotics? How would you encourage companies to think about this? [00:00:00] S1: The technology is changing so rapidly. I want to illustrate why this question is difficult with an example. Here is a particular uncertainty about the tech. Will the robots rely more on demonstrations or on reinforcement learning from autonomous data? We're working on both of those things, and they're clearly both important. But how somebody should prepare for the technology will be pretty different if they're expecting that they need lots of teleoperation to produce lots of demonstrations, a little bit of autonomous experience. Versus the opposite, a tiny number of demonstrations and huge amounts of autonomous experience. Is it 90-10 or 10-90? That's something we're hopefully going to learn about over the next few years, but it does change the correct approach pretty dramatically. That's kind of a case study of how changes in technology will dramatically alter this. [00:00:00] S0: From a business standpoint, is the right way to think about it, get really clear on the economics of the labor in your business? I'm curious how you think about that, the way that this will change the nature of labor itself.
· Patrick O'Shaughnessy, Sergey Levine
[00:00:00] S0: From a business standpoint, is the right way to think about it, get really clear on the economics of the labor in your business? I'm curious how you think about that, the way that this will change the nature of labor itself. [00:00:00] S1: Coding tools are a really nice example to look at for a template of how this might work. It's not like coding tools came on the scene and suddenly we don't need software engineers anymore. It's that the coding tools increased the productivity of individual software engineers. There's some amount of work that needs to be done to make sure that people are able to use them. There's some amount of technology development that needs to be done to make them useful for the appropriate use cases, and these things are still evolving and they're also still changing. Coding agents are different than code completion tools and so on. But I think it's a nice template for us to look at to see how AI tools combine with people doing a job, increase their productivity, and also raise new challenges. And I think we'll actually see something like that with robotics too. A more realistic template is not like, the humanoid goes in and the people just leave. There are some aspects of the job that can be done by a robot, some that can be done with a robot working together with a person. Some that can be where the person needs to do something special to make the robot more productive. Some where it's the other way around, where the robot does something that makes the human more productive. And it'll be this dance that we've seen with coding tools.
· Sergey Levine, Patrick O'Shaughnessy
[00:00:00] S1: Coding tools are a really nice example to look at for a template of how this might work. It's not like coding tools came on the scene and suddenly we don't need software engineers anymore. It's that the coding tools increased the productivity of individual software engineers. There's some amount of work that needs to be done to make sure that people are able to use them. There's some amount of technology development that needs to be done to make them useful for the appropriate use cases, and these things are still evolving and they're also still changing. Coding agents are different than code completion tools and so on. But I think it's a nice template for us to look at to see how AI tools combine with people doing a job, increase their productivity, and also raise new challenges. And I think we'll actually see something like that with robotics too. A more realistic template is not like, the humanoid goes in and the people just leave. There are some aspects of the job that can be done by a robot, some that can be done with a robot working together with a person. Some that can be where the person needs to do something special to make the robot more productive. Some where it's the other way around, where the robot does something that makes the human more productive. And it'll be this dance that we've seen with coding tools. [00:00:00] S0: Do you have a favorite robot that's not part of what Physical Intelligence is doing, and if so, why? Could be anything. Could be a factory robot, could be an Optimus or Boston Dynamics.