00:23:14 · Sachin, Anastasis Germanidis, Adam Miller
[00:23:14] S1: If I think of some of the kind of companies we profiled, physical intelligence, we went down to SciQuantum in the South Bay a couple of days ago. I I imagine there's been so many use cases of technology like this for companies like that. In terms of becoming like physical intelligence, how do you think they could use technology like this? [00:23:32] S2: Yeah. So a company like physical intelligence and other robotics labs and, you know, we're we're in collaboration and conversations with a lot of them. They spend a lot of time running evals, basically understanding how the robotics models that they've trained can perform, in in the real world, and it's very difficult in order to differentiate between, you know, you've trained a few models, you wanna see which models are good at what tasks. That's extremely time consuming to do. Very often it can break the robots, like it's... And so being able to minimize the number of real world evals that you have to do, by doing the evals in simulation and scaling them, you know, as much as you want is is very, very useful and kind of significantly accelerate kind of the the research, kind of progress of them. [00:24:22] S0: And is there a big risk there? Because this data would need to be pretty perfect. Right? Because if you're using synthetic data to train other models then that would just degrade quality over time right unless these sort of generations are near perfect?
00:24:22 · Adam Miller, Anastasis Germanidis, Sachin
[00:24:22] S0: And is there a big risk there? Because this data would need to be pretty perfect. Right? Because if you're using synthetic data to train other models then that would just degrade quality over time right unless these sort of generations are near perfect? [00:24:38] S2: So they don't need to be perfect as long as the performance inside the world model correlates with real world performance, I think it doesn't need to be that, you get this perfect kind of, recreation of how a robotics model will perform in simulation versus performs in real world as long as two robotics models, if one wins over the other in simulation it also transfers in the real world. But yeah there's definitely a lot of work that we're doing to further improve the diversity of the data we're trying on to make sure that, we improve the realism even further, but it's still quite useful to be able to, to basically create scenarios so quickly and increase and that makes the models a lot more robust as well because now they can be trained on the same the different variations of the setting different variations of the task of the objects and so they're much more robust in different variations in the real world as this is is also an interesting one to show, so this is a generated video of, an accident in an AV setting. [00:25:51] S1: Wow, so that could be used for, self driving cars and?
00:25:51 · Sachin, Anastasis Germanidis, Adam Miller
[00:25:51] S1: Wow, so that could be used for, self driving cars and? [00:25:56] S2: Yeah, so, so self driving and AV research specifically, it's very difficult to collate data of, edge cases of crashes of accidents it will be very expensive to generate to create data sets of accidents but it's very easy to generate those scenarios with a world model and so you can basically create you know, adverse weather conditions, different scenarios of pedestrians moving, moving in the streets, accidents, and that really helps in terms of, like, improving the robustness of the self driving models that essentially control the car. [00:26:39] S1: And when would this technology actually be available and where do you think is the first use case where it's gonna go really mainstream? [00:26:47] S2: So this is already... These kinds of models are actively being used today, to accelerate robotics research. So we're, we're working with a lot of robotics labs to essentially, incorporate GWM one into into their work. So this is already available and already proving to be useful. Of course, there is a lot of room to further improve those models, but GWM one is already kind of essentially in the private beta at the moment and will be generally available in the coming months. [00:27:21] S0: Wow. Sounds exciting.
00:27:21 · Adam Miller, Sachin
[00:27:21] S0: Wow. Sounds exciting. [00:27:23] S1: Something I'm very curious about is how a company from New York with kind of three founders outside of like Central Silicon Valley casting without as much money as, you know, some of the big kind of players in the space has been able to, you artificial analysis publishes these benchmarks and you guys are often at the top. How have you been able to do that in the kind of most simplest way?
00:27:47 · Anastasis Germanidis
[00:27:47] S2: Yeah. I mean, it's it's always sometimes the question I wanna ask the, you know, the big labs and big companies. Why is a smaller company able to too much, you know, to to build competitive models? I think there's a few a few ways to answer that question. I think the one of the most important thing is focus like video models and world models are the core main thing that Runway is focused on. We're not having a whole suite of products and like a lot of different kinds of models that we're working on. So getting this right is very important for us and it's the mission of the company that we succeed at this the second piece is that we've been doing this for a long time we released the first public available text to video model. Like we've learned through many iterations how how to build those models, like all the tricks that go into training them. A big focus for us has been this idea of data efficiency of essentially learning more from the same data of figuring out the water techniques that you can introduce in both pre training and post training to to get better performance with the same amount of compute. We have a team that's about 10x smaller and 100x probably roughly less compute and I think we've been able to be successful in being competitive with those larger players and I think a lot of it comes through with like the strength of the the research team and, like, all this accumulated knowledge and taste that has... For over years of, like, building those models.
00:29:21 · Adam Miller
[00:29:21] S0: So I think publicly, there's a little bit more information out there about how video foundation models are made. They've been made for a few years now, and the in vogue thing, world models, it feels like a little bit of a black box. So I'd love to hear from your perspective about how Runway is approaching world models in terms of research, whether there's potentially new form factors to utilise the technology and the interactions you're having with other industries?