00:24:22 · Adam Miller, Anastasis Germanidis, Sachin
[00:24:22] S0: And is there a big risk there? Because this data would need to be pretty perfect. Right? Because if you're using synthetic data to train other models then that would just degrade quality over time right unless these sort of generations are near perfect? [00:24:38] S2: So they don't need to be perfect as long as the performance inside the world model correlates with real world performance, I think it doesn't need to be that, you get this perfect kind of, recreation of how a robotics model will perform in simulation versus performs in real world as long as two robotics models, if one wins over the other in simulation it also transfers in the real world. But yeah there's definitely a lot of work that we're doing to further improve the diversity of the data we're trying on to make sure that, we improve the realism even further, but it's still quite useful to be able to, to basically create scenarios so quickly and increase and that makes the models a lot more robust as well because now they can be trained on the same the different variations of the setting different variations of the task of the objects and so they're much more robust in different variations in the real world as this is is also an interesting one to show, so this is a generated video of, an accident in an AV setting. [00:25:51] S1: Wow, so that could be used for, self driving cars and?
00:25:51 · Sachin, Anastasis Germanidis, Adam Miller
[00:25:51] S1: Wow, so that could be used for, self driving cars and? [00:25:56] S2: Yeah, so, so self driving and AV research specifically, it's very difficult to collate data of, edge cases of crashes of accidents it will be very expensive to generate to create data sets of accidents but it's very easy to generate those scenarios with a world model and so you can basically create you know, adverse weather conditions, different scenarios of pedestrians moving, moving in the streets, accidents, and that really helps in terms of, like, improving the robustness of the self driving models that essentially control the car. [00:26:39] S1: And when would this technology actually be available and where do you think is the first use case where it's gonna go really mainstream? [00:26:47] S2: So this is already... These kinds of models are actively being used today, to accelerate robotics research. So we're, we're working with a lot of robotics labs to essentially, incorporate GWM one into into their work. So this is already available and already proving to be useful. Of course, there is a lot of room to further improve those models, but GWM one is already kind of essentially in the private beta at the moment and will be generally available in the coming months. [00:27:21] S0: Wow. Sounds exciting.
00:27:21 · Adam Miller, Sachin
[00:27:21] S0: Wow. Sounds exciting. [00:27:23] S1: Something I'm very curious about is how a company from New York with kind of three founders outside of like Central Silicon Valley casting without as much money as, you know, some of the big kind of players in the space has been able to, you artificial analysis publishes these benchmarks and you guys are often at the top. How have you been able to do that in the kind of most simplest way?
00:27:47 · Anastasis Germanidis
[00:27:47] S2: Yeah. I mean, it's it's always sometimes the question I wanna ask the, you know, the big labs and big companies. Why is a smaller company able to too much, you know, to to build competitive models? I think there's a few a few ways to answer that question. I think the one of the most important thing is focus like video models and world models are the core main thing that Runway is focused on. We're not having a whole suite of products and like a lot of different kinds of models that we're working on. So getting this right is very important for us and it's the mission of the company that we succeed at this the second piece is that we've been doing this for a long time we released the first public available text to video model. Like we've learned through many iterations how how to build those models, like all the tricks that go into training them. A big focus for us has been this idea of data efficiency of essentially learning more from the same data of figuring out the water techniques that you can introduce in both pre training and post training to to get better performance with the same amount of compute. We have a team that's about 10x smaller and 100x probably roughly less compute and I think we've been able to be successful in being competitive with those larger players and I think a lot of it comes through with like the strength of the the research team and, like, all this accumulated knowledge and taste that has... For over years of, like, building those models.
00:29:21 · Adam Miller
[00:29:21] S0: So I think publicly, there's a little bit more information out there about how video foundation models are made. They've been made for a few years now, and the in vogue thing, world models, it feels like a little bit of a black box. So I'd love to hear from your perspective about how Runway is approaching world models in terms of research, whether there's potentially new form factors to utilise the technology and the interactions you're having with other industries?
00:29:46 · Anastasis Germanidis, Sachin
[00:29:46] S2: Yeah, so the most important thing to build a great world model for us is to build a great video model first. It's foundation that makes world modeling work is that the base video model needs to understand has a really good performance in physics and, in, that enables it to be, to be effective as a as a world model. World models, the way we build them is essentially as a post training phase on top of the base video model training that we do. As as we find all those applications of our world models in different industries it's very important to work very closely with the companies that have been doing this for a while. For example, kind of collaborations in robotics, of figuring out ways in which we can provide value to those companies in exchange for like the understanding how they operate, what are the biggest problems that they they need to solve. And, a big piece of also building, world models is getting kind of, a real time interaction to work. So, if you if you use if you use a video model, you typically see, you know, you wait for like a minute or two minutes to get back an output. With a world model, it needs to be a real time interaction, and getting to that kind of, low latency is a very difficult both research challenge and engineering challenge that that we [00:31:11] S1: need to solve. What's the most compelling use case you've seen your world models used for so far?