00:15:06 · Anastasis Germanidis, Adam Miller
[00:15:06] S2: The vision is so there's a lot of things that are very motivating internally as we you know it's building those models is difficult as you're saying and it requires a lot of iteration and a lot of false starts and and challenges. I think one thing that is always motivating for the team is like the, you know, the outputs of your work are empowering people to make really really compelling amazing stories And I think that's always, like, a big part of, like, what makes Runway a very motivating place to to work is that you you get to see, like, the fruits of of your work be... Be... Being translated into films, which is quite amazing. I think we're very inspired by this idea of simulating the world, of essentially being able to build models that can deeply understand and help us solve real world problems. And that's a broader application from storytelling, which is our core. We think those models are far more powerful and they can... The the ability to, as we scale those models, are gonna be useful kind of beyond film, beyond kind of storytelling and towards solving real world problems. [00:16:25] S0: On that last point, you are talking a lot about world models these days. Silicon Valley is also talking a lot about that and companies are raising huge seed rounds for them. Could you explain from your perspective what a world model actually is and how Runway are going to be utilising them and just sort of juxtapose them against the sort of previous generations of models that you've built.
00:16:25 · Adam Miller, Anastasis Germanidis
[00:16:25] S0: On that last point, you are talking a lot about world models these days. Silicon Valley is also talking a lot about that and companies are raising huge seed rounds for them. Could you explain from your perspective what a world model actually is and how Runway are going to be utilising them and just sort of juxtapose them against the sort of previous generations of models that you've built. [00:16:46] S2: So, world models are systems that build representations of environment and help to predict what happens as a result of a specific action. So the biggest change from video models, so video model, you type in a prompt, you wait for a bit, you get back a full video. With a world model is interactive and that means you're continuously providing control, so you're continuously moving around in a space or you're continuously interacting with a character or you're continuously, controlling a robot arm so the ability to try different actions and see the effect and of of of the... Of your the... Your actions, that's what makes... That's the main differentiation from, from video models, which are a bit more, like, offline use of just you type in a prompt, you get a packet output, and you're not continuously influencing. [00:17:41] S0: And what would be the practical application of that for Runway? Like, how would somebody eventually engage with, a world model?
00:17:41 · Adam Miller, Anastasis Germanidis
[00:17:41] S0: And what would be the practical application of that for Runway? Like, how would somebody eventually engage with, a world model? [00:17:49] S2: So even if you take the use case of, filmmaking and storytelling and making videos, imagine the experience of essentially being on set right, and being able to continuously kind of direct the camera continuously direct the the actors in in the scene, that's that's something that's a very different experience from the experience of typing a prompt and getting back an output. It's it's something that allows for much greater degree of control. So even if we take the use cases that of, like, video generation, it's a much better experience if you actually can walk on set and just be able to direct the scene. But the the the applications are far broader than that. So if we if we take... We're very excited about how we're seeing our models already being used in physical AI and robotics. So collecting data for robotics is very, very expensive, very time consuming, very, unsafe in a little... In in in in some ways, in some domains. Being able to generate data to train robots is a big application of our models that we're very excited about. Being able to simulate how robots perform without having to test them in the real world is another application that we're seeing really promising results as well.
00:19:18 · Adam Miller
[00:19:18] S0: So I'm sitting here in Chiperdal right next to our office, and Sachin and I just came back from New York. We're on a long documentary tour there, and now we're just heads down building our Stealth startup. And the thing I can't really get out of my mind is this idea of trust. Can customers really trust that what we're building is gonna help them create great content and marketing that helps reach the buyers that they need to reach. That's the thing I think about all the time. And that's one version of trust, but there's so many different types of trust you need to build with customers. Does your product work? Is it gonna help them get value? Is it gonna work in six months? Do they trust your business model? Do they trust your team? There's so many different things. It takes a really long time to build that trust. And that's data security and compliance. Because with a company like Vanta, they can help you automate your compliance in weeks instead of months. We all know those acronyms, SOC two, HIPAA, the complex acronyms that enterprise buyers will throw at you when a deal starts to get serious. Vanta helps you take care of all of that. Using Vanta is really a no brainer solution. Those aren't things that I wanna focus on. I want Vanta to do that for me. Because, you know, the worst story that you'll hear about in startups is when you've found great customers, you've built a great product, you're talking to these big enterprise customers, and you're, you know, you're going over a deal, you're negotiating, you're getting to the final dotting the p's and the i's in the contract, and then they realize that you don't have all the compliance that they need and the deal falls through. No startup founder wants that to happen. We want companies like Vanta that can take care of that for us so then we can just focus on our product and on our customers. So if you're interested in this, head to vanta.com/sachinandadam and you'll get a thousand dollars off.
00:21:04 · Anastasis Germanidis, Sachin, Adam Miller
[00:21:04] S2: Yeah. So this is, essentially, this is an example of, our world model being used as a simulator for robotics. So what you're gonna see is, our robotics model that's basically, rolling out a policy, which is basically performing an action, performing a task inside the imagination of, GWM one, which is our world model. So [00:21:36] S1: what we're looking at is instead of a robot going into this desk room or whatever and and actually just collecting training data, you're collecting training data through simulating it through the model [00:21:46] S2: exactly and you're essentially simulating how a robotics model will perform and you can see you can simulate things like you know like folding clothes which is traditionally very difficult things that to simulate with traditional methods [00:22:01] S1: and what are the inputs of these simulations actually look like? [00:22:05] S2: So the inputs is essentially you're providing you're controlling this robot arm in a similar way that you could control the robot arm in the real world so the input you're providing is the movement of how how that robot arm moves in space. [00:22:21] S0: Yeah. Are we looking at AI generated video or are looking at a robot that has been trained on it? [00:22:27] S2: So this is all generated footage. So our... This is our world model GWM one that's generating two different views of a scene and a robot performing this task. [00:22:40] S0: And with your recent models, you're able to generate a hundred milliseconds per frame, believe. How is that even possible?
00:22:40 · Adam Miller, Anastasis Germanidis, Sachin
[00:22:40] S0: And with your recent models, you're able to generate a hundred milliseconds per frame, believe. How is that even possible? [00:22:48] S2: It's a difficult research and engineering challenge to essentially start from a base video model that's traditionally very slow to getting to distilling it so that it can actually, you can run with inference at real time, frame rate. So, it's it's necessary in order to provide interactive control, and it's something we've been really focused over the past past year of how to distill our base video model to be real time. [00:23:14] S1: If I think of some of the kind of companies we profiled, physical intelligence, we went down to SciQuantum in the South Bay a couple of days ago. I I imagine there's been so many use cases of technology like this for companies like that. In terms of becoming like physical intelligence, how do you think they could use technology like this?