Inspired with Alexa von Tobel
00:06:46 · Cristóbal Valenzuela
[00:06:46] S0: Sure. So you can think about basically three main use cases and I'll walk you through each of those in a minute, but the most, the first one is with this idea of using AI or video and world models to create entertainment or create media. So you're, if you're, if you're ever watching a movie or if you're going to stream something in your house, you are watching the content or the scenes or the videos someone either recorded or produced with traditional kind of VFX, CGI techniques and tools. Today, where AI is, we are at a place in time where a model can simulate or generate almost all of those scenes. And so if you're in the business of storytelling, the business of filmmaking, in the business of making medium, because this also helps and works really well for advertisers. And basically for any other honest, honestly content creator out there, you can use these models to, to do that, to do that at a fraction of the time that it would have taken you to do it traditionally. And so the more concrete use cases, well, this is, this is a new kind of camera. This is a tool instead of AI models that can help you tell stories visually in ways that you couldn't do before, for high end professional cinematic worlds and for casual creators. And so that's a little bit of the user base of Runway. We work with the best and greatest Hollywood studios in LA and the world. And we also work with just consumers and creators who are using Runway to have fun. So that's kind of one concrete use case that's actually getting deployed a ton these days. It's just as a means to create video and content. A second kind of like possibility is what happens when you start running these models in real time? So again, you can create movie like content or really compelling stories, really compelling scenes. What happens the moment you start running them in real time? Real time, this is real time, right? We're speaking to each other. You're responding to my questions. You... I'm asking your questions as well. You can do a version of this with AI models, you say, visually, which is kind of surprising and kind of magical to be honest. And so there's a lot of interactions and things you can do. You can, have personalized tutors, you can teach people about new languages, you can just have a game live experience where you can chat with characters you couldn't do before.
00:09:06 · Cristóbal Valenzuela
So that's another kind of compelling use case of, of our technology these days. And the other one is, is what happens when you start thinking about not only creating content and videos we humans want to watch, because we found those kind of two categories like entertaining. We can also create videos for robots to watch. What I mean by that is if these videos are good enough for us, they might be as well good enough for robots And robots can watch them not for the sake of, finding this entertaining. They can... You can... They can basically use this as a way to learn about the world, learn about how the physical world works. In other words, you're creating datasets or training datasets. And there's ways of also evaluating how effective these datasets are. And so we are kind of focusing on kind of those three main areas these days. It's like this idea of like async or or story... Like narrative storytelling, this idea of real time or interactive kind of experiences, and then the robotics kind of world physical AI aspect as well.
00:10:07 · Alexa von Tobel
[00:10:07] S1: There's a lot that you just said, I'm trying to think about the best way to ask this question. So Runway's first models, like Gen one and Gen two, were some of the first widely used systems for video to video and text to video generation. I wanna ask this very specific question, which is, technically speaking, what had to become true in order to have that step function change that Runway created, which in so many ways, you've now made video creation wildly accessible to a layman, a layperson. What technically had to happen for that to work?
00:10:46 · Cristóbal Valenzuela, Alexa von Tobel
[00:10:46] S0: Yeah. That's a good question. A couple of things had to happen. I mean, there's definitely, like, compute and, and, and like resources had to be in place for training models at like that scale. And again, early emergence of Runways, video models were decently small compared to where things are these days. But you had to have the infrastructure, the technical infrastructure to train those models. The second part was there had to be interesting algorithmic ideas around how you can actually deploy and train the models in the first place. And the second one, data. There needs to be like a good amount of really well captioned and kind of curated and well served data. And a lot of those things have been kind of progressing over the last couple of years, but I think over the maybe four years or five at the most things have converged in terms of compute availability, good algorithms and good data. And that's, I think some of the results is the models that we we... You just mentioned and and hopefully more. And this is not only for video, this is just generally for the industry. So language models and coding models, and all of them have benefited from those three variables kind of becoming better over time. [00:11:54] S1: In the last few years, what has surprised you about the progress of Runway? What has been surprising to you, Chris?
00:11:54 · Alexa von Tobel, Cristóbal Valenzuela
[00:11:54] S1: In the last few years, what has surprised you about the progress of Runway? What has been surprising to you, Chris? [00:12:03] S0: There's a lot of things that are surprising every day. That's the goal of building some of the stuff we're building. You know? So a couple of things, I mean, definitely come to mind that I think have been kind of consistent patterns. I've been, one, underestimating the speed in which things are changing. I think it's a common misconception to look at a model and see all the imperfections and the flaws of the models these days, and tend to think that it will never work because X, Y, or Z doesn't work. And I've heard that for the last ten years consistently, and we keep like moving the goalpost. People thought images will never be able, models will never be able to generate images. And here we are, we've, we haven't solved the images, but we're very decently close to solving the idea of generating pretty much any image that you want. People thought video generation was ten years away and it was not. People thought then you couldn't generate hands with five fingers and all of the things that I think we've keep coming back as to what models are not going to be able to do. We've proven in the industry, I think this is a collective research effort of many other people have proven that models should just keep on getting better. So speed and underestimating the progress is perhaps one of the key lessons and surprises I've learned over time.
00:13:19 · Alexa von Tobel
[00:13:19] S1: On that thread of kind of how people are maybe not thinking about this properly, where are people still using the wrong mental model when they think about this shift in video? And how does that lead to maybe some of the wrong conclusions?