Inspired with Alexa von Tobel
00:09:06 · Cristóbal Valenzuela
So that's another kind of compelling use case of, of our technology these days. And the other one is, is what happens when you start thinking about not only creating content and videos we humans want to watch, because we found those kind of two categories like entertaining. We can also create videos for robots to watch. What I mean by that is if these videos are good enough for us, they might be as well good enough for robots And robots can watch them not for the sake of, finding this entertaining. They can... You can... They can basically use this as a way to learn about the world, learn about how the physical world works. In other words, you're creating datasets or training datasets. And there's ways of also evaluating how effective these datasets are. And so we are kind of focusing on kind of those three main areas these days. It's like this idea of like async or or story... Like narrative storytelling, this idea of real time or interactive kind of experiences, and then the robotics kind of world physical AI aspect as well.
00:10:07 · Alexa von Tobel
[00:10:07] S1: There's a lot that you just said, I'm trying to think about the best way to ask this question. So Runway's first models, like Gen one and Gen two, were some of the first widely used systems for video to video and text to video generation. I wanna ask this very specific question, which is, technically speaking, what had to become true in order to have that step function change that Runway created, which in so many ways, you've now made video creation wildly accessible to a layman, a layperson. What technically had to happen for that to work?
00:10:46 · Cristóbal Valenzuela, Alexa von Tobel
[00:10:46] S0: Yeah. That's a good question. A couple of things had to happen. I mean, there's definitely, like, compute and, and, and like resources had to be in place for training models at like that scale. And again, early emergence of Runways, video models were decently small compared to where things are these days. But you had to have the infrastructure, the technical infrastructure to train those models. The second part was there had to be interesting algorithmic ideas around how you can actually deploy and train the models in the first place. And the second one, data. There needs to be like a good amount of really well captioned and kind of curated and well served data. And a lot of those things have been kind of progressing over the last couple of years, but I think over the maybe four years or five at the most things have converged in terms of compute availability, good algorithms and good data. And that's, I think some of the results is the models that we we... You just mentioned and and hopefully more. And this is not only for video, this is just generally for the industry. So language models and coding models, and all of them have benefited from those three variables kind of becoming better over time. [00:11:54] S1: In the last few years, what has surprised you about the progress of Runway? What has been surprising to you, Chris?
00:11:54 · Alexa von Tobel, Cristóbal Valenzuela
[00:11:54] S1: In the last few years, what has surprised you about the progress of Runway? What has been surprising to you, Chris? [00:12:03] S0: There's a lot of things that are surprising every day. That's the goal of building some of the stuff we're building. You know? So a couple of things, I mean, definitely come to mind that I think have been kind of consistent patterns. I've been, one, underestimating the speed in which things are changing. I think it's a common misconception to look at a model and see all the imperfections and the flaws of the models these days, and tend to think that it will never work because X, Y, or Z doesn't work. And I've heard that for the last ten years consistently, and we keep like moving the goalpost. People thought images will never be able, models will never be able to generate images. And here we are, we've, we haven't solved the images, but we're very decently close to solving the idea of generating pretty much any image that you want. People thought video generation was ten years away and it was not. People thought then you couldn't generate hands with five fingers and all of the things that I think we've keep coming back as to what models are not going to be able to do. We've proven in the industry, I think this is a collective research effort of many other people have proven that models should just keep on getting better. So speed and underestimating the progress is perhaps one of the key lessons and surprises I've learned over time.
00:13:19 · Alexa von Tobel
[00:13:19] S1: On that thread of kind of how people are maybe not thinking about this properly, where are people still using the wrong mental model when they think about this shift in video? And how does that lead to maybe some of the wrong conclusions?
00:13:32 · Cristóbal Valenzuela
[00:13:32] S0: Well, I think it's rooted in this, this idea of how we've, and industry experts have understood how video and role models are used is maybe coming from a place of comfort and a place of understanding of how the world works. And so when people who work in filmmaking look at video models, they tend to think of them in this very like neat, specific way of how every other tool fits within their like workflow and way of doing things. And the reality is that for someone to truly get the best and the most successful outputs of these models, you need to rethink things from the ground up. It's like first principles thinking all over again. And I need to understand that it's less about making things 10, 20% better is maybe you need to just rethink the entire work you're working in the first place. And I think we've made the mistake in, in when understanding new mediums, we've made the mistake many different times. Even with like when cameras were surfed, like around people made the mistake of thinking them as an extension of painting. And, and of course it's not an extension of painting, it's a different medium. You need to understand their world differently to use a camera. And we've gotten used to that, but that wasn't that wasn't the norm back then. And so again, it's... For me, it's a new medium, and so you'd really need to think from... For a different mental model, how to best use the models to to do what you wanna do.