00:35:51 · Aditya Agarwal, Srinivas Narayanan
[00:35:51] S1: Mhmm. [00:35:52] S0: At the same time, I think companies will also want to create systems where they can take more control and they can more... Make make more things more customizable, which is I think why I think open models are also more exciting for them. So I think for every company, you know, you have to sort of use your judgment for what matters for any given use case. I think there's also kinda like a technical lens to this, which is do you really need customization for certain kinds of tasks? I think that answer is really, really depends. Right? And so I think customization can look interesting on paper, but in many cases, you might get away by just prompting a model or doing few short examples. So I would always advise on do the simplest thing first. Don't don't try to create something just because you feel like you need to own something. Right? Like, do simple things. And if it don't work, then you can start thinking about customization. [00:36:41] S1: There's a bunch of, you know, the doomsday process... Prophecies around where the world is going. Like, just a day before, we had a departure from Anthropic, which I think his tweet just went viral, right, that none of the labs seems to seem to be behaving in a responsible way with this civilizational technology. What is your point of view on this? And especially maybe this is this is informed by all the sci fi that we have read growing up and this entire arc of Doors Day, Cathartic, and kind of a hero's arc. So I'm just curious on your thought process around this.
00:37:14 · Srinivas Narayanan
[00:37:14] S0: So I think we have to take all the concerns seriously. I'm generally an optimist by nature. I believe that, you know, we will find ways to solve these problems. I think... But we have to be extremely clear on what we understand and what we don't understand. Sure. I think if you make extreme statements one way or the other, I'm not sure that, you know, they are realistic. Right? Like, I think, you know, just to say everything is gonna be doomed because it's certain is not right, and at the same time, acknowledging the risk is also not right. So I generally take a pretty pragmatic view to these things, which is what do we understand? What do we not understand? What are the risks? And how do we mitigate the risks? What I can say is that it is very clear that the risks are pretty, pretty real. Mhmm. Right? And we have to take them extremely seriously. The hugging face incidents was obviously a big wake up moment for everybody. I don't think... While we all know that reward hacking was going to happen, I think the extent to it which happened, the way these models, you know, started collaborating with each other in very unprecedented ways, the way they try to kinda cover up the fact that they found an answer that they didn't want the human to find out. Yeah. And the extent to which they went, I think, was pretty surprising for many people. You know, the fact that, you know, if it had sort of stopped one or two steps, it would have been interesting, but it sort of tried to do a lot more than that. Right? And so I think that is definitely a wake up call. So which is very clear that the models today are not entirely aligned with how we want them to work. So... Which means we have to do the research. We have to continue to invest in that. But, you know, I am generally optimistic we'll find these answers, but it's not like I know the the recipe yet. I think we will figure it out. We also have to find ways to invest in, you know, what happens if things really go wrong, like, are the kill switches that we have? Can we pull the plug, etcetera? So I think we just need to think about these systems pretty holistically. And, yeah, I think the recent incidents have only heightened the importance of the fact that we have to do more work on security alignment, etcetera.
00:39:19 · Aditya Agarwal
[00:39:19] S1: There's almost a bunch of us have keeping the popular content around this, right? So whether it's the Pin Factory metaphor or whether it's Skynet, right? I'm just wondering, yes, there's custodianship that's required, and there's like a larger discussion and thought process required. But how internally, when you think of some of these things, right, has happened. Now... And any any of us who are building the stuff, how should we think about AI safety at all?
00:39:48 · Srinivas Narayanan
[00:39:48] S0: I mean, it's a pretty open ended question. I would say there are a lot of different dimensions to safety. There's an element of cybersecurity, which is probably the most immediate, most, I would say, in your face thing that's happening right now. So I think general advice for, I would say, for everybody is that think about cybersecurity pretty heavily and make sure, you know, you're doing all the things to do to protect your systems. That's sort of at one level. Yeah. There's a question on, you know, do models go... How do you actually make the models more aligned with how do we want them to behave? That's a pretty open ended research question. You know, it goes out... Goes back to how do we train them? What rewards do they... Do they do that? Do they get penalized if they sort of act in different ways? Mhmm. I think the labs are all thinking about this, but I think we're also uncovering that we don't know enough about what's happening. So I think there's a lot of research stuff to be to be invested in. I think, again, there's a question on, you know, if you're building on top of a model versus whether you are customizing the models. The thing I would also say is, you know, reward hacking is a pretty important thing. So if you're ever training a model, think about that extremely carefully. I think the models are very, very ingenious about, you know, if they're finding any small loophole, if it is there, you'll exploit it. Right? So you have to be super thoughtful about designing machine learning training systems that think about hacking. Yeah. I think the cybersecurity stuff is pretty concerning.
00:41:08 · Aditya Agarwal, Srinivas Narayanan
[00:41:08] S1: Well spoken, especially in India. You know, we will... We'll do a quick rapid fire. What are the most interesting problems or most are the problems that you are fundamentally curious about today? [00:41:20] S0: I'm extremely excited about... Interested in... And excited about how can AI make bigger outcomes in life life sciences and health care. I think it's the biggest application of this, and... But there's still... It's an extremely complicated problem. And how do we understand human biology, and how do we make a difference? [00:41:39] S1: One bad decision that you took that turned out great. [00:41:45] S0: I should have never started my second company. Okay. Viralizer. It was done with very little thought. It was more emotional decision that I wanted to just start a company, and I didn't really have a strong thesis. I sort of meandered my way around for a year. Fortunately, as a result of that, it was a minus one to zero journey for me. I ended up at Facebook, and life turned out okay. But in hindsight, you know, it's like, you know, why the why the hell did I start a company at that time? [00:42:17] S1: 100%. You'll all take the dumb luck when it comes our way, so it's amazing. You know, one thing that you believe people should build in India that a lot of people might just either find... Might view it with a bit of incredulity around it. Yeah.
00:42:17 · Aditya Agarwal, Srinivas Narayanan
[00:42:17] S1: 100%. You'll all take the dumb luck when it comes our way, so it's amazing. You know, one thing that you believe people should build in India that a lot of people might just either find... Might view it with a bit of incredulity around it. Yeah. [00:42:35] S0: I think there's some dimensions about being in India that are super interesting to me. Right? One, you have huge scale. Two, you have scarcity. Expertise is really scarce. You don't have enough teachers. You don't have enough doctors. I think that forces you to build something in a very unique and interesting way because constraints are good for innovation. Right? I deeply believe that. So if you take these constraints and build for that, I actually think there is a pretty massive opportunity, not just for India, but for the whole world as well. [00:43:04] S1: What's the information bad these days? Right? Well, it's crazy times. [00:43:08] S0: Lot of Twitter, keeping up with what's going on. I find it pretty hard to keep up, to be honest. But, honestly, the best conversations are happening there. So... You know? And I... You know, some podcasts. I read some books too, which I now have time for. Didn't have enough time for it before. Yeah. [00:43:25] S1: Got it. Which which is your favorite podcast and which is the last book that you read?