Unsupervised Learning with Jacob Effron
00:26:50 · Patrick Chase
[00:26:50] S2: The modal roadmap in each board meeting is extensive and, you know, very daunting. I'm always impressed by the amount of software that the team can ship and hardcore software while keeping the system running. How much is... You guys were building the company pre ChatGPT and then post ChatGPT moment. How much has that changed, you know, maybe on the product side, you know, what you guys were doing, but then also when you were thinking about building and the velocity that you can ship software at?
00:27:24 · Erik Bernhardsson
[00:27:24] S1: Yeah. I I mean, we've always benefited from, you know, being quite well positioned at a time where, like, AI kinda blew up, and that's been incredible to see. And we're powering so many different AI applications today that, like, I had, like, no idea would exist, like, know, four years ago or five years ago when we started models. So that's been kind of incredible. And and and like you said, like, software in itself is also... Like, how we build software has changed a lot. People write, like... I I mean, like, I I think probably, like, at this point, like, 95% of the lines of code that gets merged as Modal is probably written by agents to some extent. Infrastructure cell, feel like, relies a lot on, like... Agents are, like, kinda good at it. I've actually been on a tier, like, the last few days, like, building... Or last few months, actually, like, writing, like, an insane amount of code using agents. And, like, it's kind of incredible, like, what they can build, but, like, you kinda still need humans to, like, supervise them quite a lot. And, like, you know, infrastructure has a lot of tail risks around the reliability and taking... And making sure that things scale and, like, stuff like that. So so so so at least for now, I I think software engineers are pretty safe, and and I think we we still benefit from having incredibly good engineers that can supervise these agents, and I think probably for the next few years. Another thing I I think is is being kind of interesting to see is, like, actually, like, I think the adoption pattern is actually the opposite of what I would have expected. Like, I thought it would be the junior people that would adopt agents the most. I've actually seen almost the other way around. It's like the very senior people. Because for them, it's like giving them, like, 10 interns. Like, they've been kinda constrained by, like, the typing speeds for a long time, but they have all these ideas in their heads. And I think using agents, now they can get it, like... They can get it out, like, much faster. They're really good at building a test suite. Like, you know, give the agents, like, here's an extensive text... Test suite. It implements all of, you know, the post six standard for file systems. Now go write a new file system. And it just goes on like just writes a file system and takes like three days to do it.
00:29:08 · Erik Bernhardsson, Patrick Chase
We've seen some of the very senior engineers like go like complete agent maxing and I don't know. I I mean, it's kind of incredible, like, what you could do with agents. You can move very fast with them. At the end of the day, it seems like humans are still safe as software engineers for the next few years. I don't know. We'll see. [00:29:23] S2: How have you found the velocity of shipping software has changed what you guys need to do with code review or CICD or deployment? Have those pieces of your development life cycle changed at all? [00:29:36] S1: I think a little bit. I think we're coming up against some of those constraints. Like, we've had to spend a lot more time on CICD. Like, I always, like, you know, even long before agents, I always, like... You know, I was a CTO for many years. Like, I was always, like, obsessed with CICD as, like, you know, very, very important to to figure out. So, like, in that sense, like, I don't know if it's, like, changed, like, my view as, like, you know, you, like, you really need to have, like, automated deployments, automated tests, like, these things. Now, by way, agents are pretty good at writing tests too. They weren't, like, a year ago. So I I I don't know if it, like, we've necessarily, like, yet, like, run up against the foundational limits. But, I mean, Anthropic, like, you know, a couple days ago, they announced they're moving off Git, I believe, or at least GitHub. We might we might run up against those limits at some point when we have, you know, 10 x the amount of commits every day.
00:30:20 · Jacob Effron
[00:30:20] S0: But it's interesting to think about kind of the the... You know, how different the world is with these really powerful coding agents. And obviously, think, you know, part of of of what I believe your your co founders talked about is is kind of the shift even in who the consumer of modal is over time, right, from a from a developer to, you know, to agents themselves, right? I'm curious, like, how you think about what that means for model as a business and some of the decisions you might make differently as, you know, more and more of of of folks interacting with model are either doing it via an agent or or purely end to end an agent.
00:30:48 · Erik Bernhardsson
[00:30:48] S1: It's actually changed, a lot less than than you... Than I think I would have expected. It turns out agents... I mean, like, in in the early days, like, we we built model in order to give engineers a better developer experience and the ability to move faster. It turns out, like, agents kinda want the same thing. Like, they want fast feedback loop. They want, like, good error messages, good documentation, a lot of the stuff like that that in... Engineers want too. So so so most of the stuff is actually quite similar. I I do think it's, certain things are a little bit harder. Like, one of the things, like, we found, for instance, like, deprecations is much more annoying because agents are trained often on, like, a lot of old code that exists online. So removing features is much harder for us. Like, we can't just rely on, like, you know, remove it from the documentation and wait three months because, like, it's in the training data. People will use, you know, LMs to generate model code, and that will be, like, you know, using deprecated features. And and that's, like, quite hard to change. So so we've had to change our deprecation window quite... For quite a lot. Another thing we... We've started doing quite a lot is also setting up a lot of automated pipelines to to basically have, like, LMs continuously write model code and making sure it it compiles and... Or or not compiles, but it builds and and runs. Trying to just, like, catch, like, things that agents struggle with. And and then maybe another thing we we spend a lot of time on is more like kinda like agent optimization stuff, which is actually more like similar, like, search engine optimization. Like, if you you you can go and ask, you know, ClockCode, like, something like, hey. Like, I wanna do something with inference. I wanna deploy this model and and not say anything about model and just, like, see, like, which tool does it recommend. And and and that obviously is something we we wanted to recommend Bodle as much as possible. So so that's something we track internally to make sure that, you know, over time, like... And I actually think that's, like, something that that will change software engineering a little bit. Maybe I was, like, to the worse. Like, I like...
00:32:39 · Erik Bernhardsson, Jacob Effron, Patrick Chase
I I feel like, in a way, like, because of the training data and and the training loops, like, there's more, like, you know, like, incumbency power in a way. Like, there's more staying power in software today. Like, if you get inside the training data and you get the models to recommend, you know, a a certain framework, I... I've heard people say, like, oh, I'm just gonna use React for everything because agents are really good at it. I'm not gonna try these, like, new fancy front end frameworks. Right? Like, I I think there's, like, something similar with, like, a lot of the frameworks today. It's like, that staying power is actually very, very valuable now. So so we obviously wanna make sure that agents recommend modal as much as possible because that creates, a little bit of flywheel. [00:33:19] S0: And one thing I'm I'm struck by is you obviously had this, like, very prescient bet in 2023 on on sandboxes. Right? It sounds like you guys had kind of this instinct of, like, hey. This is gonna be something that's required for coding agents, and then it took a few years for it to really take off, and then it became this really important primitive. I'm curious, is there anything else that you guys are working on today that you feel like you're a little early on, but you're equally as excited. And and how much of your product portfolio is, like, those kind of bets versus, like, okay. We know this is a hair on fire, like, here and now problem. [00:33:48] S2: Jacob, are you are you fishing for the next thing we should be investing in?