Unsupervised Learning with Jacob Effron
00:14:09 · Diogo
[00:14:09] S1: That's a very pragmatic take for the short term, and I'm in favor of it. This is, like, where it's about giving engineers power because, like, we shouldn't be overly opinionated about what they use it on. And if they need to do things to make it work, abso fucking lutely, hell yeah. Like, if you need to do, like, the worst dirtiest hacks to provide value in the world, like, you know, like, we are not gonna stop you. Like, that is awesome. And, also, if you have to do too many hacks, that's still our fault. You know? Like like, at a deal where you wouldn't need to do that. And the more it becomes close to, like, semantic understanding, the more beautiful the code is, the more reusable it is, the more you can plug it in into places that you wouldn't expect it to. It's kind of like the Unix philosophy of, like, doing, like, incredibly small useful things, but because people have a mental model of it, they can compose it together into big ones. I feel like this is great. And, also, like, we should not be, like, recreating regexes. We should be recreating, like, beautiful primitives. Not that I hate regexes. I just don't like reading them. You know? And, like like, you know, like, it's it's it's like it's like more... What's the word? Is it write only code type stuff? And, like, we we want to be beautiful, composable. We want to, like, build on layers. Like, LMs have failed to do so, and we want stuff to be more like software. You know? I think I have a... An okay mental model of software engineers at worst. Like, it's kind of weird that people even want to update to the latest and greatest models. I hate updating my dependencies. Like, updating dependencies is, like, the worst thing ever for production software. It's fun for personal use. Like, I used to use Arch Linux. I get it. And, also, like, why would I want to bear the burden of that? And the only reason people keep doing this treadmill of, like, maybe the next model will fix my problems is because I don't think they solve those problems so well that they can run-in the background, fuck off for literally ever, and maybe cut... You know, revisit it every, like, couple of months to see, like, is there a a cheaper or better alternative available, but it works fine right now. I don't I don't know of anything like that. And that is, that's a sad state of the world.
00:16:03 · Diogo, Jacob Effron
Like, it's very counter to how software engineering tends to work. [00:16:07] S0: I guess in your vision of the world, like, Dell, have a place in the world in, like, ten years?
00:16:07 · Jacob Effron, Diogo
[00:16:07] S0: I guess in your vision of the world, like, Dell, have a place in the world in, like, ten years? [00:16:11] S1: Super duper. Absolutely. I I I hope if that they do not have a place, then something strictly better exists. I... I'm not opposed to LMs. I'm not actually opposed to either chatbots or coding agents, completely different things. And I I really don't want to go to a world where they don't exist. The thing is I think people overuse them for a lot of tasks, and I actually hope the market for that increases too as people get better at it. And I'm I'm I'm, like, basically certain that there's tons of use cases for it at basically every single intelligence cutoff that has existed, maybe since GPT four. Like, all of these models had beautiful, awesome use cases that maybe would get Pareto dominated by another model, but, like, they should be able to run-in production. And it takes a lot of nuance and understanding, and it... It's like it's like really not built for engineers. So I love LMs. I hope that I I can continue to, like, chatbot anything that I have, like, I need, like, x... Pseudo expert level answers to that don't need to be correct. That is the use case in my mind for chatbots. And I love being able to, like, spin up, like, throw away software instantly when I want custom dashboards and stuff, because I think it's the best use case for coding agents. And that should stay for the rest of time. I don't think people should be doing that. And we should keep on building our toolbox, so to speak. Like, it's... I think it's just awesome.
00:17:31 · Jacob Effron
[00:17:31] S0: Is your intuition that the toolbox will include, like, many other shapes of models in the future with, you know, other RL acronyms? Or if we're talking ten, fifteen years from now, like, are there, you know, are there a bunch of different, like, tools in this toolbox, or is it kind of like your approach and the LLM approach cover most of the use cases?
00:17:48 · Diogo
[00:17:48] S1: I'm gonna end up saying something spicy here, but I would say when I first... When we first launched this, I was really excited for novel shapes to emerge. Really, really excited, and I still think that that will be the case. I think new shapes are freaking awesome. We plan to keep shipping them. I'm a little bit worried that, again, what we talked about... Maybe this was, like, before we got on the record, that ML people especially have, like, kind of, like, lost the actual message I was trying to say. And I'm worried that people are now overfocused on, like, the shapes of tasks rather than the reliability. The shape of task is incredibly... You know, like, it's super easy to do that. When I thought this would take a week, I thought it would take a week to make the shape of the task, which it did. And this is what's happening now with other people making similar models with similar shapes. But the shape really doesn't matter compared to actually it being useful, and that is the thing I think people really, really need to focus on. So there will a 100% be way more shapes than these two. Even if no one else does it, we will do them. I think people are making too many new shapes right now and not caring enough about you... Like, the utility of it. And I'm worried that this will, like, actually burn the goodwill that people... People don't like being, overpromised, underdelivered to. And, like, these things are, like, pure overpromised, underdelivered. And I think that that's, like, not good for the world. You know? Like, if people have, like, creative, playful energy, we don't want them to waste it on stuff that clearly doesn't work. As far as I can tell, people dislike that. And even if they disliked it, I hate it. Like, I... Oh, man. And I think there's going to be a balance. There will definitely be a lot more future shapes to be had. And I just really, really hope that people start giving, like, unbelievable levels of fucks for utility, usefulness, reliability. What would a developer want in this situation rather than ML people who I think focus on, like, I want to flag plant. You know? Like, I want to be the one who names the thing that gets cited. I want to, like, get revenue in the short term because people will play around with it, and I want to do the same thing that every other AI company does.
00:19:57 · Diogo, Jacob Effron
I really hope we don't do this, but overpromise, underdeliver in order to get attention because it feels like a lot of the AI economy ish is really driven by that attention. And that is so, so bad for developers. It's, like, actually, the absolute worst for the developers who do not have, like, the time to test these things themselves because they just get... They're the ones who get swept up and all. This is this is also why I'm anti benchmarks. It's a it's a... It's it's so bad for those people. You know? [00:20:26] S0: You have to play around with some of the stuff yourself to have any sense of of of of how good it actually is. [00:20:31] S1: Yes. And, like, that is tragic. You know? Like, it it makes sense. Like, I actually think that as far as I can tell, people have stopped caring about, like, benchmarks for our class of model. Maybe I'm, like, in the wrong niche for this. But my sense... And we... There could be survivorship bias, is that people already don't care about new releases here because they've all been lied to so many times. And this is good for us as a company, but bad for the field because, actually, if someone makes a genuinely good thing, there should be a way to acknowledge it and shout it out and say, like, hey. This is awesome. But, like, in the world where everyone optimizes for benchmarks and demos and things that don't require liability, it's very hard to communicate that other than this very manual process of evaluating models themselves, which is actually super duper hard because Yeah. Benchmarks also don't capture that.