Unsupervised Learning with Jacob Effron
00:20:31 · Diogo, Jacob Effron
[00:20:31] S1: Yes. And, like, that is tragic. You know? Like, it it makes sense. Like, I actually think that as far as I can tell, people have stopped caring about, like, benchmarks for our class of model. Maybe I'm, like, in the wrong niche for this. But my sense... And we... There could be survivorship bias, is that people already don't care about new releases here because they've all been lied to so many times. And this is good for us as a company, but bad for the field because, actually, if someone makes a genuinely good thing, there should be a way to acknowledge it and shout it out and say, like, hey. This is awesome. But, like, in the world where everyone optimizes for benchmarks and demos and things that don't require liability, it's very hard to communicate that other than this very manual process of evaluating models themselves, which is actually super duper hard because Yeah. Benchmarks also don't capture that. [00:21:24] S0: One thing I'm struck by, you know, is obviously this idea of, like, these shapes of problems. Right? And the chatbots are very good for some shape of a problem you're trying to solve. Coding agents, like you said, very good for some shape of a problem. Obviously, folks have found Jev really helpful in in in different shapes of problems. Are there other problems out there that you're still like, god. There's just nothing out there today that's good for [00:21:42] S1: So many.
00:21:42 · Diogo, Jacob Effron
[00:21:42] S1: So many. [00:21:43] S0: What what are like some that come to mind that like... Of of these future areas? Hi. I'm Anne Maris, and I'm the producer of Unsupervised Learning. Thank you so much for listening to the show. I love working on this show, and I have the challenge of hitting 35,000 subscribers by the end of the year. If I don't hit that, I'll be fired, so please subscribe. Thank you. Bye. What what are, like, some that come to mind that, like... Of of these future areas?
00:22:06 · Diogo
[00:22:06] S1: What I realized is that a lot of people are still stuck in, like, fairly basic patterns, and we have, like, a gigantic job to both push the frontiers of model intelligence, reliability, all of these things that we really care about, as well as education to push the limits of software. And it... Like, there's, like, a thing in, like, the intersection of education and pushing the frontier forward, which is like this collaboration between research and product, which is, I think, where all gigantic, like, ML wins come from. And this is, I think, is, like, kind of an unexplored place. Like, if we had more help, we could do so much more here to, like, encourage people in the world or even fund the kinds of things that we think should exist and really allow people to push their front... The frontiers of what software can possibly do. So I'll I'll give a couple of examples of this, and, you know, this is going to be a little bit of a spoiler for a launch... A release we will have sometime in the future. Is that we want to, like, fund these things, and there's so many cool shapes of stuff missing. Like, one of the things people know I'm passionate about is coding agents that are free from the tyranny of the KV cache. I think that so much coding agent is built around the KV cache. And the KV cache is freaking awesome too. We... I don't want to, like, toss it away, but I want to be aware of it and design around it and make different sets of trade offs to make things that were, like, pure sci fi possible. Another thing that I think would be so unbelievably cool is... Do you know what dependent typing is? Dependent typing is a lot about, like, adding constraints to your programs that are, like, higher level than purely the type system. You know? Like, you can make sure that your function returns an int, but dependent typing would allow you to do things like it is an int greater than zero. And then now downstream... Or or downstream applications can use those properties to have their own sets of guarantees, etcetera, etcetera, etcetera, to get higher and higher levels of composability with your software because they're more legible, and they don't require as much, like, unit testing edge cases and also as much human understanding to understand the whole system. And one of the things I think would be so freaking cool would be a dependently...
00:24:13 · Diogo
A semantic dependent typing, where instead of, like, having to represent your outputs as pure types, like int, float, you could actually say specific properties of that in natural language because there's so many properties you could care about in software and then have a type system that uses Jev like models verify that everything works together. I think that that would be so unbelievably cool. Maybe it doesn't matter in a world where everyone uses coding agents, but I think that that... Like like like, we should do stuff like that just because it's us. Awesome. And that's what early Internet energy is about, like, to me. So security is a huge thing as well. Like, as these models get cheaper, faster, all over the place, where can we add extra levels of security kind of to avoid if you guys saw the news about, like, the Cohere, Qwen 3.8 Next Flash Yeah. Debatably makes a sketchy call to, like, phone home. It's it's debatable. I'm not gonna talk about whether or not that's legit or not because I haven't read enough about it. But why can't we just have checks on top of every single tool call that actually are semantic to your organization rather than whatever Anthropic thinks should go into their auto router? You know? Like, maybe in this part of the code base, they shouldn't be making any calls to this other part of the code base at all. And how do you add these checks? You know, you you need, like, extremely fast and cheap security all over the place, which could be super awesome. You know? I'm not saying everyone should build it right now, but I think that we should collaborate with, you know, the making... Pushing capabilities and software in order for, like, thinking
00:25:43 · Jacob Effron
[00:25:43] S0: It's so interesting. Because a lot of those problem... Like, the the problem you're just talking about now, like, you could imagine, you know, approaching from the the perspective of a model company or, like, there's all these, you know, enterprise agent security companies that are popping up, right, that aren't model builders themselves, but they're like, yeah. Let's put in some, like, you know, some actual, like, deterministic software to to to monitor these things. And it's interesting to, like, think about you approaching a similar set of problems, but more from, like, a model perspective versus, obviously, there's plenty of other folks that are just trying to build software that you can put around an LLO.
00:26:11 · Diogo, Jacob Effron
[00:26:11] S1: So my philosophy is that we should enable those software players to win. Just give them tools to build up people. Exactly. But, ideally, they also tell us... We we don't want their data, but they should tell us, like, hey. You have jaggedness on this particular axis that makes our thing less reliable. Let's work together. And, actually, they could just tell us, like, we are not gonna tell you what we work on at all. Here's the rough shape of this task. Fix it. And I would just be like, hell yeah. Thank you so much for telling us this. Hopefully, it solves your problem and a whole bunch of other people's problems. I hope you produce tons of value in the world. Like, I hope you make, like, a decent amount of money, and I hope that this enables other people to build all sorts of other things because now this jaggedness thing is healed. And that is the relationship I hope to have. And this is why I'm, like, so so jazzed about software. [00:27:00] S0: No. I love it. You you said something there that I I wanted to come back to because it was just a great phase. I think you said that the tyranny of the KV cache in in coding agents. Maybe explain for our listeners a little bit about, like, some of the limitations in coding agents, you know, due to the KV cache and, like, how you think about that problem space.