Unsupervised Learning with Jacob Effron
00:07:00 · Jacob Effron, Diogo
[00:07:00] S0: I mean, AI world, don't know if I can ask, I don't know if I can ask longer than that. Right? [00:07:04] S1: I wouldn't say jaggedness is purely, like, a... Our road map. There are some things that models truly are weak at. And, like, this is where, like, that taste stuff comes in. Right? Like, what I was talking about, like, the reality of what you can extract from these models. The reason we call them system one models and among other things... And system one, actually, people did not want us to name that it either because system one thinking tends to be a bad thing. Like, people use that as kind of, like, an insult. Right? [00:07:29] S0: Yeah. It's like the the part that just naturally happened from evolution, you know, from back in the day before Exactly. Instinctual yelp, which makes mistakes, etcetera. Brains. Yeah.
00:07:29 · Jacob Effron, Diogo
[00:07:29] S0: Yeah. It's like the the part that just naturally happened from evolution, you know, from back in the day before Exactly. Instinctual yelp, which makes mistakes, etcetera. Brains. Yeah. [00:07:37] S1: Exactly. But... And, like, people often, like, say that that is the mistake making side. And I believe that the shape that actually AI gets compressed to happens to be, like, very system one, and therefore, the honest thing to do is to communicate its weaknesses. Because I think that on its own can be incredibly, incredibly valuable for software, and most of what people need actually is system one stuff. You know? Like, why can we solve Millennium Prize problems in math but not automate anything touching accounting? Like, that is... That should be really weird to a lot of people. And, like, I think that that is, like, still our north star even if people get distracted by, like, cost, speed. You know? Like, oh, it's it's it's good for, you know, like, really fast decisions or whatever else. Like like, I see these as, like, distractions compared to what is really needed for AI to meet its potential, and that is my my north star. [00:08:29] S0: And so, obviously, the... You put the models out there, and you have a lot of people using them in different ways. How much does, like, the the real world usage of the models come and and feed back into the road map versus just, like, the stuff you were... You're all planning to do?
00:08:39 · Diogo
[00:08:39] S1: I would say it's almost none real world feedback. I think people should be complaining to us so much more. I think that, like, this... The field is still in a Stockholm syndrome of, like, oh, yeah. The model providers just do stuff in their one track path, and we happen to be, like, the victim is the wrong word. But, like, you know, like, in, what's that movie? Wall e. You know, like, they're, like... They're just like the people in the chairs that are just getting fed the AI. And I think this is, like, the state of things right now where, like, people plan around AI changing, but they don't have a sense of agency of where to take the field. And I think that this is not the way it should go. That's actually really disempowering and kind of lame in my opinion. And, also, I think that there's so much more we could do collaboratively as a field. And I think that the field could, like, push us a lot more to say, like, hey. We want to be doing these things. We think it is within the realm of capabilities. Let's figure out how to do it together. But not just putting stuff into the model, but figure out how to prompt them, figure out how to, unearth capabilities. Let's work together for these sci fi use cases because I think that is the way the future is gonna go. And I would want so much more of that. My my joke internally is that the number one way to get me to respond to something is to talk about model jaggedness because you have, like, very high up people say, like, hey. Can we meet with Diogo? And I'm just like, no. Can you meet with my CTO or GTM head instead? But if they complain about model intelligence, I am, like, on that and, like, debugging that myself for better or for worse because that, I think, is the thing that will really move the needle for us long term. So right now, we mostly have to go with our old road map because we we are not... I I think people are still in that mode of, like, yes. Let's just do whatever the model providers let us do instead of, like, let's collaborate on this, like, insane future together. And, actually, there's another aspect of that. I think people have a deep distrust of model providers too because they train on their data. And, you know, like, this whole, like, discussion of, like, can GPT wrappers even exist, I think, is very disempowering too because the whole application ends up becoming a chatbot.
00:10:45 · Diogo, Jacob Effron
So the moat ends up becoming the equivalent of a system message that you send every time versus, like, the moat and the intelligence and the behavior and the specialization actually existing in software where we just make little micro decisions for you, and you do all of the sick logic. Yeah. So it... There's so much to do. There's so much to do. [00:11:04] S0: No. I think your point's really interesting because I feel like when Jev came out, there became maybe, like, a common set of patterns or rules that folks were like, oh, yeah. Like, this is a really good use case for for Jev, you should, you know, try it out in this thing or or or that thing. And everyone tried it out and, you know, kind of got that sense of patterns. You know, I almost feel like people are trying to get their own mental models of like, okay. I use, like, you know, Opus five five for, like, this kind of thing, and I use Jev for, like, this kind of thing. And I feel like your point's well... You know, that your point is like this is a dynamic evolving thing that you could... There's not strict boundaries, I guess, between those two types of model categories. But, like, what's your
00:11:04 · Jacob Effron, Diogo
[00:11:04] S0: No. I think your point's really interesting because I feel like when Jev came out, there became maybe, like, a common set of patterns or rules that folks were like, oh, yeah. Like, this is a really good use case for for Jev, you should, you know, try it out in this thing or or or that thing. And everyone tried it out and, you know, kind of got that sense of patterns. You know, I almost feel like people are trying to get their own mental models of like, okay. I use, like, you know, Opus five five for, like, this kind of thing, and I use Jev for, like, this kind of thing. And I feel like your point's well... You know, that your point is like this is a dynamic evolving thing that you could... There's not strict boundaries, I guess, between those two types of model categories. But, like, what's your [00:11:39] S1: Ish. Ish, I I actually think that what you're describe... Describing is a little bit sad to me, and I, you know, I understand that, like, we are playing a role in this. I do not think that that is the most developer centric worldview. Like, for people to start building, like, theories of mind of the submodels they call and have their systems locked into that is, in my opinion, a really, really bad thing. I think the North star for intelligence is gonna be a lot like SQL. Like, we wanna be so reliable that it's actually provider agnostic. We obviously want the best models, but, like, we don't want it to be locked into, like, things that only work with Jev.
00:11:39 · Diogo, Jacob Effron
[00:11:39] S1: Ish. Ish, I I actually think that what you're describe... Describing is a little bit sad to me, and I, you know, I understand that, like, we are playing a role in this. I do not think that that is the most developer centric worldview. Like, for people to start building, like, theories of mind of the submodels they call and have their systems locked into that is, in my opinion, a really, really bad thing. I think the North star for intelligence is gonna be a lot like SQL. Like, we wanna be so reliable that it's actually provider agnostic. We obviously want the best models, but, like, we don't want it to be locked into, like, things that only work with Jev. [00:12:14] S0: So what is your, you know, mental model for how this all evolves in terms of, you know, this, like, thing people are doing today of, like, you know, hey. You know, Jev works really well for these things. Maybe, like, there's other sets of models that have been not... Like, to your point, like, RLVR works really well for code and math, other... You know, we'll see what else it it ends up working really well for. Like, does... Well, I I have to interrogate that gesture. What what what was that about? [00:12:36] S1: No. You wanna you wanna interrogate it right now? [00:12:39] S0: Yeah.