Unsupervised Learning with Jacob Effron
00:42:39 · Diogo
[00:42:39] S1: Oh, boy. It feels like other people are digging our moat for us, which is kind of not how I thought it would go, and I think that that's kind of bad for the field. I think that releasing models that are, like, kind of more benchmark optimized than real world optimized actually burns that goodwill for people to try things out, which is the only thing that matters. I I truly think that this is not great for the field, and my hope is that just giving unbelievable facts about reliability is going to be the north star. I think that will be immensely, immensely valuable for the world. There's just so much more to do there that I think that it's a little bit early to worry about competition. It's it's kind of like OpenAI worrying about a moat at at G P T 3.5. Like, who cares? Like, G P T 3 Point Five's market is nothing, or it was nothing, right, in comparison. It's still, I think, going to be nothing in the grand scheme of things. Like Yeah. We truly believe in Jevons paradox, obviously, hence the name Jev. Like, Jev and Paradox... Like like, are we seeing rebound, you know, more demand because it's so cheap? Absolutely. I believe every good sees rebound. Are we seeing backfire? Like, the total market size increases. This is this is when Jev and Paradox holds. Absolutely not. Like, it it... Like like, a key part of it has to be like, wow. This is so good and so cheap. I'm gonna be throwing it all over the place because it makes value in all of these places, and my budget for AI will actually increase.
00:44:12 · Jacob Effron, Diogo
[00:44:12] S0: I mean, speaking of investors, I guess you you obviously recently raised a a a pretty big round. How did you even think about the amount of capital that made sense to raise for for this business? I'm sure people were trying to throw money at you. So Yes. You know, how how do you how do you even, you know, think through that? [00:44:26] S1: I will say that people probably don't believe it, but, like, we can run our service profitably. And I hope I hope that we never have to do the thing of, like, temporarily subsidizing to get lock in. I I don't want to, like, axe it out entirely because, like, I'm a pragmatic person, but, like, I really hope that doesn't have to be the case. I hope that... I I find that to be, like, a very shadowy, sneaky way to get people to use your service. So I... And I hope to be the kind of person who's, like, you know, always collaborating in, like, these prisoners' dilemmas until, you know, until forced otherwise. So we actually did not need the money. We... And our goal instead was, hey. We want actually, like, the most valuable people on our side to help us build this business because we have no fucking clue how to do this. So we really opted for, like, who are the best people in AI who will help us. You can see who we got on our board.
00:44:26 · Diogo, Jacob Effron
[00:44:26] S1: I will say that people probably don't believe it, but, like, we can run our service profitably. And I hope I hope that we never have to do the thing of, like, temporarily subsidizing to get lock in. I I don't want to, like, axe it out entirely because, like, I'm a pragmatic person, but, like, I really hope that doesn't have to be the case. I hope that... I I find that to be, like, a very shadowy, sneaky way to get people to use your service. So I... And I hope to be the kind of person who's, like, you know, always collaborating in, like, these prisoners' dilemmas until, you know, until forced otherwise. So we actually did not need the money. We... And our goal instead was, hey. We want actually, like, the most valuable people on our side to help us build this business because we have no fucking clue how to do this. So we really opted for, like, who are the best people in AI who will help us. You can see who we got on our board. [00:45:27] S0: Well, I always like to end with a quick fire round where I just stuffed some overly broad questions into the end that I haven't had a a chance to ask elsewhere. I'm curious, you know, over the last month, you know, or last six weeks of having this product out there, like, what's one thing you've changed your mind on in AI in the last, like, six weeks, having just seen how Woah.
00:45:44 · Diogo
[00:45:44] S1: That's such a cool question. Woah. Woah. That's a quick fire question? Holy smokes. I will say that we were really worried if our philosophy about caring about, like, the actual reliability of intelligence was a mistake. Like, we had people in the team say, like, hey. Should we just release a Benchmax version and just hype it up and, like, do one of those early releases that, like, obviously is not great, but just to, like, test out if the market exists? I'm so glad we didn't do that. And I I say seven weeks because, like, I I don't remember when launch was, but, like, seven weeks, I think, was before launch. We really did not know this. We... It was, a big leap of faith for us, and, like, we were just really... Man. Like, I don't think we could be, like, proud of ourselves if we did the same overpromise, underdeliver. And, like, I think that that paid off super duper well. I actually don't know if it paid off. Maybe the product sold itself, and maybe it could have done even better if we, like, really optimized for for attention. But that was, like, awesome. And I actually feel like now that my brand has been solidified as our brand, I hopefully never have to do that, even close to that again, and that debate no longer exists for us. I hope. I hope. I don't know. I truly, truly hope because I have, like, a hard time, like, even giving, like, unnuanced, slightly incorrect answers because I think that that's just a disservice to anyone listening to me. I would rather just, like, not answer the question than do the standard, like, CEO compression of the truth that airs on the side of generous. So I hope that I will never really have to do that. But, like, something I've learned as well is that this reliability saga was sped run so fast for our class of models. I feel like... I I could be wrong, but my temperature check of the environment now is people have already gotten, like, extreme fatigue of, like, the benchmarks and not actually useful stuff way faster than I would have thought. Like, I I would have, like, thought that this would be a concern based on what I see because people keep falling for benchmarks every time a new model's released. Right? People are like, well, cool. Let's talk about it. And then it it it disappoints, except when it's like an OpenAI anthropic model, apparently. And that just happened. Kai...
00:48:04 · Diogo, Jacob Effron
I I could be wrong, and, like, maybe we'll we'll see. I think it just happened a lot faster than I would have thought because our model is so benchmarkable such that people could, like, blast out so many benchmark crushing things so fast. [00:48:18] S0: I feel like this next one is gonna be very anticlimactic in comparison. But I am... As a podcast host, I'm contractually obligated to ask you, you know, like, we're we're an AI podcast. The other big AI story right now is obviously Anthropic, you know, going public and and folks kinda thinking through that business. I'm And struck even in the course of this conversation, like, I actually don't know how you're gonna answer this question. Because on the one hand, I feel like you're very disappointed in, like, so many of the the lack of capabilities that LMs have, and maybe that they're, like, almost a hammer trying to, like, you know, go on every nail of of, like, different sorts of use cases. On the other hand, you have been complimentary of there are things that coding agents are really good for. And so I guess as you think about Anthropic as $1.52000000000000 dollar company, is that something you're optimistic on? It's like, hey, that's undervalued or something you're like, oh, I don't know what's gonna to that.
00:49:03 · Diogo, Jacob Effron
[00:49:03] S1: I think coding agents are massively valuable. It's unclear to me how much of the actual use is truly valuable. Like, it being, like, net positive is different than it being worth all of the money put in. That is probably a product question for them to figure out. If they actually are truth seeking, they probably have a better answer to this. Like, did some people, like, just token max too hard, or are they actually getting the relative value that spending that would make a difference on? So I don't think I'm informed enough for that. I have my guesses on what percentage gain coding agents are for various levels of projects and skills of engineer. And I wouldn't say that software is anywhere close to solved, but that doesn't mean that they're not valuable. I just think that there needs to be, like, a pragmatic take here. I think that it's possible that the coding... I could see the coding agent market going either way. Like, I could imagine it going very up and very down, very up in the sense that as people learn to actually get more value, can people use it more? I believe so. I could imagine it going down because a lot of people had, like, overdone it because, like, there's so much FOMO in the field. And, actually, my sense of the state of AI is not that it's been, like, an upward exponential, but a lot of, like, hills that keep going up, down, up, down, up, down, and then eventually having, like, a little bit more at the trough trough trough of disillusionment. I actually don't know how that word's pronounced. [00:50:25] S0: The trough. Yeah. Yeah.