Venture with Grace
00:30:57 · Grace Gong, Jerry Tworek
[00:30:57] S1: What motivates you? Since you want to I don't know, I don't think... I mean, I listened to a bunch of your podcasts, but I didn't know if you ever shared the story of coming from Poland. And is there how you grew up and then how you kind of... Do you always know that you wanna become an entrepreneur? Or obviously, it seems like you have a you know, great, like, trading career and to, like, you know, riding the wave of AI. But what kind of, like, you know, is something that you enjoy doing? And then what do you identify as your, like, unfair advantage as a as a founder? [00:31:30] S2: Yeah. Yeah. Like, think what mostly drives me, in many ways, is I just like building technology. This is, to me, what makes me get out of bed and what makes me wanna do things. I think building specifically new solutions to our products. I'm not AI. But I very much, very much enjoy building things and solving problems using science and engineering. And that's kind of what I just like doing and what I wanna be spending my days. And the rest of the question is how in which configuration for the rest of the world I want to be doing that. I have thought about being an entrepreneur on and off throughout my life. It's usually question. Is it the right time? Do I have the right idea? Is the world ready for what I'm trying to do? [00:32:17] S1: The world is not ready. Will be ready soon.
00:32:17 · Grace Gong, Jerry Tworek
[00:32:17] S1: The world is not ready. Will be ready soon. [00:32:21] S2: Exactly, exactly. There are doors. I think we have an idea for a company starting in college, which I don't think it was a very good company because it was mostly meant to be a company making personalized and interactive study materials for mathematics. Was my first You're [00:32:37] S1: newest company. Interactive mathematic... That sounds like ChatGPT. [00:32:40] S2: Exactly. In some way, working on AI and ChatGPT helped me realize that, and I think that is that is pretty nice. I I think I think back in the day, there wasn't really a good time, and there wasn't really a very good very good moment for this idea. These days, a few hours of byte coding, and you already can have a prototype of exactly [00:33:00] S1: You were just like ten year earlier. [00:33:02] S2: Exactly. Timing is very hard, and one of the most important things in starting a company. The question is, is this the right time to build a company I want to build? It's extremely, extremely critical. And you don't wanna be too early and you don't wanna be too late. [00:33:17] S1: Like, are you in the right time? [00:33:19] S2: I I assume so. That's why that's why I decided [00:33:21] S1: to do it now.
00:33:21 · Grace Gong, Jerry Tworek
[00:33:21] S1: to do it now. [00:33:22] S2: So I was thinking that the window is actually very short for starting a type of company I want So to I didn't have much time to hesitate. The first and foremost reason is you need agents to be good enough to kickstart you. Without agents, starting an AI lab is a very slow process and requires a lot of time. If you see the neo labs that started a year ago, two years ago, they were moving in a glacial pace initially. The amount of systems you need to do, the amount of work that you need to build out to get anywhere is just a lot. With modern AI solutions, we're able to probably compress two years of work in three months. It's amazing how much quicker you can move right now than ever before. And the second thing is you don't want to be too late from a perspective of already someone being pretty far advanced into some types of research that are doing. I think we are leading the guard into finding something that is very different to transformer better than transformer in agentic workloads. But eventually, other companies will be trying to ask the same questions. They are, I think right now, today, still busy with training transformers. But that will definitely change. [00:34:36] S1: How do you meet Rohan? Plug from our show. So, like, actually, Rohan? Let's look at our subject.
00:34:36 · Grace Gong, Jerry Tworek
[00:34:36] S1: How do you meet Rohan? Plug from our show. So, like, actually, Rohan? Let's look at our subject. [00:34:45] S2: Rohan, my co founder, great scientist and great passionate of deep learning. When I was trying to assemble a crew for that company, you can imagine you give people phone calls that you roughly know and roughly know their research. Hey, I'm assembling a crew. This is what we are trying to do. Are you in or not? And Rohan was one of the people who answered that call and said, like, you know, you, Jerry. I mean... [00:35:10] S1: Oh, wow. Like, no one... Oh, I mean, like, I'm sure, like, is there anyone said no? No? [00:35:14] S2: Like, there are definitely some people that said no. And, like, our our our kind of, like, ratio of people who decide to join is fluctuating over time. But yeah, especially early on, we had a pretty good close rate. [00:35:26] S1: What would you think... I guess, what makes people join you guys? Is that just because you guys are solving the word class? The word first... Only the first word problem right there.
00:35:26 · Grace Gong, Jerry Tworek
[00:35:26] S1: What would you think... I guess, what makes people join you guys? Is that just because you guys are solving the word class? The word first... Only the first word problem right there. [00:35:36] S2: Yeah. At least one thing, we have a very good time of hiring excellent researchers, because I think we are one of the companies with both an ambitious research plan, solving interesting problems. And actually, we are able to pretty well articulate it for those who come long enough in journey. And definitely, for other roles, we are slightly more eye to eye with other companies. If you work on systems or on data at Core Automation, the main part of the story is you will try to develop those things in as agentic way as possible using as much automation. You will be trying to become a person that does the equivalent of a work of a team in places. And that's our advantage. [00:36:29] S1: According to Twitter, I don't think you like eval these days. [00:36:32] S2: Yes. I think I think evals mostly lead you to good hearting and to thinking about things too narrowly. There were... [00:36:39] S1: Eval's kinda like advertising. Right? So it's like you just pick whatever you're good at and then pick the right evaluation things too.
00:36:39 · Grace Gong, Jerry Tworek
[00:36:39] S1: Eval's kinda like advertising. Right? So it's like you just pick whatever you're good at and then pick the right evaluation things too. [00:36:45] S2: There there there is very little signal in those things. Everyone who ever releases a model, they say, oh, we are good on those evals, and they cherry pick, and they kind of present their models as good. And the actual market value of those models is very different to what those benchmark says. And because those evals are public, everyone just tries to look good. And there are much deeper differences between models. And unfortunately, because there's just not really a good way to take, people are trying all those different models and trying to figure out at least those that want to spend energy on it and people who don't want to spend energy just take the most popular model and run away with it. But I don't think it's even a very good way of thinking about both models, about evals. It's a bit of an acronus of times when the models were frozen and were not really meaningfully learning after they are deployed. So if you take a GPT off the shelf, it has some evals because that's what it was been trained on. But if the model can learn, then those eval scores are different tomorrow. Even can be different from minute to minute. [00:37:50] S1: How do you use... I mean, how do you know if your stuff works well?