Unsupervised Learning with Jacob Effron
00:50:10 · Danila Shtan
[00:50:10] S0: In a sense, we see a lot of push and concerns about people like claiming that OpenAI models or Anthropic models are too expensive. Everyone is puzzled or surprised by the bills they get, they get to the end of the month. So this might be like an answer to that. Well, if you're like, oh, go try open open models. At the same time, open models are still behind the closed source... Closed models. Right? However good those models are, they're still lagging behind. And giving people an option to try, like, lower scale with the ability to switch back to the top frontier model, it also be a good incentive. Like, anyway, I think like OpenAI is doing a very interesting move with that partnership. And I like it really much because like, this kind of shows flexibility. Right? They're not set and so. Like, they're exploring different options, different things that they can or cannot do. Like looking for different options to like so... But I I I think in a sense, this all plays into one single thing. Like OpenAI wants... Same as Anthropic. They want to be one single, like touch point. Like they want to be one single thing for everything. Even that is like, okay, but sometimes I want to use open source models within my codex controlled environment, by the way. Right? So like, well, good for them. But if the company, the person, the user is still within your ecosystem, you still have a lot of options to steer them, to upsell them, to introduce them to new services, etcetera, etcetera, etcetera. So if that, like, alleviates one of the concerns, so be it. Why not?
00:52:07 · Jacob Effron, Danila Shtan
[00:52:07] S1: Well, I always like to end these interviews with some, like, overly broad questions that I stuff into the to the very end. And so if you'll if you'll humor me, I guess Sure. First question is just, what's one thing that you've changed your mind on in AI in the last year? [00:52:19] S0: In the last year? Oh, a lot a lot has happened in the year. No. Like, the obvious answer would be software engineering. Like, a lot of people forget, but a year ago, we lived in a totally different world. [00:52:34] S1: It's amazing. Like dog years in AI world. [00:52:36] S0: Less Way less like capable. Yes. Way less capable in terms of the model capabilities and in terms of how good the modern harnesses and tools are. It's like the AI coding. Because like a year ago, it was total bullshit. Right now, it's it's a solid part of, like, your everyday tooling. But for example, if you're not talking about the industry trends or, like, talking about more, like, my personal thing, I've started relying more on the, like, AI enabled, like, local agents and harnesses to offload a part of my cognitive load, to be honest. [00:53:18] S1: I mean, obviously, like, run a really large technical organization. And and I think everyone's trying to figure out, you know... I mean, obviously, the models only recently got really good on the coding side. So everyone's trying to figure out what it means to build teams given the new model capabilities and run an organization. I'm curious what you've... Where you've settled and how you've thought about that.
00:53:39 · Danila Shtan
[00:53:39] S0: Cautiously. Conservatively. I'm very concerned that modern models and the attractive user experience that they provide, like modern agents, modern models, etcetera, etcetera, etcetera, is something that will lead us to two issues. You like... You see slop, like, being revealed in, like, in chats, in merge requests, all over. And I, like, I try to pace that, and I try to, like, be very observant and vigilant about that. So for example, like, don't restrict talking usage. Like, people can use whatever they want. Our budget is quite significant. We don't impose, like, per person limitations or per department limitations. But the... At the same point, we try to, like, look at the usage, figure out what it is used for, look at the most ineffective ways people use it. We, like, constructed a set of policies that doubles down in the idea that whatever you've done with the agent, it's still your result. It's your responsibility. It's your pull request. It's your code. It's you who is on call. It's you who's gonna wake up during... In the middle of the night if something goes awry, etcetera, etcetera, etcetera. And if something was created by a completely autonomous agent, it is marked like that. Completely autonomous agents will not touch, like, the core services for a while because, like, this is, like, this is, like, out of bounds. Like, sometimes people are surprised that, like, I work in a company that's kind of participates in this whole Yeah. AI revolution, and I'm very conservative. But at the same time, like, I've seen how this sausage is made, and I, like, I'll be cautious.
00:55:48 · Jacob Effron, Danila Shtan
[00:55:48] S1: What's the biggest mistake you feel like you've made in in building Nebius so far? Obviously, a lot's gone right. But I'm curious, you know, something that in retrospect you wish you guys had done differently. [00:55:56] S0: It might sound funny these days, but we underestimated the scale. Like... And it's not about the growth of the company. We were pretty optimistic from the very beginning. But I personally... And that was, like, my strong bias for the first, like, significant part. I underestimated the scale that a single client operates at nowadays. Because, like, five years ago, if you're talking about a 30 megawatt data center, it was a serious facility. It hosted, like, hundreds of thousands of servers, like, run glorious... Gloriously large foreclosed. By now, 30 megawatts sometimes is not enough for a single client, which I'm still getting used to. Right? The storage I've mentioned. Right? Like, four years ago when you were talking about network attached storage, you were talking about, like, hundreds of megabytes, maybe gigabytes per second of throughput. Right now, some of our clients routinely fetch, I don't know, seven hundreds of gigabytes per second from a single shared file system. I'm still... I underestimated it in the beginning, and I'm still getting used to this scale. Like, it's... Like, I'm an infrastructure person. I've worked and have seen these numbers throughout my career multiple times. And all of a sudden, you add a couple of zeros to the end and call it a new norm. Yeah. Still puzzling.
00:57:31 · Jacob Effron
[00:57:31] S1: It's crazy. And I guess lastly, like, there's obviously a lot of, you know, unsettled questions in the in the AI field today that, you know, lots of people trying to figure out where where the world goes. I'm curious, like, you know, what you're paying attention to. And I guess, you know, with the lens toward, you know, if it goes one way or another, it would really change kind of the strategy for for Nebius as a whole. What are you kind of most paying attention to on on, like, these these unsettled questions of where the AI world's going?
00:57:57 · Danila Shtan
[00:57:57] S0: Well, I stopped caring about, like, the new models being larger or smarter. I just accepted that it's a given. I'm kind of trying to look out for, like, new novel architectures and approaches, but I don't see any as of today. I hope somewhere deep down in like a stealth lab, something new is cooking. But for now, I frankly don't see anything that like is a thing that I would be, you know, like would place even a small bets on. But I'm more and more interested in the whole outcome optimization part on the agent side. Right? Because, like, we we are used to talking about models and size of models and families of models, etcetera, etcetera, etcetera. But now that we've shifted to this multi turn, long running agents that we are talking not about requests, but rather outcomes, companies are talking about bidding for per outcome. Right? I'm very interested in how that part will shape up. And we have, like, a small research group internally with the Nebius that's working on this problem, actually. How is, like, how do you increase the chances of getting to that outcome? Not by increasing the model or, like, changing the model. That that is, of course, part of the discussion. But I believe there is a lot of optimization on the part where you introduce more sophisticated or maybe more simplified tools. You introduce the harnesses that start to evolve according to a particular task. And this kinda gets us back to, like, a significant CPU consumption. Because, like, a lot of that... A lot of, like, modern AI workloads actually go back to becoming code. Like, this qualified. It's, like, very formal. You see, like, quality gates. You see sophisticated tools, etcetera, etcetera, etcetera. And sometimes your agent works for minutes and minutes minutes without even issuing a back end request to an LLM because the workloads start to, like, to sit on this part. And this is the part that really interests me. Because I think, like, a lot of future AI adoption lies in how reliable your agentic scenarios are and how reliably you reach those outcomes. And I think, like, the whole optimization loop outside the model is important part of that.