Venture with Grace
00:34:43 · Vitaly Gordon
[00:34:43] S1: Yeah. So there's a of couple of ways people think about it. Right? So maybe, you know, the most sophisticated users, they will spend most of their time on the context and the evals and then kind of let machines kinda do their their thing. All that, there are there are people who are, you know, a little bit maybe kind of... Again, we'll go and have the local horizon task, and then they will just kinda go and validate kind of, like, the result and and do it. And there's people who are, you know, kind of a little bit more, you know, kind of guiding the agents a little bit more. So one of things that we also start now getting data for is, like, a lot of session data. And we actually, like, look and we identify that people are using AI very, very differently. And and and so this is... I was thinking of, like, sorry. I I would go on kind of a different path for for a second there. I I think we kinda underestimated of what it means to move from, like, descriptive coding languages where, you know, there are very, very concrete best practices on how you like, do things to, like, just start in English, to kind of like, machines. And the thing is, like, you can speak English, but, like, there is, like, so many ways that you can ask you, you know, your helper to achieve a task. And what we found is like, there is just very different ways of... Very different ways of kind of using using these agents in order, to achieve those goals. And so therefore, it's kind of, you know... Yeah. There was just, like, no no one way. I would add another thing that I just noticed on myself where the Terra model that just came in. So I'm a Codex user mostly, Terra just came out. The model is pretty fast. It's, like, three times faster than, you know, solo or Astra. Now what it means is actually, like, if before, I would like... My work thing would be, hey. I cannot go and define task and then I go and do something else. In this, in many cases, like, the task actually completes very quickly. And so that's gonna also change my style. And so and so it got me start thinking, we go with, like, diffusion models and there's, like, new models that appear that are now, like, let's say, 10 times or or somehow we get a lot more capacity. What does it mean if, like, now a model can finish a task, like, times faster than before? Like, I don't know.
00:37:14 · Vitaly Gordon, Grace Gong
It probably will change the way I work again. And so those are also some of the things that we're thinking about. [00:37:25] S0: So when, like, how does your current workflow look like at your own company? Like, do you guys like generate codes and then like, you know, I guess, obviously you guys generate codes, like, what I meant is like, how do you implement your own product and your own kind of like day to day? [00:37:40] S1: Yeah. So we're a little bit of a weird company because part of what the... What we offer to customers is like, is we will have answers for different ways of working. So And we actually are almost like forcing people to work in different ways just so we test the different kind of... It's could be like different harnesses, different types of model, know, more expensive, cheaper, kind of things like that. So we try we try to kind of distribute and kind of, like, understand, like, what works. So it's not necessarily the... While our product, you know, allows to, like, one optimized way, we're constantly, you know, trying to kind of increase the the exploration. But we definitely have ways that we found that some ways are... You know, could literally be 10 times more efficient than, you know, than other than other ways, and that's also something that we share both internally and with our with our customers. So while we, you know, we know the answer, actually, we kind of, like, almost, like, force ourselves to work in the not not the most efficient way. [00:38:51] S0: What do mean by like, you're like asking them to try out different things?
00:38:51 · Grace Gong, Vitaly Gordon
[00:38:51] S0: What do mean by like, you're like asking them to try out different things? [00:38:55] S1: Yeah. Like, like, I'll give an example is, you know, we have our, you know, our own harnesses that we, you know, that have a lot more context and that we do, but, like, not all of our engineers are using it because we don't want them because, like, our customers will not be using them. Right? That's kind of the things, like, things like that. And there's, you know, certain, you know, skills and, you know, and ways to, like, add context to things. So there's, like, a lot of, you know, automation. And in some cases, like you said, we have a codecs and clog, but also we have open weights models and we have other things. So and sometimes, like I said, we wanna, especially if we have customers who, let's say, are, maybe a little bit more, I would say, regulated and they actually are not using the frontier models, but they, like, literally actually running, you know, some open source models on their own kind of bare metal hardware. Like, we also wanna understand, you know, what it feels like, what... You know? So we will have some engineers who, like, literally, will buy buy machines, put them in our in our office, and have assigned some engineers to work that way just so we understand the implications, and we can go in. So it's kind of like a weird analogy of, like, how people used to, like, you know, test mobile apps on many many different mobile devices. Like, that's kind of like, we have this, like, you know, weird AI equivalent of that.
00:40:24 · Grace Gong
[00:40:24] S0: How do you feel like the... Each engineering team should kind of allocate their budget after looking at all the data?
00:40:32 · Vitaly Gordon
[00:40:32] S1: Yeah. So I would say there are two extremes, right? One is look, look, sometimes you're you're, you know, a large organization with a very, you know, strict budgeting and you just need to kind of like, okay, you have a very specific AI budget, You cannot go beyond it. And I would say that if that's the thing, you should just now thinking about within the budget, how do I maximize out? And I'll tell you, like, the best way to maximize outcomes is actually to make sure that you don't run out of budget. And let's say if you have a monthly kinda quota, don't run-in the middle of the month because it's very, very hard to go back from, like, using AI to not using AI. Like, most people will, like, probably will, like, do nothing and, you know, and that's kinda the weird part. And so it's probably better to, like, you know, have them use, like, much, much cheaper models, but have them use AI for, you know, the entire period of the month as opposed to, like, just have them to use kind of, like, a more expensive models. Then in the middle of the month, like, they just, like, have no access to AI. Right? So that is... And again, these are, like, strategies that we also work with customers on, like, how to detect that, you know, the forecast of using it goes above their, you know, quota. And then, you know, once it happens, we kind of, like, can redirect some of work to cheaper models to get them back in line so... To make sure they stay on on budget. The second thing, which is, you know, more common, would say, with AI native companies, is, like, as long as you actually, like, producing, you know, great outcomes, you should do more and more. And so that is more about, like, I would say, budget allocation. Like, you... If you have teammates that are just, like, you know, crushing it and they're, like, you know, every dollar that you give them on AI turns into, like, 10 features, you should give them more budget. Right? Like, that's what we should be... What you should be doing. And so, like, this... But but, again, all of these things on large scale, like, you just cannot manually do it. Like, so we kind of, like, created this, like, fully automated system that automates all of these decisions, you know, for, for the companies. And so... And there's, like, you know, some variations in between those two extremes.
00:42:40 · Vitaly Gordon, Grace Gong
But, like, I I would say that, you know, on one hand, you know, maximize outcomes, but stay within budget. On the other is maximize outcomes and add more budget, as needed as long as you continue getting the outcomes that you want. [00:42:53] S0: When you're thinking about, I guess, the competitive advantage between you guys and some of the other players in the similar domain, where would you say is a place that you actually want to build and mold on? Since I feel like one of the things from my understanding, it's in the space, obviously, I think I would assume it has a lot of stickiness as you have to have access to the entire code base and everything's tooling and everything else. But before you enter, what is the journey looks like for you guys to identify the client that actually needs this product? And then is there any clients or is there any specific type of AI company that just don't need this at all?