00:36:31 · S2, Rajkumar Sen
[00:36:31] S2: Yeah. [00:36:31] S1: So if you have a very high throughput, this is the classic, like I came back to the Arcion example, right? People are doing batch data movement, but then they want that continuous CDC, and I feel like training and inference is kind of the parallel thing, right? The inferencing, because the continuous process and live operational has to happen at super high throughput, super high uh scale, but also a lot of custom things have to be done per application. Like if you're running an application that does 700, 800 calls per second, that has some bespoke features that can only be done with a combination of base 10 or together with a chemie or one of the OSS deep sick models. Sure, entropy can solve that problem, but it's all about like you know, can you solve all the problems on Earth? [00:37:18] S2: Yeah. [00:37:19] S1: So there is that section of I'm sure people when we use Cloud or OpenAI, or obviously you're doing inferencing there. But if you want to run that kind of scale application where you need that at high throughput, you have to go to another vendor. Maybe one of these vendors will be bought by somebody at some point. But uh same thing when things get big, they will need scale, they will need uh you know, throughput, you will need one customization, which bigger companies may not want to do, but then that's where the startups thrive. And uh, I'm I'm super uh excited on the influencing companies because they feel like with the combination of open source and uh you know, then they can bring that value, right?
00:37:19 · Rajkumar Sen, Siddhartha Ahluwalia
[00:37:19] S1: So there is that section of I'm sure people when we use Cloud or OpenAI, or obviously you're doing inferencing there. But if you want to run that kind of scale application where you need that at high throughput, you have to go to another vendor. Maybe one of these vendors will be bought by somebody at some point. But uh same thing when things get big, they will need scale, they will need uh you know, throughput, you will need one customization, which bigger companies may not want to do, but then that's where the startups thrive. And uh, I'm I'm super uh excited on the influencing companies because they feel like with the combination of open source and uh you know, then they can bring that value, right? [00:38:06] S0: We have about two companies which are in inferencing. One is a company started by the founder of Qure.ai. Qure is one of the leading radiology AI companies, 40 million R valued at 400. So the the hospitals that were using Qure, uh they never relied on models, right? They said we want influencing in-house. The founder came out of it and just built inferencing in-house.
00:38:06 · Siddhartha Ahluwalia, Rajkumar Sen
[00:38:06] S0: We have about two companies which are in inferencing. One is a company started by the founder of Qure.ai. Qure is one of the leading radiology AI companies, 40 million R valued at 400. So the the hospitals that were using Qure, uh they never relied on models, right? They said we want influencing in-house. The founder came out of it and just built inferencing in-house. [00:38:30] S1: Yeah, that's the whole point, right, with the open source thing, right? Like if you're building IP, depending on your relationship with the uh with these big companies, you may want to trust them or not trust them, right? It's it's subjective, or maybe for certain applications you trust them. So if you want to build in-house, you've got to, you know, and then I think the inferencing space now is going to be like, you know, all day there's only one craigslist. Now we have Airbnb, DoorDash, everything, right? Now instead of horizontal inferencing, now you have inferencing for video, inferencing for you know image, like some something. [00:39:01] S0: Okay, but another company that was running inference for voice. So because everybody wants to specialize voice. [00:39:06] S1: Yeah, yeah. So I think uh as a category, it's very fascinating to me. Yeah. Uh, because as a systems guy, I feel like that's one space where a lot of what I learned to scale databases, all the techniques that were there to scale data management will apply. [00:39:24] S0: But can inference get bigger than cloud?
00:39:24 · Siddhartha Ahluwalia, Rajkumar Sen
[00:39:24] S0: But can inference get bigger than cloud? [00:39:27] S1: Can inference get I truly believe that can be true. Because if you think after five years, if every app is AI, it's only inferencing, right? That was the grand vision of Arcion. At some point, data will become so big that you cannot do batch data copy, you have to do differential data copy. It doesn't happen, yeah, it's happening. Uh now there are so many GDC companies including PATB, but the vision was that. Now think of like the same thing, like you can't you will have to train periodically training, but inferencing is similar, like the continuous thing. And if every app is shifting to that, it is going to be bigger. Uh it will be at very big. I don't know how you know, but um, and I feel like most of the innovation at least will be in inferencing side. [00:40:30] S0: Yeah, because cloud today combined AWS are at between 180 and 190 billion, uh Azure is at 140 billion, they run rates, uh, 140 billion run rate, and GCP is at 90 to 100 billion. So combined if you see 180 plus 140, the other 500 billion.
00:40:30 · Siddhartha Ahluwalia, Rajkumar Sen
[00:40:30] S0: Yeah, because cloud today combined AWS are at between 180 and 190 billion, uh Azure is at 140 billion, they run rates, uh, 140 billion run rate, and GCP is at 90 to 100 billion. So combined if you see 180 plus 140, the other 500 billion. [00:40:46] S1: No, no, totally true. And see, I think all this valuation is also valuation, these are the revenue. Yeah, revenue. Yeah, no, I'm seeing the like the rev. Sorry, the the revenue also this is like an app ecosystem, right? You had Aurora and all the databases and data management. All of them will like start. They you're not going to charge premium for databases right now. It's like an operating system right now. You will charge on the hardware and maybe some premium. Where you are, the money people will pay is on AI services. Innovation will happen in the database space, but it will be one you know table stack feature where you can't get a new customer by just building a new database. So given that the top dollars will be because that's net new thing, right? Yeah, databases are awesome, but um, can you deploy a new AI application which is like doing some something new for you just by using a new database? No, you know, you need to have all the inferencing and all of it. So yeah, I'm very excited. Uh, I actually started looking at, you know, so you know, uh, some uh as just as an academic in me, you know, I look at that very interesting.
00:40:46 · Rajkumar Sen, Siddhartha Ahluwalia
[00:40:46] S1: No, no, totally true. And see, I think all this valuation is also valuation, these are the revenue. Yeah, revenue. Yeah, no, I'm seeing the like the rev. Sorry, the the revenue also this is like an app ecosystem, right? You had Aurora and all the databases and data management. All of them will like start. They you're not going to charge premium for databases right now. It's like an operating system right now. You will charge on the hardware and maybe some premium. Where you are, the money people will pay is on AI services. Innovation will happen in the database space, but it will be one you know table stack feature where you can't get a new customer by just building a new database. So given that the top dollars will be because that's net new thing, right? Yeah, databases are awesome, but um, can you deploy a new AI application which is like doing some something new for you just by using a new database? No, you know, you need to have all the inferencing and all of it. So yeah, I'm very excited. Uh, I actually started looking at, you know, so you know, uh, some uh as just as an academic in me, you know, I look at that very interesting. [00:41:58] S0: Um, let's see, you have now been inside Databricks, partnered with Databricks, you know, Ali closely, you know, inside what makes Databricks so successful? Because for a long period of time, uh, you know, they were nobody for the first five, six years.