Tech Talks Daily
00:11:10 · Neil C. Hughes, Paul Speciale
[00:11:10] S0: Oh, wow. And for people listening, maybe their their teams need to be testing that governance and sovereignty rules still hold when data moves across so many different locations and storage tiers. Always trying to give people listening a valuable takeaway. Any tips, hints, advice, or best practices that you could provide there? [00:11:29] S1: Yeah. I think our rule of thumb is that if it... If you have policies for this and you don't test it, then you're really just residing on hope. Yeah. You know, people have gotten really good. Corporations have gotten really good at testing code. It's continuous. It's after every change. I think what we still see is that governance and sovereignty checking happens kind of once a year at audit time. You need to do these tests, and you need to do negative tests. Try to move data somewhere where it isn't allowed and see that the system refuses. K? You can actually tag data and say its jurisdiction is here. Now you can make sure that every test moves it to somewhere that is enforcing that rule. Right? And always just keep an always current record of where all the data is. Number one problem in the organization is lack of visibility. Where are my backups? Where are things stored? Which one's the good one? Make sure that you have a current visible record of that.
00:11:29 · Paul Speciale, Neil C. Hughes
[00:11:29] S1: Yeah. I think our rule of thumb is that if it... If you have policies for this and you don't test it, then you're really just residing on hope. Yeah. You know, people have gotten really good. Corporations have gotten really good at testing code. It's continuous. It's after every change. I think what we still see is that governance and sovereignty checking happens kind of once a year at audit time. You need to do these tests, and you need to do negative tests. Try to move data somewhere where it isn't allowed and see that the system refuses. K? You can actually tag data and say its jurisdiction is here. Now you can make sure that every test moves it to somewhere that is enforcing that rule. Right? And always just keep an always current record of where all the data is. Number one problem in the organization is lack of visibility. Where are my backups? Where are things stored? Which one's the good one? Make sure that you have a current visible record of that. [00:12:22] S0: And you've seen so many big changes in this space. I know you've been in this... The the space a long time, seeing so many big changes, all these different trends. But what is it that changes for infrastructure teams listening when their AI inference runs on infrastructure that they actually own?
00:12:22 · Neil C. Hughes, Paul Speciale
[00:12:22] S0: And you've seen so many big changes in this space. I know you've been in this... The the space a long time, seeing so many big changes, all these different trends. But what is it that changes for infrastructure teams listening when their AI inference runs on infrastructure that they actually own? [00:12:38] S1: Yeah. I think the parallel here is to the cloud. Right? We had public cloud starting in the 2006, 2010 era with Amazon and all the like. Public cloud sounded great. And then everybody went into it and had this shock bill of all these surprise charges. I really think the same thing can happen here in AI. So the big changes, infrastructure teams need to become the owner of the AI bill. Okay? They're the ones who can manage it and make it smaller. If they do that, they will start tracking metrics like how much time to the first token, how much reuse is there, right? How much of the context becomes data that they need to actually manage? They can then make decisions more consciously about when does a model need to change, right? They'll find out things like the network matters more than people expect it. It's really the limiting factor a lot of times in the performance to keep the GPUs fed. So I think the lesson is the infrastructure team needs to become the owner of the AI bill, then they'll optimize it.
00:12:38 · Paul Speciale, Neil C. Hughes
[00:12:38] S1: Yeah. I think the parallel here is to the cloud. Right? We had public cloud starting in the 2006, 2010 era with Amazon and all the like. Public cloud sounded great. And then everybody went into it and had this shock bill of all these surprise charges. I really think the same thing can happen here in AI. So the big changes, infrastructure teams need to become the owner of the AI bill. Okay? They're the ones who can manage it and make it smaller. If they do that, they will start tracking metrics like how much time to the first token, how much reuse is there, right? How much of the context becomes data that they need to actually manage? They can then make decisions more consciously about when does a model need to change, right? They'll find out things like the network matters more than people expect it. It's really the limiting factor a lot of times in the performance to keep the GPUs fed. So I think the lesson is the infrastructure team needs to become the owner of the AI bill, then they'll optimize it. [00:13:40] S0: And I must admit, when you started talking today, especially around the topic of storage, I immediately thought you'd be a great fit for the IT press tool, which is something I've been on multiple times over the last, what, four, five years. And I noticed before you joined me, you were actually on that tour in, what, way back in 2010, 2011, 2012. How much has got. Yeah. How much has changed around storage in in that time? What excites you?
00:14:07 · Paul Speciale
[00:14:07] S1: It's been such an inflection point. I'll tell you back in that era, object storage was kind of emerging. Right? And I was actually with another one of the object storage vendors that was coming out at the same time. You know what we all all were pained by at that time? There was no standard API, no standard interface, no protocol. In fact, I remember being the product manager for such a product and everybody said, the number one feature we need is file, file based access. Well, that kind of defeats the purpose of an object store. So one big inflection point, everybody embraced Amazon's s three API. It's a big deal. Okay? Number two, a big inflection point was cloud native, containerization and Kubernetes. Right? Because it meant the way that storage was provisioned became more automated. That was a really big deal. We had to react differently. We had to have different APIs. And then third, this idea that storage would be instrumental in cybersecurity. Backups, as I said a few minutes ago, are critical. Right? They're kind of your last line of defense. That was a big deal. Before that, we were just thinking, oh, backups need to be faster than tape, so let's put them on disks. Right? That was the whole deduplication era. Now we realize it's not about the backup, it's about the restore. Right? Yeah. So you need to be able to have that copy, it needs to be safe, it needs to be come back really quick. And look, I can't ignore it, but I got to say this last era, this current era, AI, is the other inflection point. Right? We never really thought that we'd be used for AI inferencing. Right? That's a big change. And now making sure that storage is fast enough for that is the real key.
00:15:39 · Neil C. Hughes, Paul Speciale
[00:15:39] S0: It really is. And of course, we're talking today just a couple of weeks after you launched your AI Inference Factory to bring enterprise AI on premises. Where do you go from here? What does 2027 look like for you? What excites you about that and where you... [00:15:53] S1: Well, think now it's a lot of the things we've talked about here. Right? Making sure that people understand that controlling their own fate in terms of their own infrastructure, we think sovereignty is a huge deal, especially in Europe, but also broadly speaking around the world. People are gonna know what the lineage of their data is, what the lineage of their code is. This whole open code model is something we're pushing. AI Inference Factory uses open weight models. We think that's a big deal also in terms of making sure you can lock down and trust the models that you're using. You've seen this too if you use one of the frontier models. They change it on you almost weekly and it's very hard to control. So we think controlling, maintaining the cost, maintaining the sovereignty of AI is what's next for enterprises. If you read a lot of the industry literature, you'll see the prediction is that more and more of inferencing becomes on prem and really to an end state of hybrid. So people are using it partly in the cloud and partly on prem. I think that's the future.