Tech Talks Daily
00:15:39 · Neil C. Hughes, Paul Speciale
[00:15:39] S0: It really is. And of course, we're talking today just a couple of weeks after you launched your AI Inference Factory to bring enterprise AI on premises. Where do you go from here? What does 2027 look like for you? What excites you about that and where you... [00:15:53] S1: Well, think now it's a lot of the things we've talked about here. Right? Making sure that people understand that controlling their own fate in terms of their own infrastructure, we think sovereignty is a huge deal, especially in Europe, but also broadly speaking around the world. People are gonna know what the lineage of their data is, what the lineage of their code is. This whole open code model is something we're pushing. AI Inference Factory uses open weight models. We think that's a big deal also in terms of making sure you can lock down and trust the models that you're using. You've seen this too if you use one of the frontier models. They change it on you almost weekly and it's very hard to control. So we think controlling, maintaining the cost, maintaining the sovereignty of AI is what's next for enterprises. If you read a lot of the industry literature, you'll see the prediction is that more and more of inferencing becomes on prem and really to an end state of hybrid. So people are using it partly in the cloud and partly on prem. I think that's the future.
00:15:53 · Paul Speciale, Neil C. Hughes
[00:15:53] S1: Well, think now it's a lot of the things we've talked about here. Right? Making sure that people understand that controlling their own fate in terms of their own infrastructure, we think sovereignty is a huge deal, especially in Europe, but also broadly speaking around the world. People are gonna know what the lineage of their data is, what the lineage of their code is. This whole open code model is something we're pushing. AI Inference Factory uses open weight models. We think that's a big deal also in terms of making sure you can lock down and trust the models that you're using. You've seen this too if you use one of the frontier models. They change it on you almost weekly and it's very hard to control. So we think controlling, maintaining the cost, maintaining the sovereignty of AI is what's next for enterprises. If you read a lot of the industry literature, you'll see the prediction is that more and more of inferencing becomes on prem and really to an end state of hybrid. So people are using it partly in the cloud and partly on prem. I think that's the future. [00:16:52] S0: Awesome. And for anybody listening interested in exploring that AI Inference Factory and and learning more about how they come up to bring their enterprise AI on premises, where should they go? Where can they find out more information and and maybe connect with you or your team?
00:16:52 · Neil C. Hughes, Paul Speciale
[00:16:52] S0: Awesome. And for anybody listening interested in exploring that AI Inference Factory and and learning more about how they come up to bring their enterprise AI on premises, where should they go? Where can they find out more information and and maybe connect with you or your team? [00:17:06] S1: Yeah. Well, do you mind if I give a couple of plugs? Yeah. Course. The website, scalety.com. We have an ancillary blog on another property called solved.scalety.com. You can read all about the thought leadership and these ideas that I've talked about. And then selfishly, me and my CTO host a podcast on YouTube, not anywhere to compete with yours, but it's called the Scale Out Podcast, and we talk about topics that are relevant to us, right, things like k v cache and inferencing and all the tech underneath it. So those are three places people can find out more. [00:17:39] S0: Awesome. I will add links to everything. For people listening, if you go to techtalksnetwork.com, there'll be a blog post associated with this episode. Not only will I include the links that you've mentioned there, I'll also, embed one of your YouTube episodes. So please go check that out. But more than anything, Paul, just thank you for coming on here, talking about all this very complex technology in a language everyone can understand and the value behind it too. Really appreciate your time. [00:18:03] S1: Thank you. I value the opportunity, Neil. Thank you.
00:18:03 · Paul Speciale, Neil C. Hughes
[00:18:03] S1: Thank you. I value the opportunity, Neil. Thank you. [00:18:05] S0: I think Paul's explanation of reusing context gives infrastructure teams a way to to think about costs. The question is how much the work the system repeats, where the information is held, and whether it can be retrieved efficiently. Owning the infrastructure, yes, it brings responsibilities because teams will need to understand utilization, operating cost, model changes, and the rules that are governing their data. So I think ultimately, the business case will depend on the workload and the organization's ability to operate it reliably. I also appreciated Paul's advice on testing restrictions because trying an action that should be refused can reveal whether a policy actually holds in practice. And recovery exercises, they show whether protected data can be restored when needed. So a big thank you to my guest. You'll find all the product information, technical blog, and scale out podcast over in the episode notes at techtalksnetwork.com. But over to you, which of your AI workloads would you consider running on your own infrastructure? And what evidence would you need before deciding to do that? After listening to today's episode, show me your thoughts over at techtalksnetwork.com, and I'll get straight back to you. But that is it for today, so thanks for listening as always. Bye for now.