411: Jeremy Burton, GM of Observability at Snowflake: The Data Layer Was Never a Commodity. It’s the Whole Game

00:18:57 · Dan TurchinPlay from here

[00:18:57] S1: Now the AI tooling companies would say, including the Foundation Labs, say, we own the cognition, and the data layer is a commodity. And presumably Snowflake and maybe your competitors would say, your cognition is dumb without the context provided by our data. Where do you where do you come out on on this? Who who owns the value from the perspective of the user and and the enterprise when nobody wants to cede any ground to to to anyone else?

00:19:31 · Jeremy BurtonPlay from here

[00:19:31] S0: It's a... It's an an entertaining debate. I think I think in the business that we're in observability, when you think about hundreds hundreds of terabytes a day, okay, if you if you if you just wanna point an LLM at your hundreds of terabytes of telemetry and have it go figure it out, like, good luck. It's just not happening. And so what what we do is not not just manage the volume of telemetry and, you know, sort of put it in a in a big dumpster for you to to dive into. We actually do a lot of work on the data to structure it, to organize it, and to relate it. And that context, particularly when you've got hundreds of terabytes of data a day, that context is critical. Whereas I think that the frontier models, clearly they can pick up knowledge around Kubernetes or infrastructure, it's all on the Internet, and those models are trained on the Internet, and so, yeah, it can probably do a decent fist of providing good context around Kubernetes troubleshooting. But if you're a bank, and you have a mobile banking application, okay, the knowledge about how that works is not on the public Internet. The models are not trained on it. Right? We have the data from that... Of those kinds of applications. We understand the environment, and and it is that environment that is the key context for the agent in the troubleshooting flow. And I and I think what's happened sort of thus far, not surprisingly, is, hey. We've, you know, we've got LLMs. We've got agents. Let's point them at this, this, and this, and this, and let... Let's see what they can do. And I think what we're finding is that we can burn a lot of tokens very easily, and we can blow our budget very, very easily. And I do think the thinking has gotta sort of come back or that the pendulum's gotta come back towards the center, which is... By the way, if you if you provide more context to the LLM, you will burn less tokens because you... You're gonna give it more of a clue of what to do next or where to go look next. And so my my view of this is we've got to sort of balance the the CPU burn with the GPU burn. A few queries run on your enterprise data will give you vital context that you can supply to the LLM so that you can burn less tokens over here.

00:21:58 · Jeremy Burton, Dan TurchinPlay from here

And so I feel like, in the enterprise at least, if you want to get the return on your AI investment, you've gotta do some work on your data foundation. Otherwise, you you either are not gonna get the context and and actually not get the right answer, or you're gonna blow a lot of money getting to the right answer. So it's gonna be one of those two things. [00:22:23] S1: It's clear that the Snowflake strategy in a kind of an AI first world is very ambitious, far more than what, you know, most Snowflake watchers have come to expect. Give me the bull case on what AI services am I going to force... Am I going to Snowflake for that I wouldn't traditionally think to go to Snowflake for?

00:22:46 · Jeremy BurtonPlay from here

[00:22:46] S0: Yeah. I mean, I I think fundamentally that the the the data foundation has never been more important, and it's maybe not surprising to hear... If you're in the frontier model business, it's always easy to sort of look look down your nose and say, well, yeah, all of that stuff beneath us is a commodity. I've been doing this a long time. Managing massive volumes of data, particularly critical business data, never has been a commodity, and I don't believe will be a commodity now or anytime in the near future. I think what folks are also realizing is that if if I have 10 source of truth in my enterprise, which almost every enterprise does, an an agent does not reconcile those across those sources of truth. It might average across them, but it it it would it will give you the wrong answer in a confident way. The the the... I think what what enterprise is realizing is, okay. But if I I can reconcile that source of truth, right, and and if I can provide some semantics around that data, then the agent can do it work, and it will be game changing, because what we've always wanted is sort of, I think right back to Microsoft in my formative years in the mid nineties, you want information at your fingertips, and that has been an elusive dream, I think, until now. But you only get enterprise information at your fingertips if you do some legwork on the data foundation. I would also tell you, for the longest time, Snowflake in the early days, was all about just putting the data inside Snowflake and reasoning about data inside Snowflake. I think what is interesting right now is Snowflake quite early jumped on iceberg. I think most large enterprises are gonna look to Iceberg as sort of the OpenTable format. And so what that means is that the catalog, Snowflake, you know, introduced that probably a year or so ago, becomes fundamentally important. Because if you've got a vast quantity of data in Iceberg format, some of it may have been written by Snowflake, some of it may have been written by something else, You you need to know, like, hey. What what... Where is the data that I need to query to to do my thing? And and the catalog, I think, starts to play, you know, quite a critical role. And then I think more broadly, you know, Snowflake is, I think, aggressively adopting coding agents.

00:25:23 · Jeremy Burton, Dan TurchinPlay from here

I mean, Coco, you know, is their coding agent, and you think, well, okay, we're not using it to generate Python code to build applications so much, but if you wanna set up pipelines, data infrastructure required to lay down that data foundation, I think the sense is, it's never been easier to do it. We can take all the legwork out of that, which is quite involved and hairy, and sorta no one values that much the plumbing, but you've gotta do it. It's like, I've got lots of plumbing in my house. I spend a lot of money doing that plumbing, but actually nobody sees it. And and I think that that was the feeling around, you know, any data foundation, which is I gotta do a lot of legwork, and I gotta migrate data. The promise, I think, is that AI and things like Cocoa are just gonna make this very, very easy. And there's quite a massive library of skills that Snowflake has produced around, you know, data engineering and data operations just to make life much easier. And I think that that is now starting to show up in the business. Right? It's much easier and faster to do things. It's much easier and faster to ask questions. That drives the query volume, which is really what drives Snowflake's business. [00:26:47] S1: That's what I thought you would say. I was hoping you were gonna give Coco these examples. So thanks for sharing that. [00:26:51] S0: Yeah.

00:26:54 · Dan TurchinPlay from here

[00:26:54] S1: Today's sponsor is a product that I've used and have loved for well over a decade. I encourage all the companies that we partner with to use it as well. It's Box, the leading content platform for the AI era. It acts as the secure, essential content platform where AI agents not only access unique institutional knowledge, but they can orchestrate end to end workflows across business critical systems, delivering the kind of enterprise AI transformation that you expect. It's the key to unlocking the power of AI. As we all know, LMs are pretty naive. They don't know about the specific content stored in files across your company. And without access to that rich content that lives in product road maps and sales materials, research, HR policies, financial models, and other content, there's no way that they're gonna be able to orchestrate workflows in the way you expect. Box is building the intelligent content management platform for the AI era. It's the secure bridge between enterprise content and the AI that you want to deliver to your business and your customers. Go to box.com/ai to learn more. Again, that's box.com/ai. Now back to this week's amazing episode. I think it's pretty easy to make the conceptual leap from where we are today with AI primarily based on language modeling to kind of the frontier, which is AI entering the physical world. What's the impact on Observe and Snowflake when the consumer of AI, let's say it's a rocket or, I don't know, it could be an HVAC system or, know, it's certainly a nonhuman, and it's generating different telemetry data, and and we're thinking of videos and images and a whole vastly different set of things, artifacts that need to be monitored, stored, analyzed? How does that change your business?