AI and the Future of Work: Artificial Intelligence in the Workplace, Business, Ethics, HR, and IT for AI Enthusiasts, Leaders and Academics
00:25:23 · Jeremy Burton, Dan Turchin
I mean, Coco, you know, is their coding agent, and you think, well, okay, we're not using it to generate Python code to build applications so much, but if you wanna set up pipelines, data infrastructure required to lay down that data foundation, I think the sense is, it's never been easier to do it. We can take all the legwork out of that, which is quite involved and hairy, and sorta no one values that much the plumbing, but you've gotta do it. It's like, I've got lots of plumbing in my house. I spend a lot of money doing that plumbing, but actually nobody sees it. And and I think that that was the feeling around, you know, any data foundation, which is I gotta do a lot of legwork, and I gotta migrate data. The promise, I think, is that AI and things like Cocoa are just gonna make this very, very easy. And there's quite a massive library of skills that Snowflake has produced around, you know, data engineering and data operations just to make life much easier. And I think that that is now starting to show up in the business. Right? It's much easier and faster to do things. It's much easier and faster to ask questions. That drives the query volume, which is really what drives Snowflake's business. [00:26:47] S1: That's what I thought you would say. I was hoping you were gonna give Coco these examples. So thanks for sharing that. [00:26:51] S0: Yeah.
00:26:54 · Dan Turchin
[00:26:54] S1: Today's sponsor is a product that I've used and have loved for well over a decade. I encourage all the companies that we partner with to use it as well. It's Box, the leading content platform for the AI era. It acts as the secure, essential content platform where AI agents not only access unique institutional knowledge, but they can orchestrate end to end workflows across business critical systems, delivering the kind of enterprise AI transformation that you expect. It's the key to unlocking the power of AI. As we all know, LMs are pretty naive. They don't know about the specific content stored in files across your company. And without access to that rich content that lives in product road maps and sales materials, research, HR policies, financial models, and other content, there's no way that they're gonna be able to orchestrate workflows in the way you expect. Box is building the intelligent content management platform for the AI era. It's the secure bridge between enterprise content and the AI that you want to deliver to your business and your customers. Go to box.com/ai to learn more. Again, that's box.com/ai. Now back to this week's amazing episode. I think it's pretty easy to make the conceptual leap from where we are today with AI primarily based on language modeling to kind of the frontier, which is AI entering the physical world. What's the impact on Observe and Snowflake when the consumer of AI, let's say it's a rocket or, I don't know, it could be an HVAC system or, know, it's certainly a nonhuman, and it's generating different telemetry data, and and we're thinking of videos and images and a whole vastly different set of things, artifacts that need to be monitored, stored, analyzed? How does that change your business?
00:28:51 · Jeremy Burton
[00:28:51] S0: Yeah. I think, obviously, the volume goes up. Right? And and and I think when... With the advent of Kubernetes probably, you know, ten, fifteen years ago, is it is it really that long? You you saw probably a three to five x growth in telemetry volumes. I I firmly believe you're see that again in the AI era. I I think then the the volumes of data then are are obviously so vast. To to ask a poor old human to help you figure that all out and find root cause when the rocket explodes, there's a problem, it's like, no, you're gonna need AI to help troubleshoot the data that has been generated quite often by AI. So needless to say, we think that this process of structuring and organizing messy machine generated data, AI can help with that. If I go back, observe one of the unique innovations that we produced was something called a knowledge graph, and we did this back in 2018. And the knowledge graph really is just an abstraction that sits on top of this machine generated mess. But it but it... We originally designed it to enable humans to navigate that data much more quickly. Instead of diving into a dumpster of logs and looking for a customer ID, you could say, oh, look, here's all my customers. Here's the sessions. Here's the pods they were running on. Here's the containers. So it turns out that we didn't know it at the time. It turns out that we were actually total geniuses, because it is that structuring and that knowledge graph that we use to give context to the agent. So, wind the clock forward now, okay, yes, agents can use that knowledge graph, but what if we could use agent to build that knowledge graph? Today, when we build it, you know, if if if you had an application, we would sit down with you and say, oh, and and you're... When you're troubleshooting, what kind of questions do you ask? Because every business is a little bit different. You know, our customer Dialpad looks at Kohl's. Topgolf looks at Gulf Bays. These are all different entities that businesses wanna ask questions around, and we would have our implementation guy sit down with the SRE team and ask these kind of questions, and then we would extend this knowledge graph to better reflect their business. But you start to think, okay, what is the art of the possible now? We could just ask those questions. We could analyze the query results.
00:31:30 · Jeremy Burton
The agent could suggest structures or entities extracted from that data, and then as we further ramp queries, it could optimize the query performance. So I think AI agents will essentially end up creating that graph. That graph then becomes the basis for other agents to drive the investigation flow. So, you know, you can start to see, yes, there's more telemetry flowing in the bottom. The good news is we don't have humans creating entities and structure, and we have agents doing that. Oh, and by the way, that structure they created is going to help other agents advance the troubleshooting flow more quickly. So, yes, there's going to be humans involved. There's there's unknown problems you've never seen before that, you know, will need some, you know, sort of human insight. But I think the goal here is that 90% plus of the legwork here is is really done by a collection of AI agents.
00:32:31 · Dan Turchin
[00:32:31] S1: Makes sense. In a pre AI world, as much as it sounds like an insult, a lot of observability could be performed using heuristics. Back in the day, we didn't think of that as an insult. That was just, you know, it was a nice reliable way to, observe applications. Now as we introduce AI, kind of these probabilistic or nondeterministic outcomes, there's a different level of responsibility. It's it's like... Observe such Snowflake as a vendor. It's less about just verifying the integrity of the data or, you know, its state and more about taking responsibility for some of these actions, the decisions that are made. How do you think about the shifting role of what you do with respect to these, you know, probabilistic outcomes?
00:33:20 · Jeremy Burton
[00:33:20] S0: No. It's a it's a great question. I think for the last decade, the sort of unit of currency, the thing that we've been trying to troubleshoot really is a microservice. And a microservice is a lump of code that typically talks to another microservice that talks to a database. An agent I know we profess that this is something wildly different, but what's an agent? Well, it's kind of a lump of code that will talk to maybe another agent, it'll talk to a database, but it'll also talk to an LLM. And so that changes the picture quite a lot, because I always describe an LLM as like a new animal in the zoo. We know databases. We've had them for years. We know how to monitor them, we know how to investigate them. It's a well known problem domain. LLMs, we don't know. There isn't the level of observability, if you like, into the LLM that we would get from, say, like a database. Also, I think you touched on that a second ago a database gives you sort of empirical facts back. It's a deterministic system. You will get the same answer. Given the same question, you'll get the same answer every day. That is not true of an LLM. And so I think what becomes more and more critical as we try and troubleshoot these agentic systems, is a way to evaluate the LLM. There's gonna be certain bounds within which an answer is acceptable, and we have to constantly run these evals against those LLMs to see if the answer is still within the bounds of acceptability. And if it's not, that... Like, why is that? Like, you know, something obviously has changed. It could be something, you know, with the data source. It could be something with the way it's been prompted. You know, there's many possible reasons, but I think what what we would call LLM observability, in particular, evals, are just gonna be a critical part of, I guess, now we'd call it the the harness for the LLM. You know, what what what... What's one of the essential things that you need in order to run an LLM? It's gonna be in order to determine how the responses are changing over time. And by the way, that that could be down to a, like, a new a new model been released. It's kind of topical given Anthropix new release this week. That release changed dramatically, probably, the function of a lot of existing applications. And I think this is one of the big challenges with AI right now.