01:33:06 路 John Coogan
[01:33:06] S1: So can you help me understand more about the problems of, like, CPU bottlenecks and then maybe some of the solutions? I'm just thinking about, like, I I I would think bandwidth would be an issue, obviously, CPU load. But is there a world where we get to some sort of convention around maybe it's the robots. Txt or just the user agent and when Google bot shows up or any other AI system, AI agent shows up, you're just delivering a it's almost an MCP server, but you're just delivering something that looks more like a JSON package, something that's a little lighter, little less CPU intensive. Is there a path there to optimization? It feels like we're in the era of, like, squeeze every ounce of performance out of everything. Yeah. But what's actually going to happen here? Where where oh, like, are we just screwed or is there an opportunity?
01:33:57 路 Matthew Prince
[01:33:57] S13: No. I you know, think so. I think two two different different problems. So the first is if you if you're as as as you ask these these different AI systems to perform tasks on your behalf, what what has to happen behind the scenes, especially those tasks get complicated, is that they need to be coordinated in in some ways. If you say plan a vacation for me or something like that, what what goes on behind the scenes is that there there's coordination there, and the best way to be make that as efficient as possible, what what agents are really good and and and and these various LLMs are really good at is actually writing code. And so that code needs somewhere to live. And the problem is that if that lives in a container, then you've got to bring in an operating system. You've got to bring in all all the tooling and everything else. And that's actually extremely heavyweight. And so that's the first place that you've got both a CPU and a GPU bottleneck that's there. And so with with something like Cloudflare Workers and isolates, you it's just a much lighter weight system. And so that means that you can have more agents running on the same CPU infrastructure and and be able to provide that. And, again, I think that's why a lot of these next generation, tools are actually built using our our platform. You mentioned something else, which is, you know, as as these agents go out and interact with the rest of the web, they they you wanna make sure that that is done in the most most efficient way. And so if they're pulling down, you know, all of the HTML from a from a web page, if and and it's and those web pages are designed kind of to be consumed by by humans, there's just a lot of cruft on that that isn't isn't necessarily as important. And so some of the things that we're doing are, you know, for those those customers of ours that wanna make sure their content is consumed by by agents, wanna make sure it's as as accessible as possible, We are automatically converting things into markdown, which is a much simpler Sure. System that that saves you a ton of tokens. It saves you a ton of processing. It means that your context window can be can you can fit more useful information into a into a context window.
01:35:56 路 Matthew Prince, John Coogan, Jordi Hays, Ranjan Rajagopalan, Trent Simonian
So I think there's a ton of optimizations, and we're helping both on the developer side as well as on the content side, making sure that we can have have these agents be as powerful as possible and and get as much done as possible. [01:36:09] S1: John Gruber, total victory. You know he invented markdown? [01:36:12] S0: I did not. Grubernator. [01:36:13] S11: Yeah. No way. [01:36:14] S1: Yeah. Isn't that amazing? [01:36:15] S0: Yeah. Anyway I mean, it's it's one of these [01:36:17] S13: things that just, you know, it it it turned out it was it was ahead of its time, but it's it's such a key for making sure that we can take information and make it, you know, as compact as possible. [01:36:27] S1: John Gruber created the the the format for God. It's a funny funny world we live in. [01:36:34] S0: Talk about it was last week you guys announced that the threshold had been passed around agent versus human traffic. Talk about that moment. Did it happen sooner or later than you expected? [01:36:47] S12: Much sooner. [01:36:48] S0: Much sooner.
01:36:49 路 Matthew Prince
[01:36:49] S13: Yeah. I mean, was I the at end of last year, so end of of twenty twenty five, I said that I I thought that we would pass he it it bought traffic, and and that's that's, you know, across the board. So that's, like, Google's crawler, but also, you know, the new agents which are which are coming out. I thought would would pass human traffic by the end of twenty twenty seven. About three months ago, I revised that based on the traffic that we were seeing to say that it would actually be in the first half of twenty twenty seven. And so the fact that it actually happened in the first half of twenty twenty six is just is just it's just been extraordinary, and it just shows how quickly this this is is growing. And and the and the real key here is that I think that if, you know, you or I as humans were researching to go buy a digital camera or something, we might visit five websites and do, you know, a little bit of research and some price comparison. You know, you just watch as you use these agents. They have boundless attention to be able to just go to maybe 5,000 websites to find exactly what what you're you're looking for, the best price, the best delivery, the best service, and everything that's that's there. And so that's just driving an enormous amount of consumption of of of the Internet. At the same time, the other thing that's happening is that for a long time, since about 2015, the the web has kind of plateaued. There were not there were there were more websites that were being shut down than were being created during that time. In the last eighteen months, though, we're we're back to the the web growing at a rate which is is exponential. And and what and and it looks sort of similar to what what was happening back in the early two thousands in terms of in terms of growth of the web. And so I think you're seeing both more sort of consumption of what's online, but you're also seeing more things online as it as it goes forward. And so in both of those directions, you know, that's that's that's gonna continue to just drive, you know, more and more use of of the of the Internet. And and I wouldn't be surprised if, you know, going going, you know, forward, say, five years, that that bot traffic will be a thousand times human traffic online. And we've and we've gotta make sure that we make the Internet work for for that new future.
01:39:02 路 John Coogan, S10, Matthew Prince
[01:39:02] S1: Yeah. Like every year we're just gonna add another nine to the nine nine 99.999% of Internet traffic is bots. Yep. Do you do you know what the baseline was? Like pre AI, were we at like 1% bot traffic? I mean, I've said No. [01:39:17] S10: Was was more it [01:39:19] S13: was more than that. It was about 20% Okay. For for us for a long time. So it yes. And it's you know, Google was the was the largest. And then and then obviously, there's a lot of of malicious things that that run around online. But that was that was around what what it was for and it was pretty stable over, you know, the history of Cloudflare at least, so where where we could measure it. So so since 2010, you know, it was always kind of around that 20% range. Mhmm. And then and then it's and then it's grown. You know, it's it's in the last few years, it started growing steadily, then it really accelerated in the first half of of twenty twenty six.
01:39:19 路 Matthew Prince, John Coogan
[01:39:19] S13: was more than that. It was about 20% Okay. For for us for a long time. So it yes. And it's you know, Google was the was the largest. And then and then obviously, there's a lot of of malicious things that that run around online. But that was that was around what what it was for and it was pretty stable over, you know, the history of Cloudflare at least, so where where we could measure it. So so since 2010, you know, it was always kind of around that 20% range. Mhmm. And then and then it's and then it's grown. You know, it's it's in the last few years, it started growing steadily, then it really accelerated in the first half of of twenty twenty six. [01:39:53] S1: I wanna talk about the inference stack, I guess. Like, we're seeing two things play out, sort of a bifurcation, like a WWDC, Apple's launching on device inference that's it's not going to be frontier, but it's going to be usable for sure on your phone, on a computer. They can obviously go to the private cloud as well. And then you have like the new NVL 72 models. There's stuff in between, a Cerebras chip, fast but expensive. Then there's everything from, oh, it runs on commodity hardware, but it's still pretty big. You need a real desktop for it. And I'm wondering about how you see Cloudflare fitting into that. It would be a logical extension to Cloudflare workers to have inference on the edge, inference in different places. Like, where do you see yourself offering inference, if at all?