00:19:57 · Parag Agrawal, Molly O'Shea
[00:19:57] S2: I was also an engineer before that. [00:19:59] S1: You were also an engineer before that. You've done the whole thing. You also have a PhD. I've done all this research, but I want to talk about that. So like, can you set expectations now? Like, what would scale meaningfully look like today? If like, back then it was a triple triple double double. Now it's like, what do you have to achieve? [00:20:18] S2: I don't know how somebody else could do it. For our business, turns out the answer is relatively easy. As I've said, we just build alongside inference. So if you think about inference growing year on year, to us that is the anchor. Like our right goal is to grow faster than inference for agents for knowledge work or consumer agents grow. So if we grow alongside of faster, I'm feeling pretty good. Right? And that's to us the benchmark. Now, what is the right inference benchmark? That's a question. I don't know long term the answer to, but I think in the short term, it's clear that GPUs that are coming online this year or next year are going to do inference. And so then they're going to do search alongside it. So the short term there is very clear, but our benchmark is just growth of infants. [00:21:21] S1: So what is that numbers?
00:21:21 · Molly O'Shea, Parag Agrawal
[00:21:21] S1: So what is that numbers? [00:21:22] S2: I think the easiest approximation is 10x. You have to think about like a 10x goal as the very simple round number. If you think about today, like, if you'd asked me last year, I would have probably said lower number. But the pattern here is, even when I have thought I was being provocative in making up a high number, I have somehow managed to underestimate. So now I feel like I can go to 10x, feel a little bit uneasy, it feels crazy to grow 10x for too many years. You know, how long can that happen? Maybe I'm wrong again. [00:21:59] S1: I'm curious because you get to see so many different market insights from internally, externally, all around. What are what are some surprises that you've come across, and what are the signals you're looking for?
00:22:15 · Parag Agrawal
[00:22:15] S2: So I think it might be useful to tell people about the kinds of customers we end up working with. Okay. All of our insights, or at least my insights, come from working with people doing really, really interesting and unique things. We started working in the beginning with all other people pushing the limits of building agents. So some for customers like Harvey Notion, they were one of the early people building agents for doing real work, where the quality bar for that work was quite high and everyone in the adoption chain was like, oh, can we trust this agent to do our work? That was the question being asked. And we, by working with like really demanding customers or customers who had really demanding customers in turn, We ended up really pushing our systems capabilities on the quality dimension. The next phase, I would say is like voice agents are starting to work And the interesting thing there is, initially there was some acceptance for your voice agent pausing a little bit unnaturally or being like, let me look that up and come back to you. Like that was acceptable for like a three month period, I think. And then very quickly today, if that happens to you, at least the people who've interacted with multiple voice agents, they'll like, that is ridiculous. Like that's a really bad product. And so now you're seeing all these voice agents, are like supernatural, but also super fast. So what does that mean for web search? You gotta be able to search the web in like one hundred and fifty or two hundred milliseconds. Human web search was like a sub second product, like six hundred, seven hundred millisecond product, roughly end to end. Because we don't notice it getting any faster. Right? For voice agent, that latency constraint on web search is unacceptable. So now you have to push yourself on the quality dimension and the cost dimension and the latency dimension, and you're to crush all three. And that's been something like we get pushed by sort of the frontier customer who is like, listen, the product experience that we can build today is going to be unacceptable very, very quickly. So what does every underlying infra piece need to solve in order to create the market expectation, which changes so fast. Right? Like I think, I know going back to web search, one of my...
00:25:07 · Parag Agrawal, Molly O'Shea
You should at some point try, even with the new model, turning off web search in ChadGPT once. Okay. I think we all used ChadGPT for the first, eight months, maybe nine months. It did not have web search and we were just talking to a model. Try to do that today and see what happens. My bet is one. 90% of things you ask, it will disappoint you. Two. You will feel like a caveman or a caveman in using that product. So it's only three years, about three and a half, four years ago now. But our implicit expectations of a product that looks the same have changed so dramatically. And that is what sort of mind boggling to me. And I think that is essentially what drives all technical progress. Yeah. Is because it becomes table tics. [00:26:02] S1: Right. And now they might be getting subconscious, you know, they might have some consciousness to them, and start judging us, and attacking us. [00:26:14] S2: I actually, I am more of an optimist on this. I think they will actually get proactive. So, they will know us. They will know our explicit and implicit needs and they will try to proactively do work. That's the phase. The next phase, I think you're trying to, you're starting to see glimmers of proactivity in instinct and muse and dot. Glimmers. I think they're still more reactive than proactive. They're always on, but they're not really proactive. I think the next era they will be hyper proactive, which I'm really excited about.
00:26:14 · Parag Agrawal, Molly O'Shea
[00:26:14] S2: I actually, I am more of an optimist on this. I think they will actually get proactive. So, they will know us. They will know our explicit and implicit needs and they will try to proactively do work. That's the phase. The next phase, I think you're trying to, you're starting to see glimmers of proactivity in instinct and muse and dot. Glimmers. I think they're still more reactive than proactive. They're always on, but they're not really proactive. I think the next era they will be hyper proactive, which I'm really excited about. [00:26:57] S1: I guess, to peel back because this is a brand new economy and layer of the internet and what could, I don't know, the internet might not exist anymore. Maybe we'll just use agents. This is what people are saying. What are those components of the underlying infrastructure? You're one of them. What are like, what makes up that base?
00:27:18 · Parag Agrawal
[00:27:18] S2: There are two ways to talk about this. I think before going into the internet, let's first talk about the underlying infra components for building an agent. You need a model, which is inference. You need perhaps a sandbox. I think you need web search like we provide. You need an action space, which includes a browser and a few other API hooks. And you need access to a bunch of connectors to access your personal data or your company's data. You build these together and you can build an agent. And then the rest of it is all of the same stuff we've always had, which is observability, feedback loops, security. And then you have a cloud for agents. Right? Now you go into the web piece of the infra layer. You now need to do web search, but also you need to figure out to your point around the internet shutting down. I do think the primary consumer of the internet is going to be an agent, which means the internet will get re architected. The first thing that's happening is agents are reading the web, which is what we do. What we haven't figured out is, what's the new business model? Like previously, if you put your content out there on the web, you were either building a subscription business, which is like you put some content out there, a bunch of users come, some fraction of them subscribe for a monthly subscription to a newspaper. Other internet businesses were monetized by saying, I'm going to put ads and I will get paid based on human visitors seeing ads. Other business models were like either affiliate business models or you drive a transaction of some sort. All of these business models are fundamentally relied on us random walking the web as humans, opening pages and taking the next action after the next action on it. Some of which are monetizable actions for these participants. If agents are doing this, the buying action still remains. Every other action breaks. So you can't monetize attention anymore. So ads business model per seat subscription business model, these things are not durable. So what replaces it? That's part of what gets me super, super excited about building parallel with like, I think solving just technology problems, I think is far less interesting than solving a technology and a business model problem.