Invest Like the Best with Patrick O'Shaughnessy
00:03:24 · Neil Movva, Patrick O'Shaughnessy
[00:03:24] S2: Sale Research is a token factory. We have an API where anyone can send us requests, where they can use large language models, open source large language models for any task they want. We will serve those tokens to them at a price that is unbeatable in the market. We also support their ability to build agents on top of this. We host what we call Sailboxes, which are long running agent virtual machines hosted in the cloud that are designed for agents that run for hours, days, or weeks. [00:03:48] S0: So I should think about you as a peer company to others that serve different kinds of inference. You're serving one specific kind of inference and your goal is to be the absolute cheapest provider and enabler of a certain kind of use of intelligence. [00:04:01] S2: Exactly. The theme of our company is abundance. We want to deliver this new commodity of intelligence to as many people as possible at a cost that is sustainable for almost every industry. We think that whenever you make something 10 times cheaper, it's a new product category. We aspire to do that for tokens. We think it's so profound that the machine can think, and now our job is to make as many machines as possible in the world work towards thinking. [00:04:22] S0: So if you think about the theme of the day being token costs, is token cost the right way
00:04:22 · Patrick O'Shaughnessy, Neil Movva
[00:04:22] S0: So if you think about the theme of the day being token costs, is token cost the right way [00:04:27] S2: to think about this? Is there some other way you'd put it? To start with, absolutely, token cost. Today, my North Star is I wanna have the lowest cost per token in the industry and do that by a mile. I don't think tokens are the final unit of work or intelligence, but they are what we use today. After tokens, you start to move more towards more outcomes, which is like a vague direction. You can imagine, for example, today when you consume tokens through an agent, you don't actually control how many tokens the agent reasons for. It can reason for a certain amount of time or it can call a certain number of tools. And increasingly, think we will have agents do some unit of work, take as many shots on goal as they can. And however many tokens they used to get there is going to be a dependent variable depending on the task. So you think about like agents that self administer a token budget as opposed to a company setting a budget for how many tokens engineers can spend per month. [00:05:15] S0: Why is there an opportunity that you can tackle? It seems like the entire world is oriented around more, better, faster, cheaper tokens right now. It seems like the world is trying to solve this problem very aggressively. What was the unique opening that you saw that's maybe the market's not being efficient in its attempt to tackle this?
00:05:33 · Neil Movva, Patrick O'Shaughnessy
[00:05:33] S2: So I think there's two things that are tailwinds for our company. One is gotta be the rise of open source. I have to tackle that first. I think we're starting to see an increasing number of our customers and the broader market care about owning intelligence. They want to have control sovereignty over the thing that they depend on. That created a much more robust market for our customized models or even just like these vanilla open source models that no one can ever take away from you. You always have the weights, you always have the right to deploy them however you like. In that world, there's been a reasonably robust market for the past couple years serving these models at large scale. The challenge is, all those companies, you could take your pick, base 10, fireworks together, they all focus on low latency inference. And they were pulled in that direction by one very important customer, Cursor. I think that that was the right choice about a year ago. And as of six months ago, it started to look like maybe low latency wasn't the only thing you wanted from an agent. You wanted more persistence, more long horizon tasks. And now it's, to me, very obvious that the future of agentic inference is long horizon tasks. You're gonna run the machine for hours or days at a time. It doesn't matter if it's been set tokens at a 100 tokens per second. Maybe 10 is just fine. That comes with corresponding advantages and efficiency. [00:06:44] S0: Why are you so confident in that? To me, it seems like I want everything as fast as possible.
00:06:44 · Patrick O'Shaughnessy, Neil Movva
[00:06:44] S0: Why are you so confident in that? To me, it seems like I want everything as fast as possible. [00:06:49] S2: When you're waiting on it, you absolutely deserve the fastest answer possible. Yeah. My trick is I don't want you to be waiting on it. I want it to be proactive. I want it be in the background. One way to say it is like the best latency is no latency at all. When you wake up in the morning, the work's already been done overnight. You didn't even have to ask for it. That's the dream. We're not quite there yet. More importantly, I think the more you're in the loop as you prompt agents and wait for a response, in fact, you're the bottleneck in having agent do more or less work. What we'd like is the agent to operate on more human time skills. You don't manage your colleagues every five minutes. You ask them to do a high level task and you come back and maybe every day but more likely once a week. And that to me is the future of human agent collaboration, more like human time skills. [00:07:28] S0: Say more about the early indications that this is happening and therefore you should be building this company.
00:07:28 · Patrick O'Shaughnessy, Neil Movva
[00:07:28] S0: Say more about the early indications that this is happening and therefore you should be building this company. [00:07:33] S2: So the first and most important thing is the idea of test time compute scaling. The idea that you can give an agent more time and it will give you a better answer. So that was theorized about two years ago now. But it wasn't really something that we could actually bet on until, I would say, late last year with OPUS 4.5. OPUS 4.5 was the first agent that was at all suitable for longer horizon tasks. It was pretty mediocre when it first came out, but you look at the more recent models and what we've done on open source as well. And you see that agents are capable of running for an hour at a time. I wouldn't say it's days, but definitely an hour is quite suitable today. Seeing that, like, average task length get longer and longer doesn't take many points to have you draw out the exponential and see that agents are worth running for longer periods. [00:08:14] S0: What do you think will be the market share of long running agents in three years or something like this?
00:08:14 · Patrick O'Shaughnessy, Neil Movva
[00:08:14] S0: What do you think will be the market share of long running agents in three years or something like this? [00:08:19] S2: I love this market because it's unbounded. There's no human in the loop, so you can consume as many tokens as you like in the background. Versus human attention span, if you tell me to consume 10 x as many tokens at codex or at cloud code, I'm actually not sure if I can anymore. I'm already in the loop and locked in coding for most of the day that I'm at the laptop. What is undoubted is how many tokens can be consumed in the background or proactively. Long term, I think, you know, we're gonna end this year at maybe fifty fifty background and real time workloads. But I see this going to ninety ten in favor of background. What are [00:08:49] S0: your favorite examples of something that gets accomplished much better as a background task than as a human in the loop task? Most deep research.