The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
01:00:10 · Harry Stebbings, Thomas Sohmers
[01:00:10] S1: Speaking of room for improvement, can I ask you, when I was doing the research for the show, I saw that the Silicon Data Token Price Index dropped below $1 per million tokens this month, and five years ago, it was $60 per million? $60 to one. What does a million tokens cost in 2028, say, two years from now, do you think? [01:00:33] S0: I think the thing I care a lot more about than just that 60 to one is the fact that a $60 token five years ago, no one would pay a cent for today. Like, that was a complete garbage token, relatively speaking, five years ago. And the level of quality of... For a token that you pay a dollar per million tokens for now is, so much astronomically more valuable. [01:00:56] S1: And so that's because of token efficiency and what can be done?
01:00:56 · Harry Stebbings, Thomas Sohmers
[01:00:56] S1: And so that's because of token efficiency and what can be done? [01:01:00] S0: No. I'm saying just in model capabilities. If you if you say, k. So so it's 2026. So the best model in the world in 2021 was GPT three. It's kinda crazy at the rate models get released today that GPT three was the best in the world, basically, from 2020. I think it was August 2020 when it released all the way up to... They they didn't have a new release until ChatGPT in November 2022. So it was, you know, two, two and a half years between between model releases. And, really, GPT 3.5 was just doing reinforcement learning with human feedback on the same base model. So if you remember how bad a GPT 3.5 was and, like, what was the value of that in terms of economic productivity value of GPT 3.5 versus GPT six today or, you know, pick whatever comparison points you want. The value per token in terms of what it can improve a person's life, you know, a company's, you know, business practices, etcetera, is orders of magnitude. I would say a 100 or a thousand fold. I I think there's actually two points to to your your access of going from $60 to $1. Yes. That's a decrease in cost. But that token today is, let's just say, I think, conservatively, a 100 times more valuable. So I would actually be saying that there needs to be some multiplier there as well where, like, the value per, you know, unit of intelligence is probably closer to a thousand fold, not not just the 60 fold you're you're talking about.
01:02:27 · Harry Stebbings
[01:02:27] S1: What does that mean then if we extrapolate that out to 2028? What does that mean that... Like, does the cost of a token then actually matter? Like, is that the primary unit that we should measure? Everyone talks about cost of token. Is there actually a different metric that we should measure?
01:02:40 · Thomas Sohmers
[01:02:40] S0: It is interesting that with the GPT six launch, Greg Brockman had said that he doesn't think that they're going to be pricing things in tokens much longer and that they they want to be moving into, you know, cost per useful result, like, to that. And I don't think that's where it'll end up because that's really difficult to price and, you know, qualia, etcetera. But I think that the price per token is really great because you can easily calculate the cost to generate a token. So determining a margin on that and pricing it in in bulk volume to generic customers is really easy. And I think that's going to stick around in large form because that is so easy. We'll see for the largest providers of tokens how they potentially evolve their business models in terms of if you have a GPT seven or eight that is superhuman and and can fully function as an employee in an amazing capacity. Capacity, and OpenAI calculates through whatever method that's running at full tilt, etcetera, it's only going to cost them, you know, however many hundreds of thousands dollars to produce tokens continuously with that, they may decide that it's actually easier and they'll be able to get more adoption if they just charged a million dollars a year just, you know, using a random number to have full unlimited usage of that virtual agent worker. So that may be how things evolve.
01:03:58 · Harry Stebbings, Thomas Sohmers
[01:03:58] S1: Can I ask you? I... I'm always very careful of being, like, the young naive one. I'm not that young anymore, but, like, being the naive one who's not seen cycles. But Gavin Baker says it well when he says, I can't speak to a company that don't have numbers that are parabolically up into the right and just everything is better than it's ever been. What would be the first signs of a crack in the chasm by a shift from frontier models to open weight models? Anthropic and OpenAI are not continuing in the same level, not quite growth rate because it's impossible to say, but level of growth missing numbers next year and then the bubble getting burst a little bit whether the two core leaders are having some form of strife. [01:04:40] S0: I I agree that that's a possibility. The reason I don't think it's likely is I think that development of open source models and and things happening locally, etcetera, will actually drive greater token volumes for for the big guys. [01:04:53] S1: I'm sorry. How does that work? I thought they were competitive.
01:04:53 · Harry Stebbings, Thomas Sohmers
[01:04:53] S1: I'm sorry. How does that work? I thought they were competitive. [01:04:56] S0: Yeah. No. I I think that the smarter and more capable that Siri is on my phone is going to result on in it doing a whole bunch of background tasks and things that remove me from having to be the one that instigates having requests and and data be processed by even smarter models. I I really do think that in a lot of AI applications right now, the bottleneck is actually a human making some sort of decision and different tasks have different levels of autonomy that that will result in things getting sent to be processed by by a model by OpenAI or Anthropic. But I think the the next really big order of magnitude... Couple orders of magnitude increase in token volumes is going to come when us humans trust a local LLM that has access to all of our data all the time to have it decide to do things on its own that it is not smart enough to do. Right now, I trust Astra a lot more than myself on a whole lot of different things, but I still prompt it to do things, and maybe it'll go run autonomously for twelve hours or three days. I think the next big leap is when I trust a model running on my laptop or on my phone to prompt the smarter models to do even more wider set of tasks.