Radical Talks
00:03:51 · Aidan Gomez
[00:03:51] S2: really unique period because we had gone from a world where, machines that could write compelling language were totally out of reach. Like, we could barely get them to spell a sentence properly and, certainly none of it made any sense to almost overnight one where there was pretty much entirely plausible human text that appeared in front of us. And I still remember getting this email from Łukasz Kaiser, who was my manager at Google. The subject was like, Aidan, look at this. And the body was a Wikipedia page. By the way, sorry. I'm I'm sick right now. So if I'm clearing my throat or coughing, I I apologize. Yeah. The body of the email was, a Wikipedia page, and the idol of the Wikipedia article was the Transformer. And then it talked about this, like, Japanese punk rock band that had gotten together, done a few albums, broke up. The basses got replaced. We got back together, did a tour. And then at the bottom of this email after this Wikipedia article on this Japanese punk rock band called The Transformer, Lukash had written, I just wrote The Transformer, and the machine wrote the rest. And that was a really jaw dropping moment. Like, now it's so quaint. Right? Like, of course, you know, you could open up one of 20 different apps and prompt that and, like, obviously, it does... And it will do it better. Right? But back then, it was like getting hit in the face with, something that was out of your dreams. Right? Something that was out of, like, a movie. Like, you just... The night before, you couldn't even by the end of my career, if we had machines that could do that, it would have been an extraordinarily productive life in machine learning. You know, what a contribution to humanity. And it just showed up in front of us. And so it was obvious that this was gonna be impactful, like machines had sort of woken up and could talk back to us. And so that opened up a totally new interface to interacting with machinery, computers. I didn't know how to make money off of that. I didn't know how to build a business around that, but I had a pretty good idea of how to do the tech, how to do the modeling. Right? Like, I knew how these models were built. The trend that seemed to be working was scaling them up, and that's what I was doing my PhD on. And so then when I went to go... For my birthday, I came back home to Toronto.
00:06:38 · Aidan Gomez, Jordan Jacobs
I was, like, you know, hanging out with friends. I was mostly there to party. But then I was gonna give a a talk at TD in layer six because that was, like, one of Toronto's primary ML groups, that was in industry. And I wanted to give a talk on my research to see if they were interested in it so that they might adopt some of this, like, efficient scaling, of neural net techniques that I I was developing. Presented that at layer six, met Tommy. Tommy told me, you know you know who you have to speak to, Jordan. And the next morning, Jordan and I met. And Jordan was like, do it. Let's figure it out. And then later that day, I went to hang out with Nick and Ivan, who are now my cofounders. And I said, like, I had this crazy meeting with this this guy, you know, Jordan Jacobs. Like, he he had... He he thinks this might work. He thinks this technology might work. Should we do it? And, you know, it took some convincing. It took a few months of work, I think, both on my part and and Jordan's part. But, eventually, we were like, alright. Let's quit Google and and do this. [00:07:56] S1: And so... Yeah. And and just to put it in reference, like, we're talking about, you know, almost eight years ago, more than seven Yes. Seven and a half years ago. [00:08:10] S2: At this point, it would be, like, seven and a half. Yeah.
00:08:10 · Aidan Gomez, Jordan Jacobs
[00:08:10] S2: At this point, it would be, like, seven and a half. Yeah. [00:08:13] S1: And so... And just so people understand the context, what's... That's, like, 3.5 before ChatGPT. So it is early. Right? It is... There's there's no such thing as anthropic. There is OpenAI, but they've not put a product out that anyone's really using at scale. [00:08:36] S2: They were doing the robotic hand solving a Rubik's cube, and they were doing Dota. Those were, like, their two their two bets. [00:08:43] S1: Exactly. The robotic hand became covariance, which we invested in, the robotics company, separately. Okay, so you guys decide to leave to do it. And it's true, Aidan was like all in early, and there required some convincing for I would say Nick was the most reticent. Nick's like, I have the most incredible job in the world. I worked for Jeff Hinton at Google Brain. And you know, what could be more amazing than this? So I remember some of those conversations. It was it was fun to try and convince him there was even brighter things ahead and bigger things ahead. In those conversations, there's a couple of things that were considerations that were you made decisions that I think were very consequential. One was we're going to focus on enterprise, not on consumer. And and the second was we're going to do this in Toronto. When you had been working in Mountain View, there's a possibility of doing it in London. Talk us through the decision of those two things. Why focus on enterprise and why do it in Toronto?
00:09:56 · Aidan Gomez
[00:09:56] S2: So enterprise, that was more because we didn't know the product to build. We started, like, well, like like Jordan just said, well before there was any product market fit. No one was buying large language models. No one was really buying models from a third party. They would build stuff internally inside their company, and it might be, like, re rankers for social media, feeds, or, like, yeah, like, recommender systems, basically. That was, like, the predominant actual industry deployment of these models, or Google would build models for itself for translate, that type of thing. And so there was no market. There's just like a totally... And I remember some of these, like, when we were starting to build this, like, how do you price it? How do you charge for this thing? And, like, you know, the unit of compute is a token, but no one's gonna understand what a token is. Should we charge based on characters? Should we charge, you know, like, or something? Like, have some completely different pricing model. It... There was no market. And so the initial bet was we don't know what product to build, and so we're gonna build a platform so that when someone figures out the right product to build, we'll be the place that they go. And that pointed us towards companies, right, and serving b two b. It was... Yeah. I I think it was, like, the only... I think if we had have gone very deep into consumer and we had made some, like, very hardcore product bet at that stage of the technology, we wouldn't be around right now. We we wouldn't be having this conversation. I I don't know if that's transferable advice. You know? I think a lot of people build on spaces where actually you should not build the most general thing, and that's not the fastest path to revenue, and that's not the best strategy. But if you're building on top of, like, an extremely new technology that has no users and no market, that's probably kind of what you're forced to do. And if you bet the whole farm on one specific product, you're you're gonna die with high likelihood. Maybe you'll, you know, get lucky. On being, like, very global from the outset, I think that was inspired by advice from you again, Jordan, which we were like, you know, the default advice, go to the Valley. Like, you're building a tech startup, do it in Silicon Valley.
00:12:45 · Aidan Gomez
And the argument Jordan made was, in Toronto, you're well networked, and you will have a monopoly on talent. Like, you'll you'll be the only game in town, and people wanna stay where they live. They don't wanna have to uproot their lives and move. And so, you will be very unique in that regard. So don't just start the company in the Bay Area. Start the company in the Bay Area, but also have an office in Toronto. And then that... Well, we founded, and then COVID happened. And then we were just like, okay. Fuck it. We might as well go all in, and we'll just hire wherever the talent is. So we hired a guy in Singapore. We hired a guy in, you know, London. We hired all over the world. And that led to a very global orientation for the company. Like, we were really good at multilingual and still hold, like, state of the art in in translation, very, very early. And so it paid dividends to the company at the technology level as well, not just the the talent level. There were real costs. Like, I hate remote working, And I have, like, the most remote company, and my direct leadership team is, like, in Miami, SF, Montreal, Toronto, London. I think that might be it. But, like, that's, like, just my reports are all over the place. And it's super, super hard. If I could, like, magically wave a wand and put us all in one city in one room, I could totally do that. It would be very good for productivity, potentially bad for our strategy, which has sort of fallen out of that global footprint and is serving the world. And as a product of us having so many investments in so many different places, we can... We're sort of locals everywhere. Like, we're... I'm in London. My cofounders are in Toronto. You know, we have a HQ now in Berlin, offices in Seoul, Tokyo, Riyadh, like, all all over the place. We're sort of, like, local to the world. And for Sovereign AI, that's an extremely good property to have. It's very, very salient because they see... The country sees your... And the local players see your commitment to their ecosystem, their talent, and you're not showing up there just to extract money from them. You've been there for years without asking anything from them. And so I think it's a it's a high trust position, but it's a huge cost, huge price to pay.
00:15:31 · Jordan Jacobs
[00:15:31] S1: Let's talk about Sovereign AI a little bit. I mean, this was something that, it came up very early in the discussions about Cohere. But again, there was no market, and I don't think that anyone really understood what we were talking about when we talked about sovereign AI at that point. Now everybody does. Can you describe what it means? Like, when people talk about Sovereign AI, what are they talking about? How have you guys leaned into it, and what has the market opportunity become recently?