Venture with Grace
00:00:02 · Grace Gong, Ori Goshen
[00:00:02] S0: We're good. Okay. Hi. We're live. Hi, Ori. Welcome to Venture with Grace. [00:00:08] S1: Hi. Great to be here. Thank you for having me. [00:00:10] S0: I'm super excited about our conversation, and I wanna give a quick shout out to for having us at your beautiful office. And also Ori was talking about the book Mike wrote, it was on the wall. So Mike is a great investor who invests in a lot of iconic companies. And Mike and Ann were actually, fun fact, were both in my book. And this is maybe nine years ago. So it's very crazy to be sitting in the office. And also shout out to my mentor, Eris, for making this happen. So I want to start with your journey. So you were originally at 8200, which is obviously the most iconic place from Israel that produced the most amount of talent in whether it's AI and in tech in general. So do you feel like that in that era, it kind of like shaped your thinking as like an entrepreneur?
00:01:01 · Ori Goshen
[00:01:01] S1: Yeah, absolutely. I mean, going into 8080 to 100 is a really life shaping experience. And I'll try to describe what it feels like. Generally, you're coming in in a very young age, about 18, you're getting a very short amount of training for just a few months or so, and then suddenly you get a huge portion of responsibility. You're You're suddenly facing with challenges you've never faced before, and at a very rapid pace, which, by the way, looks, sounds very familiar and similar to what, you know, the entire AI ecosystem is experiencing right now, right? It's like very dynamic. There's a lot of uncertainty, but there's also a lot of responsibility with these very powerful tools. So I think there's some analogy to these two experiences, but more broadly speaking, I think you're... You just need to figure things out on your own. There's a lot of, as I said, uncertainty. There's a lot of things that can be researched and can find creative solutions. I mean, creative solutions is actually kind of almost like an inherent part of your job. You have to figure out how to solve things that seems impossible. So I think that's actually where I started my career. I started programming when I was like five, six year old, but professionally, started in 8200. And that was a very life shaping experience that really kind of amplified and kind of made me prepared for my entrepreneurial journey.
00:03:06 · Grace Gong, Ori Goshen
[00:03:06] S0: For sure. I feel like one of the things that like, I've seen from you building AI21 Labs is like, you you have been very early in the AI trends, right? And then before that you were a founder yourself, and maybe we could talk about like, how did you kind of like essentially started the company? And obviously you guys are building like the user based, like AI labs. And obviously at the beginning, you guys are kind of like technically still, like, kind of like in the same range of like, you know, the the open eye and then flower pick kind of like arena. So maybe we could talk about, you know, what do you see as an opportunity back then? And then how did it evolve? How did the company evolve in the past couple of years? [00:03:49] S1: Yeah. And as you said, we started very early in the super cycle. So the company started back in end of twenty seventeen. [00:03:58] S0: That is like the super cycle. That's like before crypto. [00:04:01] S1: Yeah. [00:04:01] S0: Pre crypto. And that was like the first AI era. [00:04:04] S1: Yeah, exactly. It was actually the same month the famous paper, Attention is all you need, was published. I can't [00:04:12] S0: believe that was like ten years ago. Yeah.
00:04:14 · Ori Goshen
[00:04:14] S1: Time time moves. Five. It it seems like fifty years in AI. Yeah. But... So we we started the company, Yoav Shoham and myself. So Yoav was a professor at Stanford for many years, and he came back to Israel. That's how we kind of got together and started this AI lab company, which back then was really out of the ordinary. So back then, if you had an idea in AI or you want to do some research, the most natural place to do it is in academia or in Google in some shape and form, or there was like Facebook AI research. There were very, very few labs that you could actually do AI research. So, but the kind of the original idea of the company was first to establish an AI lab in Israel. And second, we really had this thesis that deep learning is necessary but not sufficient. And, you know, if you think about it, up until now, learning is kind of the dominant method in AI, and it's pretty amazing what we've been able to achieve with the scaling laws, right? So our thesis was that this, you know, deep learning statistical method needs to be complemented by more symbolic approach, some more deterministic code and so on and so forth. I think actually what we're seeing right now in terms of, you know, coding agents and building all these verifiable, more deterministic code that complements the statistical inferences, I think that's kind of our was our original vision when we started back in 2017. So those were the two core concepts based which in which we created a company. And then in the first three, four years, we were operating as a pure AI research lab. So we hired, you know, the the best engineers and AI researchers in in in Israel. So we actually were fortunate and very delighted to hire these extreme talented people early on and create various kind of the probably the highest density of AI talent in Israel back in the days. So, and back then we were mostly researching the natural language field. We were building all types of models. There were BERT type models, if you remember back in the days, and there were the initial versions of autoregressive models, what we know today as GPT. So we were experimenting with various variations and publishing papers and so on. But back in 2020, we realized, well, this actually is starting to become a useful technology.
00:07:37 · Ori Goshen, Grace Gong
And the first place its usefulness was manifested is in reading and writing, because these models became powerful enough to help users to rewrite sentences or rewrite paragraphs. Back then it would look like science fiction. So based on these models, we were able to create our first application that was called Wordtune, which is basically Grammarly on steroids, right? It's like basically an AI version of Grammarly, and that took off very quickly. We had when we released it back in 2020, after two years, we had tens of millions of users and a lot of paying customers, so was a very growing business. So that was kind of the first step in commercializing that type of technology through helping people to better read and write. So I've been through a long journey, can keep on going probably for many hours, but I think the point is that we started as an AI research lab, we commercialized the technology through an application, and then the kind of the next step in our journey was that we realized we want to kind of own our own destiny. So we started training our own large language models. So back in 2021, we released our first model, which was called Jurassic, the Jurassic model family. It was one of [00:09:24] S0: Jumbo, Maestro.
00:09:26 · Ori Goshen
[00:09:26] S1: So so so Jamba Jamba and Maestro came after, but Jurassic was the first model family that we released, was about a 178,000,000,000 private model back then. It was basically GPT-three, which was 175,000,000,000 parameter and Jurassic that size. So we were pretty early on in training these large models and the models back then were kind of murky. They were very impressive in terms of their what called a few shot capabilities, you would give them few examples, like the task that you want to do, you'd specify it in natural language, and we will follow the pattern and we'll do something interesting like a task you would want to rewrite a sentence or you want to summarize a paragraph, it will the model will understand what you're trying to achieve and given an input, given context, would be able to perform that task. So that was the few shot era, which was again very impressive, but not unlocking all the usefulness that we're seeing in in modern type of models. Then fast forward, back in 2024, we released a model family called Jamba, which stands for Joint Attention and MABA. And it was the first model, hybrid model, commercial hybrid model that combined between mamba architecture and the transformer architecture. That's basically the Bon Ton. And in that paper, we show that you actually can have best of both worlds. You can have the efficiency of the MAMBA architecture with the level of quality and nuance of the transformer architecture if you find the right way to combine them, and nowadays it became pretty industry standard, so if you look at examples like Nemotron from Nvidia and the models by IBM and many of the Chinese labs are training models with this combination of non transformer layers with the classic transformer layers in their neural networks. So you see that this is actually becoming like an industry standard these days. And again, fast forward for beginning of... End of twenty five, beginning of twenty six, we released Maestro, which was an agent orchestration system helping customers, enterprises to build reliable agents and the idea was that this needs to be a model agnostic framework. People can build agents on top of whatever model they want, and you'll be able to scale, test, compute and control a lot of the inner workings of these agentic systems to find the right balance between quality and cost and so on.