· Alexandr Wang
[00:00:00] S1: Did you see this profile that's written on Zuck in "Colossus" written by Jeremy Stern? I saw it. If I'm being honest, it was too long, didn't read, but- It's 15,000 words. I just had Jeremy on the podcast. He's my favorite. Which I saw. I saw the- Yeah. I saw the- Yeah. I think he's writing the best profiles on entrepreneurs and investors in the world right now. I think he's literally the best in the world at what he does. But in that, Zuck says, he was talking about the advantage that he has relative to, I think, Elon and Sam Altman. And he's just like, "Listen, my skill set is very unsexy, but I'm really good at building teams and then just improving a product slowly over a long period of time." And relative to Elon and Sam Altman and others, he's like, "I'm not going to lose control of my company, and I don't need to raise any more money. And so I can just do this for an excessively long period of time." That section of the profile is popping in my mind when you're talking about this seven-month stretch of trying to refine and build Muse. I think that's exactly right. I think that it took immense patience. Seven months maybe doesn't sound like that long, but AI is fierce. Seven months in AI time. Seven months in AI time, which may be a decade in non-AI time. And I think for all of us, but I really give Mark a lot of credit for this, it took a lot of patience and restraint and a lot of trust in the process. And you can go back and look at what Mark said in earnings calls for this entire period from February through September, and he will allude to the fact that we were building personal agents. And we weren't really trying to hide it. We were building- Personal agents, and we were talking about it, but I think there's a lot of doubt in the company for that entire period. Like, people, obviously one of the dominant Wall Street narratives is like, "Oh my gosh, Meta's just burning all this money. Like, are they even going to pull anything off in AI?"
· David Senra
[00:00:00] S0: It's more than that. It's like, "Zuck cannot win AI." It's said over and over again for a long period of time. And again, going back to Jeremy Stern's profile on him, he then, off the record, starts talking to his competitors, including one former researcher that is now at a competitor that used to work with Zuck and is now at another lab. And he goes, "I don't want to have to compete with him because Mark is like the fucking..." I think the line is, "Mark is the fucking terminator." And it was like, he just will not stop. And then he talks about one of his superpowers is that he's never happy with the state of the product, so he always wants constant improvement, but he's not an asshole. Like, he's not a dict- he's definitely a dictator, but he's not rude and will destroy the chemistry of a team. He actually talks about in the piece that team cohesion is excessively important to him.
· Alexandr Wang, David Senra
[00:00:00] S1: I think that's all right. I think one of the magical things about Muse was that we really took quite a bit of time to sweat all the details before ultimately coming out with the product. Because I think that personal agents was one of these product areas that kind of rested on a knife's edge, which is if it can reliably deliver these magical experiences, it has the potential to be one of the greatest consumer products ever. But you're dealing with models, you're dealing with something non-deterministic, you're dealing with agents. They can break, they can be unreliable. Like, if people have these unreliable experiences, they think it's trash. It was one of these things where getting to the product to a point of quality and value such that it resonated with so many of the people who downloaded and used it. [00:00:00] S0: I want to talk more about this process. I'm curious how much resources, how big the team, how many people are working on it. Do you guys have any data on like, "Okay, I'm using Muse, nine great experiences, two inconsistent or shitty experiences, and then I churn out?" Like, do you have any information on this?
· Alexandr Wang, David Senra
[00:00:00] S1: You can see this from a lot of the other agent products at the start of the year. Like, almost everyone churned out of those, and it's because they were unreliable. We did a lot of user testing of Muse. Like, we did many, many rounds of new people trying to download it and use it from scratch, and just seeing what that experience was like. A huge part of the premise of Muse, and I think what's really landed, is that it is a different thing from what most people have experienced with consumer AI. Like, it is not a chatbot, it is an agent, and it can do agent things for you. And I think to a developer, this is kind of like old news because they've had to download agents and have been using them for a while. But for most people, just even that graduation process, it needs to be explained and needs to be onboarded on a very thoughtful way. And then for sure, we definitely have seen when... Because we've done tests with different versions of the model and various model versions and continuously do those A/B tests. The product experience is very, very sensitive to model quality. It's very sensitive to how reliably the models can undergo these tasks, how reliably they communicate with the user about them, how reliably they check back with the user if there's things that they require guidance on. [00:00:00] S0: But how'd you know it was ready to be released?
· David Senra, Alexandr Wang
[00:00:00] S0: But how'd you know it was ready to be released? [00:00:00] S1: Part of it was pretty numerical, which is like we had set... Go back to that spreadsheet of 100 plus things. We had set for each of those rows what threshold is launch blocking and above what threshold is good. We had many checkpoints that were green on 80 of the rows, but red on 20 of the rows. We knew that wasn't good enough. And we had our first model checkpoint, which was green across the board, which was a specially trained version of Muse Spark 1.3. So there was the very numerical part of it, and then there was just the experiential part, which is we tried the model. We could tell that it was a lot better than anything we had iterated with before. [00:00:00] S0: So going back to that seven-month period you were talking about, how many people were working on the product? [00:00:00] S1: Between the model work and the product work, like under 200 people total, which I guess sounds like a lot, but in big company land is- [00:00:00] S0: It's actually a lot smaller.
· David Senra, Alexandr Wang
[00:00:00] S0: It's actually a lot smaller. [00:00:00] S1: It's not very many. It was kind of cool looking back on this and thinking about it is like, we knew this was going to be the most important product that... The whole executive team, everyone knew that this was probably going to be one of the most important products that Meta would ship this year. But we had the restraint organizationally, I think, to make sure that it was a really small, focused team working on it, and that it didn't balloon to this like, "Oh, you have huge swaths of the company working on it," because we knew it was a single work of art. Like, it required strong point of view and strong focus to pull that through into one beautiful experience. [00:00:00] S0: Say more about this. Why did you just call it a single piece of art and that you need a single perspective or point of view?