· David Senra
With Deel enabling us to hire and support exceptional talent anywhere, we can accelerate our innovation and bring more voices, stories, and ideas to every corner of the world." Deel is trusted by over 40,000 companies and growing fast. Learn how they can help your business by going to deel.com/senra. That is deel.com/senra.
· Alexandr Wang
[00:00:00] S1: Did you see this profile that's written on Zuck in "Colossus" written by Jeremy Stern? I saw it. If I'm being honest, it was too long, didn't read, but- It's 15,000 words. I just had Jeremy on the podcast. He's my favorite. Which I saw. I saw the- Yeah. I saw the- Yeah. I think he's writing the best profiles on entrepreneurs and investors in the world right now. I think he's literally the best in the world at what he does. But in that, Zuck says, he was talking about the advantage that he has relative to, I think, Elon and Sam Altman. And he's just like, "Listen, my skill set is very unsexy, but I'm really good at building teams and then just improving a product slowly over a long period of time." And relative to Elon and Sam Altman and others, he's like, "I'm not going to lose control of my company, and I don't need to raise any more money. And so I can just do this for an excessively long period of time." That section of the profile is popping in my mind when you're talking about this seven-month stretch of trying to refine and build Muse. I think that's exactly right. I think that it took immense patience. Seven months maybe doesn't sound like that long, but AI is fierce. Seven months in AI time. Seven months in AI time, which may be a decade in non-AI time. And I think for all of us, but I really give Mark a lot of credit for this, it took a lot of patience and restraint and a lot of trust in the process. And you can go back and look at what Mark said in earnings calls for this entire period from February through September, and he will allude to the fact that we were building personal agents. And we weren't really trying to hide it. We were building- Personal agents, and we were talking about it, but I think there's a lot of doubt in the company for that entire period. Like, people, obviously one of the dominant Wall Street narratives is like, "Oh my gosh, Meta's just burning all this money. Like, are they even going to pull anything off in AI?"
· David Senra
[00:00:00] S0: It's more than that. It's like, "Zuck cannot win AI." It's said over and over again for a long period of time. And again, going back to Jeremy Stern's profile on him, he then, off the record, starts talking to his competitors, including one former researcher that is now at a competitor that used to work with Zuck and is now at another lab. And he goes, "I don't want to have to compete with him because Mark is like the fucking..." I think the line is, "Mark is the fucking terminator." And it was like, he just will not stop. And then he talks about one of his superpowers is that he's never happy with the state of the product, so he always wants constant improvement, but he's not an asshole. Like, he's not a dict- he's definitely a dictator, but he's not rude and will destroy the chemistry of a team. He actually talks about in the piece that team cohesion is excessively important to him.
· Alexandr Wang, David Senra
[00:00:00] S1: I think that's all right. I think one of the magical things about Muse was that we really took quite a bit of time to sweat all the details before ultimately coming out with the product. Because I think that personal agents was one of these product areas that kind of rested on a knife's edge, which is if it can reliably deliver these magical experiences, it has the potential to be one of the greatest consumer products ever. But you're dealing with models, you're dealing with something non-deterministic, you're dealing with agents. They can break, they can be unreliable. Like, if people have these unreliable experiences, they think it's trash. It was one of these things where getting to the product to a point of quality and value such that it resonated with so many of the people who downloaded and used it. [00:00:00] S0: I want to talk more about this process. I'm curious how much resources, how big the team, how many people are working on it. Do you guys have any data on like, "Okay, I'm using Muse, nine great experiences, two inconsistent or shitty experiences, and then I churn out?" Like, do you have any information on this?
· Alexandr Wang, David Senra
[00:00:00] S1: You can see this from a lot of the other agent products at the start of the year. Like, almost everyone churned out of those, and it's because they were unreliable. We did a lot of user testing of Muse. Like, we did many, many rounds of new people trying to download it and use it from scratch, and just seeing what that experience was like. A huge part of the premise of Muse, and I think what's really landed, is that it is a different thing from what most people have experienced with consumer AI. Like, it is not a chatbot, it is an agent, and it can do agent things for you. And I think to a developer, this is kind of like old news because they've had to download agents and have been using them for a while. But for most people, just even that graduation process, it needs to be explained and needs to be onboarded on a very thoughtful way. And then for sure, we definitely have seen when... Because we've done tests with different versions of the model and various model versions and continuously do those A/B tests. The product experience is very, very sensitive to model quality. It's very sensitive to how reliably the models can undergo these tasks, how reliably they communicate with the user about them, how reliably they check back with the user if there's things that they require guidance on. [00:00:00] S0: But how'd you know it was ready to be released?
· David Senra, Alexandr Wang
[00:00:00] S0: But how'd you know it was ready to be released? [00:00:00] S1: Part of it was pretty numerical, which is like we had set... Go back to that spreadsheet of 100 plus things. We had set for each of those rows what threshold is launch blocking and above what threshold is good. We had many checkpoints that were green on 80 of the rows, but red on 20 of the rows. We knew that wasn't good enough. And we had our first model checkpoint, which was green across the board, which was a specially trained version of Muse Spark 1.3. So there was the very numerical part of it, and then there was just the experiential part, which is we tried the model. We could tell that it was a lot better than anything we had iterated with before. [00:00:00] S0: So going back to that seven-month period you were talking about, how many people were working on the product? [00:00:00] S1: Between the model work and the product work, like under 200 people total, which I guess sounds like a lot, but in big company land is- [00:00:00] S0: It's actually a lot smaller.