The Skip Podcast
00:32:55 · Nikhyl Singhal
[00:32:55] S1: Stripe is built for whatever comes next. Your business is moving fast. Your banking should too. Stripe Treasury is a new financial operating system built for what's next. With Stripe Treasury, the place your business accepts payments is also where you can store balances, move funds, and pay vendors worldwide. When you're operating across markets, your money needs to move when and where your business does. Stripe Treasury gives you a finance stack that's automated, intelligent, and always on. Stripe Treasury is business banking on the same platform millions of businesses already use to get paid. Learn more at stripe.com/treasury. Stripe is not a bank. Banking services provided by Fifth Third Bank National Association member FDIC. What's unique about your work at Anthropic is you're probably the one product leader we have in our industry who's built maybe the most sizable and most tenured research product organization out there, just given the time and the size of Anthropic's research team now. And I think a little bit about back in the early days of product management at Google, you know, Marissa sort of made the observation that product managers as defined by the industry wasn't quite cutting it at Google and they wanted more technical folks, what they call the APM or the Associate PM program to teach product management so it would be successful at Google. In many ways, was the template for a lot of what modern product management became. In the last couple of years, everything's kind of flipped upside down. I think a lot of the notes you've made are pretty than what you experienced at a place like the APM program. What I'm interested in is how did you teach what product management needed to look like as you discovered it at Anthropic and make sure that your team was kind of all kind of singing the same tune, knowing that it wasn't traditional. You know, you mentioned, you know, really hard starter projects and being able to sort of weed people out, but there's also a skill development of trying to get people to understand what good looks like. Did you end up doing formal training or mentorship or, you know, did you just work all the time and get coaching advice? How did you end up getting people to understand it?
00:35:13 · Dianne Penn
[00:35:13] S0: It's funny you mentioned the early Google example, because I actually use that in my interviews. So I mentioned it as like product management as how we define it has to come back to like early days of like a new technology rather than this is the exact set of like things that in responsibilities a person has. And I think in particular, the core principles and the core outcomes is still around the user and how do you bring user value in a way that, is compounding compared to the past. And so the actions then fall out of that rather than like, I think my... Our current set of actions that the team does is, I think, still gonna be the same in 2027. But like, I'm constantly questioning to get the best outcome for users. Are we still doing the right thing? Rather than like, when you come on, you will definitely write evals. Maybe evals are no longer going to be the best format in a year to describe users' outcomes, right? Maybe it's actually prototypes. Like everybody builds prototypes. That could be the next thing. But the core outcomes are the north star is still the same. So what we teach might be different. There's a couple of ways we try to onboard PMs. One is having this like pretty rigorous initial onboarding process. You're just drinking from the fire hose. There's understanding users, writing an eval, writing a roadmap. I expect that within the first two to three weeks, which is like pretty fast. Usually folks are like, wait, can I have like two more weeks to do that? And it's more of like you're bringing the like velocity and the hustle, you're just getting them into like, I gotta go like at the pace of the company? I think other things we try to do, my team actually hosts a annual bootcamp teaching initially other product managers outside of the research PM team, but now we've opened it to all of Anthropic. And this is around just what we've seen in terms of how do you make interesting products that what we call ride the exponential, meaning they get better as models get better. Like they get more valuable as models get better. How do you improve Claude? How you do things like prompting and writing good emails? So we have like an annual bootcamp and we share the videos for that for every new hire. And then I think it's really around being close to an area that is a p zero.
00:37:41 · Dianne Penn
Like, I lean more on having actually smaller teams, but giving people space such that, like, they have maybe two to three areas that they're looking after. And so as part of that, like they're naturally stretched to go talk to different people to then come up with new ideas, like read up on what's happening. And so whatever they're working on is like very high priority and therefore they have more like urgency to like deliver and just bring them on pace, getting stuff done like that, I think has been important. We also have a spin up buddy for everybody on the team. So folks who are more tenured on the team always is a buddy for somebody new. We have a lot of products like Claude Tag, which helps folks with onboarding. And I think the culture of working in public and like public channels, part of it is just like I tell folks like, go back, scroll back two months, read what's happened because you're kind of joining a conversation when you're joining a company. So when everybody's talking, it's okay to say like, hey, can everybody stop and tell me what's going on? It's better to do a bit of your homework and then be like, hey, I understand this seems to be X, Y, and Z. Here's what I think. What do you all think? Right? So it's a bit of coming to the company rather than waiting for it like Anthropic to come to you as a new hire. But those tend to be some of like the initial ways of working to help people onboard.
00:39:13 · Nikhyl Singhal
[00:39:13] S1: Tell me more about the bootcamp. And the part of the reason I'm asking is that a lot of the things you mentioned are replicable in someone else's company as well. And I'm wondering this bootcamp, is it how structured is it? How much of it is like coursework versus skilled versus case studies? And it might, you know, I'm sure it's quite popular. So I would love to know how it sort of evolved over the years.
00:39:38 · Dianne Penn
[00:39:38] S0: It's very grassroots y. We thought at first we were starting in 2024 when we did the first year, myself and the team at the time was pretty small team. There was four or five of us. We thought like what we did day to day as product managers were starting to diverge as research PMs compared to other PMs. But then some of the things we were starting to do, you know, reading user consented feedback data or writing evals, those seem to be things that help our features too. So like, shouldn't more people do that? And so we just said, okay, let's write down how we do them. What like, what we look for, what we don't think is as valuable and see if people would, other parts of the team would want to come and join and listen. And so like a small group first did. And I do think a lot of it is replicable. Sometimes it's actually more to make it an open forum for people to ask questions because there's one piece about here's ways to improve Claude. But actually it's not that you have to just wait for the model. You could also prompt the model to get to the outcome you want as a PM. Like you have the power to go try to prompt, like to change the instructions And just doing it live reduces and demystifies it in a way where, you know, even our sales team would come, and they're like, oh, that's what an eval is. That's what prompting is. And so like, okay, I can do that. Like anyone, anyone can go, go ahead and learn. So it's more of like making it accessible. And I think it is something that like any company can, can do. Like there is enough, like knowledge is great, like things on the web. A lot of people have written about things like emails and prompting, obviously at this point. So I think they're all, they're all things that can be replicated and shared more scalably.
00:41:30 · Nikhyl Singhal
[00:41:30] S1: One area that I think we've talked a lot about is the challenge that large tech product managers face because they're in very structured environments that are moving information from one layer to another, and you're describing a pretty different job. I think a lot of other product managers listening to this, you know, work in, you know, SaaS or enterprise settings where there's direct user feedback that directly implements what the roadmap is. And, you know, there's a thoughtful question that came in through the live channel, which was around how do you manage the balance between research priorities and user feedback? And this is a conversation I had when I was talking to Tulsee, who's doing the sort of Gemini lab model as well, is that it's not quite the same thing as being able to say user wants X, telling the researcher, this is what we need to do next. Tell us a little bit about that finesse because that's a big part of the distinction between a traditional PM job and a research PM role.