Invest Like the Best with Patrick O'Shaughnessy
· Gokul Rajaram, Patrick O'Shaughnessy
So the PM's job is to be very clear at a high level about what the user needs are, and then have a very clear sense of whether this product is good to ship or not by evaluating it. Many times you've got to write AI yourself to evaluate the results of AI, because humans can't. So PMs are good at coming up with evaluation techniques. It's the non-determinism of software, the speed of which things are going, and overall, the notion that the capability frontier is being pushed out every two months makes it an incredibly challenging, yet incredibly exciting time to be in product development. [00:00:00] S0: If you think about, my friend Zach has this great way of thinking about AI, which is, we had the industrial revolution for goods, and that basically this kicks off an industrial revolution for services. This is an interesting opportunity to ask about what your philosophy of product is. You're such a product-centric person and builder. That's what you've done. That's what you've invested in. As we face down this industrial revolution for services, what is your broadest possible philosophy of product as we enter this era?
· Gokul Rajaram
[00:00:00] S1: Very simple. A product person, or product manager if you call them, their job is to balance customer needs and business needs. The product manager, there has to be somebody at the company who's the keeper of the why. Why are we building it? What customer need are we solving? Why is this a pain point? How intense is it and how deep is it? And second, how does it add value to the company? If you build this thing, solving this customer need, how does the value add to the company? And I think balancing those two is a very delicate act. You can build something amazing that adds a tremendous amount of value to the customer, but doesn't build any value to the business. And you can do something that is awesome for the business by raising prices, but it is value detracting for the customer. So balancing customer needs and business needs at the highest level is what I think of the product. And what it comes down to, in my opinion, over the last 10 or 15 years, I've really gone down to this notion of outcomes. Outcomes, I think, are what define the best product people, and outcomes have to be defined in the form of customer behavior. Because customer behaviors are leading indicators for every business outcome. If you think about it, the simplest thing that a product does is to make somebody go from not a customer state to becoming a customer state. And from becoming a customer state to becoming a loyal customer. And then maybe to become a loyal customer to become a paying customer. Or if you do a poor job, you can go from becoming a loyal customer to becoming a churn customer. So these are all behaviors. Everything you do or build should be attuned to the goal of, what customer state change does it lead to? What customer behavior change does it lead to? So, I tell every CEO I meet that is trying to hire their first PM or doing their first product review; you need to ask why. The only question you need to ask is why. Why are you launching this feature? And you should not let any feature go out if there's not a clear hypothesis behind this feature. And the hypothesis has to be articulated in the form of a customer behavior change. We believe that by launching this thing, the customers will go from doing X to doing Y, or from spending X minutes a month doing this to Y minutes a month doing this.
· Gokul Rajaram, Patrick O'Shaughnessy
You have to have a hypothesis, which is grounded in some data or something you know about the customer, some secret about the customer. [00:00:00] S0: You mentioned at the start, the difference between the video transcriptions tool six months ago versus more recently and how quickly that changed. It's just such a hard future to reason about given the pace of change. So how do you reason about it? Is there anything that can be truly future-proof?
· Gokul Rajaram
[00:00:00] S1: Yes. The one thing I think that's going to be truly future-proof is judgment. Why? Because what is the biggest challenge you have when you have a thousand AI engineers writing code? You have the big challenge of AI slop. Every product director I’ve talked to is extremely worried that because you have these engineers running rampant, they're just going to produce lots of code. Which of this code is even valuable? In an era when you can do everything, the question is which of these things matter and you should truly do. On the product side is judgment around what needs to be built and evaluating the output. On the engineer side is evaluating the code, because if you don't understand what the code says, I think you can have AI engineers writing beautiful code that could be wrong, that could have bugs in it, that could be vulnerable. Someone needs to review it and make sure – you have to have human review at some point, especially the critical code that is in the core of your system. And similarly in design, you have to have judgment around, does it make sense in the broader design system? So I think this judgment is the number one thing that humans are going to bring. Era of infinite productivity, the question is what are the things we're productive on, and are we building the right things?
· Patrick O'Shaughnessy
[00:00:00] S0: As you evaluate companies today, build things yourself and just think about the trajectory of these tools. Maybe walk through how someone should think about building an AI application. There's so many people excited about… it feels like a gold rush with this new technology. So many things that we can do that we couldn't do before or things that specific people couldn't do because they weren't technical that they can now do. How should people think about building an application using AI starting today?
· Gokul Rajaram
[00:00:00] S1: First and foremost, you've got to start with a deep and compelling problem. The good news is, there's a tremendous number of deep and compelling problems today, in every vertical and every industry. Why? Because till today, till recently, software was used more as a tool by humans. We finally have software that is agentic in nature, which means it can do the job of people. The question you have to ask is, what industry are there roles of people that are highly paid, that are doing somewhat of a repetitive job, and that can be done by software? Every three months, the answer gets deeper and deeper. You couldn't have told me that a designer's job could be automated by AI six months or nine months ago. You couldn't have told me that an architect's job could be automated by AI. A lawyer's job could be automated by AI. It turns out, increasingly in every vertical, these capabilities are getting better and better. So you want to start with, what industry do you want to be in, and what kind of job do you want to do? Second, you want to target a high-value workflow. You want to target a workflow that is deep, that is complex, and that requires custom data. I think one of the challenges with this whole space is that the models are becoming so good that, if you try to build a company that is light, that is not a hard problem, the foundation model companies are going to eat you. I met with the CIO of a Fortune 500 company a few weeks ago. I was asking him about a few startups I had invested in and worked with. He said, "Look, I don't know why I would use any of these startups. Gemini has an agent builder product, and I also use ChatGPT Enterprise, and they also have an agent builder product. And I have a thousand IT engineers who work for me. They all want to be retrained as AI engineers. So I'm just going to put them using these horizontal tools to build my AI agents. Why do you need any startups?" And so that's the kind of thing you're going to face, that if the CIO of a company, of your target customer, can build what you're building using these agent building tools, you're not going to be successful. So you've got to really go one step ahead of what can be built. Multiple steps ahead.