The Pragmatic Engineer
00:58:03 · Titus Winters
[00:58:03] S1: I mean, it's stressful. Like, I've had experiences of there being layoffs in the area, and just... It... It'll be weeks or months of every one on one starts with, so how are you doing? Like, how are you coping? Like, there's this huge amount of, you know, negative performance influence of like, okay. We gotta, like, emotionally support each other first before we actually even get back to doing the work. And I'm very skeptical that in the executive accounting on all of this, that they're taking into consideration just the sheer amount of lost productivity and lost time from like, yeah, people are stressed. People are spending time brushing up their resume in case they're next on the chopping block. People are spending that one on one time, like, in therapy instead of in planning. It's... It is a huge hidden cost to all of that, and it's really demotivating. It drives home... Like, it is a reminder that, okay, this is just a business obligation. This is not a lifestyle choice. And if me putting in extra doesn't give me extra safety on this, then what's the point? You know? And, like, it... There's a lot of cultural baggage that happens for that. And it's one of the things that I actually really liked about Adobe was Adobe's had the same CEO for a long time. He just stepped down. He has a replacement now. But he was there for the great recession crash. And they did some layoffs at that era, and he saw how much it hurt morale and company productivity and all of the rest of that. And he learned, like, it... We thought it made numeric sense at the time, but it was not good for the soul of the company, and that's not the company that we wanna be. And so Adobe didn't have the the layoffs in the last few years. Like, there's group by group, like, local things as things get reprioritized, but it's not the sweeping all of those. Yeah. But we...
01:00:11 · Gergely Orosz, Titus Winters
[01:00:11] S0: We've seen that. Most of Big Tech has had it, especially in 2023. Yeah. And speaking of Adobe, so you've moved to Adobe a few years ago. What are you focused on at Adobe? [01:00:22] S1: So there, I'm very senior IC focused on developer experience. I happen to say this is a polite fiction. Developer experience is the business of productivity. We just know enough not to say that we can measure productivity. It does not give [01:00:37] S0: everyone Unlike McKinsey who me and Kendra have bonded over. [01:00:41] S1: Oh. Yeah. Yes. Like, that was... That's horrible. [01:00:45] S0: But but we will link it in show notes below. [01:00:47] S1: Yeah. So it's it's what do we need for, you know, tools and culture and, you know, all of the things to improve the experience of being a developer and make you more productive and efficient because most companies spend more on engineering comp than anything else. [01:01:04] S0: So today, you've been focused on this thinking about it and working on it for both at Adobe but also to some extent at Google. What do you know today about developer productivity? How does this relate to AI, coding tool, output, metrics, all of that things? What is it that we can measure? What is it worth measuring? And what are things that are hard to know?
01:01:26 · Titus Winters, Gergely Orosz
[01:01:26] S1: Yeah. I mean, we've taken it as axiomatic that it is hard, if not impossible, to measure productivity because what are the units even? I had, a while back, a sort of clever idea, and I use that as a loaded term. I had this idea that if we don't know what the units are, like economics has prior art for this, right, you have an index metric where you measure all of the things that might influence this, and you come up with a way to effectively basket those together. And then you have a unitless, but directionally useful and useful over time, number that you can sort of pin to. And my theory had been, so measure all of the things that we think are positive or negative indicators of something is going well. Right? Releases, test coverage, number of PRs, right, like everything. Right? Just cast a wide net. My initial formulation on this was, you know, the wisdom of the crowds idea. Right? Like, no individual can estimate how many jelly beans are in the jar, but you get a 100 people and you'll get real close. So we do a, like, am I hot or not? Of like, which would you prefer? Like, one release per week more or x tests. Right? And you run a bunch of iterations of that game with your, you know, relatively experienced engineers Yeah. To work out the linear trade offs between all of those factors. [01:03:01] S0: To figure out what matters more. Right.
01:03:01 · Gergely Orosz, Titus Winters
[01:03:01] S0: To figure out what matters more. Right. [01:03:03] S1: To figure out what that is. You probably wanna dedupe some things, because they're just gonna be the same stat in a different package. Right? But find the ones that have some nonlinearity, and then mush that all together. Right? Now you have something that sort of gives you an aggregate. Like, this is kind of our productivity number. And Adobe is doing something along these lines. Slightly smarter than my initial formulation. But like, then you can watch that track up or down and, you know, over time for a given population. What I've realized is that won't capture everything in this post AI boom. And the reason that I came to that was a paper that I put out, the develop deploy operate paper Mhmm. Recently, where one of the points in there is there's four forms of impact in tech. There's product success, the stuff everyone talks about and everyone else is better at than I am. Like, that's that's not my job. [01:04:06] S0: Will people buy a product or or use a product, whatever. Yeah. [01:04:08] S1: Net promoter score revenue, like all of the stuff. Right? There's compute efficiency. Right? Like, how much are you blowing on AWS? [01:04:16] S0: Oh, yeah. Right? Or if you I'll ship send shares [01:04:18] S1: and all that. [01:04:18] S0: All too well. Yeah. [01:04:19] S1: Right. You know? Or, like, even like the Android team has a trade off for if you can reduce the size of an application download by [01:04:28] S0: a It makes a difference for
01:04:28 · Gergely Orosz, Titus Winters
[01:04:28] S0: a It makes a difference for [01:04:31] S1: many metrics. Yes. Right. You know? And so like all of those things, like that is the form of impact. Then there's engineering productivity, or we phrased it as engineering capacity. Because if we save you, you know, 5% productivity gains, capacity gains, you don't get to spend that somewhere else. You just have more capacity on doing more engineering. [01:04:52] S0: Or having slack time for innovation or picking up when something happens, that kind of stuff.
01:04:52 · Gergely Orosz, Titus Winters
[01:04:52] S0: Or having slack time for innovation or picking up when something happens, that kind of stuff. [01:04:58] S1: Right. But they're not like dollar fungible. Exactly. You don't get money in a different account. And then the fourth form of impact, and this ties back to the AI thing. Fourth form of impact is changing the game. Right? Strategic innovation. Like, research makes impossible things possible. Oh, yeah. Security makes incredibly bad things impossible. Right? But if you change the nature of what we are doing, the cost structure, etcetera, then that's a different game. And the problem that I think that we see with AI, even with a clever you know, index metric approach to trying to capture the gestalt of productivity, if you're not looking for the value of a given change is substantially different, then there's no way you'll ever be able to capture that in the signal. Right? And we don't have anything that says the value of a change of this size has changed. So for instance, it is entirely possible at this point, and I hang out in security spaces and do a lot of low level c plus plus library stuff, you can write a JPEG parser in a couple weeks. Yep. Right? With whatever feat... Additional features you need. That would be insane to say that word... That sentence out loud three years ago.